The complexity of large-scale “information technology (IT) plants” run by most enterprises has been increasing rapidly and is fast approaching a barrier. We believe that continuous evolution is a key contributor to this complexity. IT plants evolve to accommodate new software functionalities, hardware technology, application and user requirements, as well as changes in operating conditions (workload, faults, etc.). Today, evolving IT plants in a timely manner while maintaining desired levels of performance, stability, and security is an art; system evolution tasks are manual and intuition-based. At SRL, we are pursuing a long-term research agenda to conquer the complexity in IT plants through managed evolution.

Our approach involves two key steps: (1) Understand the “as-is” state of an operational IT plant proactively to determine the impact of change as well as to identify opportunities to simplify and transform the plant; and (2) derive and execute a simplify-and-transform strategy using emerging technologies. These two steps are instantiated in the TCS Sense-Understand-Respond (SURe) framework.

To understand the as-is state of an IT plant, the SURe framework collects data from a wide-range of “monitoring probes” placed at various locations and tiers of an IT plant and an organization, integrates and analyzes the data, and derives a variety of operational models (e.g., workload models, dependency models, request flow models, performance models, causality models, etc.). These models are then used to conduct “what if” analysis and predict the impact of change, and thereby simplify many tasks such as capacity planning, performance engineering, system testing, and real-time decision making (e.g., anomaly detection, alert generation, and root cause analysis). The foundation of SURe is analytics; the framework analyzes only two types of data – graphs (relations) and event streams (time-series). Once the as-is state analyses identifies simplification and transformation opportunities, the “respond” part of the SURe framework facilitates the design of a transformation strategy. The design of the transformation strategy leverages emerging technologies such as server, OS, network, storage and application virtualization; application streaming; infrastructure monitoring; dynamic resource allocations; heat and power management; among others.

In the past year, the SURe framework has been successfully applied in the context of several large TCS clients. However, many of the sense-understand-respond tasks were performed manually. The goal for 2008-09 is to create a repeatable solution framework that could automate several of these steps, and thereby eliminate dependencies on manual effort and intuition as well as on a highly trained and specialized team.