Most enterprises today are swimming in huge volumes of (structured) data. This data is generated by several external and internal sources, enriched, and disseminated (often in real-time) to several consumers. It is also stored in a large number of databases for future access. For instance, a top-tier financial service providing company runs more than 30,000 databases in its datacenters. A global investment bank receives over 100 million “market events” each day that are enriched and propagated through its trade plant in real-time, as well as stored in databases for historical trend analysis. We refer to the entire infrastructure for collecting, disseminating, and storing data in enterprises as a “data plant”.

Enterprises face a plethora of problems while dealing with their data plants. These include:

• Managing capacity of data plants in the presence of continuous increase in data volumes and workload. Ensuring that the data plant offers predictability guarantees (in terms of delays and throughput) to its consumers.

• Estimating the impact of any alterations in data plant architecture on applications.
  Reducing the complexity of data plants by reducing the diversity of database platforms. The factors govern such a consolidation strategy and deriving and executing an efficient migration plan.

• Addressing the challenges posed by globalization (that leads an enterprise to move from an 8x5 mode of operation to nearly a 24x7 mode of operation) and the corresponding shrinkage in DB maintenance windows.

• Efficiently creating “test data plants” (generally much smaller than production data plants) for testing applications such that:
  (1) The data in the test environment is representative of the production environment (in terms of data quality, diversity, etc.),
  (2) It is possible to estimate the performance of an application in the production environment based on what is observed in the test environment.

• Deriving insights about enterprise operations based on operational metrics stored in isolated databases. Addressing the challenges posed by lack of data integrity and consistency across different databases.