To make data an asset, enterprises need data observability.
This emerging discipline includes data quality observability, which studies the accuracy and timeliness of data in flight or at rest, and data pipeline observability, which studies the performance of data pipelines as well as the infrastructure that support them.
Data observability programs and solutions should address these ten use cases across four categories:
- Prepare. Infrastructure design, capacity planning, and pipeline design.
- Operate. Performance tuning, data quality, and data drift.
- Adjust. Resource optimization, storage tiering, and migrations.
- Fund. Financial operations (FinOps).
This report from the Eckerson Group defines data observability, including its challenges and benefits. Then we explore use cases for preparing, operating, and adjusting data environments, as well as managing the business aspects of analytics projects and applications.
About the Eckerson Group:
Eckerson Group is a research and consulting firm that helps business leaders use data and technology to drive better insights and actions.
It is the industry’s go-to provider for research, data strategy, data architecture, data science, data warehousing, and data governance.