Acceldata Acquires AI Leader Bewgle to Deepen Data Observability Capabilities for AI | Learn More

Acceldata Data Observability Cloud v2.7.0 Introduces Databricks and Snowflake Observability Enhancements

Acceldata Data Observability Cloud v2.7.0 Introduces Significant Databricks and Snowflake Observability Enhancements and Supports New Data Sources

Recently, we released the latest version of Acceldata Data Observability Cloud (ADOC). Version 2.7.0 is now generally available, and it provides enhancements to compute, data reliability, monitoring, alerting, and Databricks and Snowflake navigation and display capabilities.

The following video offers a snapshot of the enhancements, and more details on new capabilities and features are explained below:

Data Source and Data Plane

Check out Acceldata’s Hive integration and new data sources:

  • Enhanced Hive Integration: ADOC has been upgraded to seamlessly integrate with Hive, specifically the Enterprise version. Detailed information on this integration can be found in the Hive documentation.
  • Expanded Data Sources: ADOC now offers integration with additional data sources, including Teradata, Azure MSSQL, and Oracle. To learn more about utilizing these integrations, refer to the documentation for Teradata, Azure MSSQL, and Oracle.

Compute Enhancements

In the 2.7.0 release, you’ll see a variety of major enhancements that will provide better experiences for Databricks Query Studio, Snowflake Query Studio, cost center and cloud cost control features, warehouse sizing recommendations, and others.

  • Databricks Query Studio: ADOC now offers a Databricks Query Studio experience, akin to Snowflake Query Studio. This exciting addition enables you to validate and monitor SQL queries executed by Spark apps within a notebook. For detailed insights, please refer to Acceldata’s Databricks Query Studio documentation.
  • Enhanced Databricks Guardrails: In the Databricks Compute tab, a new Guardrail page has been introduced, presenting visualizations for clusters that have been terminated due to inactivity. To explore this feature further, consult Acceldata’s Databricks Jobs Visualizations documentation.
  • Advanced Cost Center Management with Tag Pairs: ADOC now offers the ability to create chargeback-based cost centers using Tag Pairs. This feature enables you to accurately track charges for the entire account or specific subsets of resources by utilizing multiple key values. Check out our documentation about Cost Centers and Organization Units.
  • Streamlined Databricks Navigation: ADOC's Databricks navigation now facilitates efficient correlation of data across jobs and clusters. For details, consult the Databricks Visualizations documentation.
  • Enhanced Snowflake Query Studio: The Snowflake Query Studio now displays metrics like #Avg Schedule Frequency, warehouse names on hover, easier navigation to the Query Details page, and additional information in the heatmap chart, among other enhancements. 
  • Projected Spend Budget Alerts: ADOC introduces the capability to receive alerts when your projected spend, calculated from current utilization, surpasses a predefined budget threshold. Stay informed about your forecasted expenses by exploring the Budgeting documentation.
  • Snowflake Warehouse Size Selection: You’ll now have the option to choose a Snowflake warehouse size based on intelligent recommendations which streamlines your decision-making process.

Data Reliability

We’ve made significant enhancements to the data reliability capabilities of the Acceldata platform. These include better data cadence support, importing/exporting policies, service monitoring, policy execution, and others.

  • UDT Repo Import Export Capability: ADOC introduces a streamlined flow for importing or exporting policies that utilize external User-Defined Types (UDTs). 
  • Customizable UDF Metadata for Failed Rows: ADOC now enables you to customize the feedback logic to UDT via metadata for various checks, so you can receive detailed feedback for failed records pertaining to Data Quality Policies. Check out User Defined Templates documentation.
  • Expanded Data Cadence Support: ADOC now offers data cadence functionality for Files and has extended its support to include MySQL, PostgreSQL, BigQuery, S3, ADLS, Azure Data Lake, GCS, and HDFS as data sources. Get insights into configuring data cadence by referring to the Data Cadence Tab documentation.
  • Composite Policy Application: You’ll now be able to apply individual rules to specific subsets of data within an asset through the Composite Policy feature.
  • Selective Policy Execution: ADOC now grants you the ability to selectively execute policies on specific subsets of rules or data. Explore Acceldata’s Executing Policies documentation.
  • Enhanced Service Monitoring in Dataplanes: ADOC now provides real-time visibility into the ongoing logs of the analysis service executed on the dataplane side. Additionally, you can view the running logs of crawlers on data sources. See our Managing Data Plane documentation for details.

Monitors and Alerts

This version has enhanced monitoring functionality, and new recommendation and alerting capabilities.

  • Expanded Notification Channels: ADOC now provides support for configuring Microsoft Teams and ServiceNow as notification channels, offering more options for seamless communication. See Acceldata’s Notification Channels documentation.
  • Databricks Workspace Cost Monitor: In this release, we’ve provided Databricks Workspace Cost monitor, which tracks the cost at the workspace level for the current month. Our Stock Monitors documentation has details.
  • Enhanced Recommendations: ADOC now allows you to enable or disable recommendations directly from the Monitors interface, providing greater control over the recommendations displayed. Additionally, recommendations will automatically expire when they are no longer relevant. Here’s our Monitors List documentation.
  • Alerts for Assignee Action: ADOC now includes a new email notification template that ensures users assigned to an alert receive timely email notifications. Check out our Alerts documentation.

Learn more with a tour of the Acceldata Data Observability Cloud.

Photo by Vincentiu Solomon on Unsplash

Subscribe to our monthly newsletter

Subscribe to get tips, news, updates, and best practices.