Document schema change monitors for Data Observability - #39122
Open
kevinzenghu wants to merge 10 commits into
Open
Document schema change monitors for Data Observability#39122kevinzenghu wants to merge 10 commits into
kevinzenghu wants to merge 10 commits into
Conversation
Adds a Source-to-Target monitors section to the Data Observability monitor page, covering creation steps, comparison types, custom SQL usage, and evaluation behavior. Also adds an example monitor and a pointer from the overview.
Uses the labels shown in the creation flow: the monitor type is Source to Target, source and target are chosen in separate steps, and the comparison control is Format. Also notes the Preview Monitor Evaluation panel.
The Format options are already described in the creation steps.
- Sentence-case the section headings, matching the rest of the page - Use "% Difference" instead of "Percent Difference" - Number the creation steps explicitly instead of repeating "1."
- Add concrete use cases: replication, reconciliation, migration cutover, and transformation checks - Name the supported metric types - State region availability - Explain that the difference is recorded as its own metric, so the standard detection methods apply
- Difference is the target value minus the source value, not an absolute value, so note that a negative result means the target has less than the source - Note that % Difference is relative to the source value - Remove the evaluation-skip conditions
- Add a monitor types table to the overview, since the page previously described only the data quality type while the creation flow offers four - Add a schema change monitors section covering change types, creation steps, supported warehouses, and the schema changes view - Match the detection method labels to the UI (Anomalies, Thresholds)
Source to target monitors landed on master. All conflicts were the same shape: this branch already contained that content plus the schema change additions, so the branch version was kept throughout.
Contributor
Preview links (active after the
|
jhgilbert
added
the
astro-reorg-no-conflicts
…ule options - Schema change monitors do not require requesting access, so the Preview note no longer points to support - Link the schema change creation flow and the Schema Changes view - Document the full monitor schedule options, including custom cron schedules down to every 15 minutes and the run-time preview - Add a model configuration section for anomaly monitors
Contributor
|
Created DOCS-15392 for Docs team review |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do? What is the motivation?
Documents schema change monitors, which were not covered anywhere in
monitors/types/, and corrects several details on the Data Observability monitor page.Changes:
Schema change monitorssection: the four change types (added, removed, renamed, type changed) across databases, schemas, tables, and columns; creation steps; supported warehouses; and the Schema Changes view for browsing detections without creating a monitor.Monitor schedulerewritten. It documented onlyHourly,Daily, andManual. The flow offers aScheduledandManualtoggle, withHourly,Every 3 hours,Every 6 hours,Every 12 hours,Daily, andCustom schedulebeneath it. Custom schedules take a cron expression and can run as often as every 15 minutes, with a preview of upcoming run times.Model configurationsection covering the settings available to anomaly monitors, and the behavior when data is missing for an evaluation.Anomaly/ThresholdtoAnomalies/Thresholdsto match the UI.Merge readiness
For Datadog employees:
<name>/<description>convention and include the forward slash (/). If you've already created your PR with an incorrect branch name, please rename your branch and open a fresh PR./reviewto run an automated check that catches common issues before a Documentation team member reviews your PR.AI assistance
Drafted with Claude Code from the existing page structure, the monitor creation flow, the schedule and model configuration controls, and the Schema Changes view.
One thing worth a check: supported warehouses for schema change detection are listed as Snowflake, BigQuery, Databricks, and Redshift.
Additional notes
Schema change monitors are in Preview but do not require requesting access, so the Preview note states the phase without pointing to support, matching the pattern used on the AWS Glue and Azure Data Factory pages.
One follow-up this PR does not attempt: Job monitors are named in the new table but are not documented anywhere, and likely deserve their own section or page.
A screenshot of the monitor type selector would suit the new Overview table, and one of the Schema Changes view would suit that section.