Semantic Data Monitors

Semantic Data Monitors play a critical role in ensuring data quality by validating the semantics of the data. They are used to check the consistency and validity of specific fields in the data, based on the collection schema.

Data Rule: Field

The ‘field’ rule allows you to monitor specific fields in the data. You can check for several conditions, such as if the field contains a particular string, if it is empty, if it equals to a certain value, if it’s a boolean true, or if it matches patterns like credit card numbers or social security numbers.

Semantic Data Monitor Types

String Contains

This type of monitor can be used to verify if a field contains a certain string. This is helpful for text validation and can be used to check if a certain keyword or phrase is present in the text.

Is Empty

This monitor checks if a field is empty. This is crucial for ensuring that mandatory fields in the data are always populated.

Equal To

This monitor can be used to check if a field is equal to a specific value. This is useful for data validation and ensuring data consistency.

Boolean True

This monitor checks if a field is boolean true. This can be used to validate boolean fields in the data.

PII Field Values

This monitor can validate whether a field matches patterns such as credit card numbers or social security numbers. This is crucial for data privacy and compliance with regulations like GDPR.

Semantic Data Monitor Configuration

  1. Navigate to Monitor in the console, and click the + button to create a new monitor.
  2. Name the monitor.
  3. Attach an alert to the monitor.
  4. Select the Rule Field.
  5. Specify the Field in the collection schema that you wish to monitor.
  6. Select the condition to check for the field - ‘String Contains’, ‘Is Empty’, ‘Equal To’, ‘Boolean True’, or ‘PII Field Values’.
semantic_data_monitor