Implement and Manage Data Quality Constraints in Unity Catalog DP-750 Practice Questions (Page 3)

Questions 11–15

1. 11foundation · easy

   What is schema drift management in the context of data pipelines?

   • AThe process of compressing data to reduce storage footprint
   • BThe process of adapting to changes in the structure of incoming data over time
   • CThe process of encrypting data to protect it from unauthorized access
   • DThe process of enforcing a strict schema and rejecting any data that does not match
   Reveal Solution Discussion Select an answer first
2. 12expert · hard

   A financial company has a `loans` table with `loan_amount` and `interest_rate` columns. They want to enforce that `loan_amount` is between 1000 and 100000, `interest_rate` is between 0 and 20, and `loan_id` is not null. The team is using a Lakeflow pipeline and wants to fail the pipeline if any batch violates these rules. However, they also want to avoid the overhead of checking every row individually. What is the best approach?

   • AUse a `FILTER` transformation to remove rows that violate the rules and write only valid rows.
   • BSet the table property `delta.dataSkippingNumIndexedCols` to 3 to speed up validation.
   • CDefine an expectation in the pipeline that checks `min(loan_amount) >= 1000 AND max(loan_amount) <= 100000 AND min(interest_rate) >= 0 AND max(interest_rate) <= 20 AND loan_id IS NOT NULL`, with `onViolation = 'fail'`.
   • DAdd a CHECK constraint on the table for all three conditions.
   Reveal Solution Discussion Select an answer first
4. 13application · medium

   A marketing analytics team receives event data from a mobile app. The `event_timestamp` field is sometimes sent as a string in different formats, and the team wants to standardize it to a timestamp type in the target table. They also want to handle the case where the source adds a new `campaign_id` column in the future without breaking the pipeline. What should they do?

   • ASet the table property `delta.columnMapping.mode` to 'name' and rely on it to handle type changes.
   • BUse a Lakeflow pipeline with a transformation that casts `event_timestamp` to timestamp and enable schema evolution to automatically add new columns.
   • CUse a CHECK constraint on `event_timestamp` to enforce it is a timestamp.
   • DUse a generated column to convert `event_timestamp` to a timestamp and ignore any new columns.
   Reveal Solution Discussion Select an answer first
5. 14application · medium

   A logistics company tracks shipment weights in a Unity Catalog table. The `weight_kg` column must be between 0.1 and 2000. The data engineering team uses Lakeflow Spark Declarative Pipelines for ingestion. They want to define the range check as a reusable expectation that can be applied to multiple tables. What is the most efficient way to implement this?

   • ADefine a CHECK constraint on each table with `weight_kg BETWEEN 0.1 AND 2000`.
   • BCreate a custom expectation in the Lakeflow pipeline using `expect` with `weight_kg BETWEEN 0.1 AND 2000` and apply it to all tables.
   • CUse a generated column that calculates a boolean flag for the range and filter on that flag.
   • DAdd a validation step in the ingestion job that filters out rows where `weight_kg` is outside the range.
   Reveal Solution Discussion Select an answer first
6. 15expert · hard

   A data engineering team is building a Lakeflow pipeline to process daily sales data. They have three requirements: (1) `order_id` must be unique, (2) `amount` must be between 0 and 10000, and (3) `customer_id` must not be null. The team wants to fail the pipeline if any batch violates these rules, but they also want to avoid the cost of a full table scan on every run. What is the best way to implement this?

   • ADefine a single expectation that checks `count(*) = count(distinct order_id) AND amount BETWEEN 0 AND 10000 AND customer_id IS NOT NULL`, with `onViolation = 'fail'`.
   • BAdd a PRIMARY KEY constraint on `order_id` and a CHECK constraint for `amount` and `customer_id`.
   • CSet the table property `delta.dataSkippingNumIndexedCols` to 3 to speed up validation.
   • DUse a `DISTINCT` transformation to remove duplicate `order_id` rows and filter out invalid `amount` and `customer_id` values.
   Reveal Solution Discussion Select an answer first