Design and Implement Data Pipelines DP-750 Practice Questions (Page 4)

Questions 16–20

1. 16application · medium

   A team is building a notebook-based pipeline in Azure Databricks. The pipeline has three tasks: ingest data, clean data, and train a model. The model training task must only run if the cleaning task succeeds. The team also wants to send an alert if any task fails. What should they configure in the job?

   • AUse a single notebook that runs all steps sequentially and add an alert on notebook failure.
   • BSet the model training task to depend on the cleaning task, and add an alert on job failure.
   • CRun all tasks in parallel and add an alert on job failure.
   • DSet the cleaning task to depend on the model training task, and add an alert on task failure.
   Reveal Solution Discussion Select an answer first
2. 17expert · hard

   A data pipeline processes a large dataset and performs a join between two tables. The team notices that the join is very slow and wants to improve performance. They also need to ensure that the pipeline can handle occasional data quality issues without failing. Which approach should they take?

   • AJoin first, then filter, and use a try-except block to handle errors.
   • BUse a broadcast join hint and ignore any errors.
   • CFilter both tables before the join, and use a 'badRecordsPath' when reading the data.
   • DRepartition the data before the join and rely on the job scheduler to retry on failure.
   Reveal Solution Discussion Select an answer first
4. 18expert · hard

   A team is designing a Lakeflow job that processes financial transactions. The job has three tasks: ingest, validate, and aggregate. The validate task checks for duplicate transactions and flags anomalies. The aggregate task computes daily totals. The team wants to ensure that if the validate task finds anomalies, the aggregate task still runs but the results are marked as 'review required'. If the validate task fails due to a system error, the job should stop. How should the team design the task logic?

   • AConfigure the validate task to fail when anomalies are found, and set the aggregate task to run even if the validate task fails.
   • BHave the validate task write anomalies to a separate Delta table and always succeed, and have the aggregate task read that table to determine if the output should be marked as 'review required'.
   • CCombine the validate and aggregate logic into a single task that writes the output with a 'review required' flag if anomalies are found.
   • DUse a conditional task that checks the output of the validate task and decides whether to run the aggregate task or a separate alerting task.
   Reveal Solution Discussion Select an answer first
5. 19foundation · easy

   When creating a notebook-based pipeline in Databricks Jobs, how do you specify that a notebook task should run after another notebook task?

   • AWrite the second notebook to call the first notebook using %run
   • BUse the same cluster for both tasks
   • CPlace both notebooks in the same folder and run them together
   • DAdd a dependency in the job configuration by selecting the upstream task in the 'Depends on' field
   Reveal Solution Discussion Select an answer first
6. 20expert · hard

   A team is building a notebook-based pipeline in a Lakeflow job. The pipeline has three tasks: ingest, clean, and publish. The ingest task reads data from an external source and writes to a staging table. The clean task performs transformations and writes to a final table. The publish task copies the final table to a production location. The team wants to ensure that if the clean task fails, the publish task does not run, but the ingest task should not be retried. They also want to be able to manually retry only the clean task without re-running ingest. What should they do?

   • AUse a Lakeflow Spark Declarative Pipeline instead of notebooks, because it allows granular retries of individual steps.
   • BSet the clean task to depend on the ingest task, and set the publish task to depend on the clean task. Configure the job to not retry the ingest task on failure, and allow manual reruns of individual tasks.
   • CCombine the ingest and clean tasks into a single notebook, so that a failure in clean also retries ingest.
   • DSet the publish task to depend on the ingest task, and set the clean task to depend on the publish task.
   Reveal Solution Discussion Select an answer first