Design and Implement Data Pipelines DP-750 Practice Questions (Page 6)

Questions 26–30

1. 26application · medium

   A company wants to build a data pipeline that reads from a streaming source, performs windowed aggregations, and writes to a Delta table. They prefer a declarative approach that minimizes custom code and is easy to maintain. Which Azure Databricks feature should they use?

   • AA notebook with Structured Streaming
   • BA scheduled job that runs a SQL query
   • CLakeflow Spark Declarative Pipelines
   • DAzure Data Factory with a copy activity
   Reveal Solution Discussion Select an answer first
2. 27application · medium

   A data pipeline uses a notebook task in a Lakeflow job. The notebook reads data from an external API and writes to a Delta table. Occasionally, the API returns a 503 error, which is transient. The team wants the pipeline to retry the API call a few times before failing. They also want to log the number of retries for monitoring. What should they implement in the notebook?

   • AUse a Delta Live Tables pipeline with a 'retry on transient errors' option.
   • BUse the Lakeflow job's retry policy on the notebook task, which will automatically retry the entire notebook on failure.
   • CWrap the API call in a loop that retries up to 3 times with exponential backoff, and log each retry attempt using the notebook's logging capabilities.
   • DCatch the exception and continue without retrying, but log the error for later review.
   Reveal Solution Discussion Select an answer first
4. 28application · medium

   A data engineering team needs to build a pipeline that ingests raw JSON files from a cloud storage location, performs schema validation, and then writes the cleaned data to a Delta table for downstream reporting. The team wants to minimize the amount of custom code and prefer a declarative approach where the transformation logic is defined as a series of steps. They also need the pipeline to be easily version-controlled and reviewed as part of their CI/CD process. Which approach should the team choose?

   • AUse a Lakeflow Spark Declarative Pipeline to define the read, validation, and write steps as a directed acyclic graph, then run it as a Lakeflow job task.
   • BCreate a notebook that reads the JSON files, applies validation logic, and writes to the Delta table. Schedule the notebook as a single task in a Lakeflow job.
   • CConfigure a Delta Live Tables pipeline with a single SQL query that performs the validation and write operation in one step.
   • DWrite a Python script that uses the Databricks SQL Connector to execute a series of SQL statements for validation and insertion into the Delta table.
   Reveal Solution Discussion Select an answer first
5. 29foundation · easy

   In a Lakeflow Job, what is the primary purpose of designing a task to write its output to a temporary table before the final task aggregates results?

   • ATo enable parallel execution of all tasks
   • BTo eliminate the need for cluster configuration
   • CTo allow the final task to read from a stable, complete intermediate result
   • DTo reduce the number of tasks in the job
   Reveal Solution Discussion Select an answer first
6. 30expert · hard

   A team is designing a Lakeflow job that processes data from multiple sources. The job has a task that ingests data from Source A, a task that ingests data from Source B, and a task that merges the data from both sources. The team wants to ensure that if either ingestion task fails, the merge task does not run, but the other ingestion task should still complete. They also want to be able to see which ingestion task failed and retry only that task. What should they do?

   • ACreate two independent ingestion tasks and a merge task that depends on both. Configure the job to not stop on the first failure, so both ingestion tasks run, and the merge task only runs if both succeed.
   • BCreate two ingestion tasks with a dependency between them, so Source B waits for Source A, and the merge task depends on Source B.
   • CCreate two ingestion tasks and a merge task that depends on only one of them, and have the merge task check if the other source is available.
   • DCreate a single ingestion task that processes both sources sequentially, and if one fails, the whole task fails.
   Reveal Solution Discussion Select an answer first