Building and Testing an ETL Pipeline with Lakeflow Spark Declarative Pipelines, SQL, and Apache Spark on the Databricks Platform DATA-ENGINEER-PROFESSIONAL Practice Questions (Page 4)

Questions 16–20

1. 16foundation · easy

   What is the primary purpose of the built-in debugger in Databricks?

   • ATo deploy the code to production
   • BTo step through code and inspect variables to troubleshoot issues
   • CTo automatically fix errors in the code
   • DTo optimize the performance of the code
   Reveal Solution Discussion Select an answer first
2. 17foundation · easy

   Which of the following is a key feature of Autoloader that enables it to track which files have already been processed?

   • AA separate metadata database
   • BA unique identifier in the file name
   • CA timestamp column in the source data
   • DA checkpoint directory that stores metadata about processed files
   Reveal Solution Discussion Select an answer first
4. 18foundation · easy

   What is the primary purpose of the AUTO CDC feature in Lakeflow Spark Declarative Pipelines?

   • ATo automatically generate documentation for the pipeline
   • BTo simplify the process of capturing and applying change data from source tables
   • CTo automatically scale compute resources based on workload
   • DTo automatically detect and fix data quality issues
   Reveal Solution Discussion Select an answer first
5. 19expert · hard

   A company is migrating their existing batch ETL jobs to Lakeflow Spark Declarative Pipelines. They have a complex dependency graph with multiple tables. Some tables are updated incrementally, while others are fully recomputed. They also need to ensure that if a table fails, only the dependent tables are affected, not the entire pipeline. What is the best way to structure the pipeline?

   • AUse a single pipeline but disable retries to prevent cascading failures.
   • BCreate separate pipelines for each table to isolate failures.
   • CDefine all tables in a single pipeline and rely on the pipeline's dependency management to handle failures.
   • DUse a single pipeline with a materialized view for all tables to simplify the logic.
   Reveal Solution Discussion Select an answer first
6. 20foundation · easy

   What is the primary characteristic of Lakeflow Spark Declarative Pipelines that distinguishes them from imperative Spark code?

   • AThey require the use of the Scala programming language exclusively.
   • BThey define the desired data output and let the system determine the execution plan.
   • CThey are limited to batch processing and cannot handle streaming data.
   • DThey require specifying the exact sequence of operations to execute on data.
   Reveal Solution Discussion Select an answer first