Debugging and Troubleshooting DATA-ENGINEER-PROFESSIONAL Practice Questions (Page 6)

Questions 26–30

1. 26application · medium

   A job run failed due to a transient network error while writing to a cloud storage location. The job logic is correct and the data is unchanged. The engineer wants to rerun the failed run without modifying the job definition. Which action should the engineer take?

   • AQuery the `system.jobs.runs` table to find the run ID and then manually rerun the job with that ID.
   • BCreate a new job with the same configuration and trigger it manually.
   • CModify the job's notebook code to add retry logic and then run the job again.
   • DUse the 'Repair run' option on the failed run to rerun it with the same parameters.
   Reveal Solution Discussion Select an answer first
2. 27application · medium

   A Spark job fails with the error `java.lang.ClassNotFoundException: com.example.MyUDF`. The UDF is defined in a JAR that was attached to the cluster. Which is the most likely root cause?

   • AThe JAR was not included in the cluster's library configuration for the job.
   • BThe cluster does not have enough memory to load the JAR.
   • CThe UDF class has a syntax error that prevents compilation.
   • DThe UDF is not registered in the SparkSession.
   Reveal Solution Discussion Select an answer first
4. 28foundation · easy

   When triggering a job run manually, where can you specify parameter overrides?

   • AIn the 'Run now' dialog
   • BIn the cluster configuration
   • CIn the notebook's code
   • DIn the job's settings page
   Reveal Solution Discussion Select an answer first
5. 29foundation · easy

   In the Spark UI, which tab provides detailed metrics about individual tasks within a stage, such as task duration, shuffle read/write sizes, and any task-level errors?

   • AStreaming tab
   • BSQL tab
   • CStages tab
   • DEvent Timeline tab
   Reveal Solution Discussion Select an answer first
6. 30expert · hard

   A Spark job fails with an `OutOfMemoryError` in the executor, but only when the cluster is running multiple concurrent jobs. The Spark UI shows that the executor's storage memory is nearly full, and the cluster logs show frequent garbage collection. The job runs successfully when run alone. Which is the most likely root cause?

   • AThe executor memory is insufficient for the combined memory footprint of concurrent jobs sharing the same cluster.
   • BThe job has a data skew issue that only manifests under concurrent load.
   • CThe cluster's autoscaling is not adding executors quickly enough under load.
   • DThe driver is running out of memory due to too many concurrent tasks.
   Reveal Solution Discussion Select an answer first