Diagnose Cluster Startup Failures, Library Conflicts, and Out-Of-Memory Issues. DATA-ENGINEER-ASSOCIATE Practice Questions (Page 1)

Questions 1–5

1. 1foundation · medium

   A data engineer is diagnosing an out-of-memory error in a Spark job. Which configuration setting should be reviewed to understand the memory allocated to each executor?

   • Aspark.sql.shuffle.partitions
   • Bspark.driver.cores
   • Cspark.default.parallelism
   • Dspark.executor.memory
   Reveal Solution Discussion Select an answer first
2. 2foundation · medium

   A Spark job fails with an OutOfMemoryError in the executor. Which Spark UI tab should be examined first to identify which stage or task is consuming excessive memory?

   • ASQL tab
   • BEnvironment tab
   • CStages tab
   • DExecutors tab
   Reveal Solution Discussion Select an answer first
4. 3foundation · medium

   A data engineer encounters a library conflict where two libraries depend on different versions of the same package. Which tool or feature should be used to analyze the dependency tree and identify the conflicting versions?

   • ASpark UI's environment tab
   • BLibrary dependency graph in the cluster's library tab
   • CCluster event log
   • DNotebook's print statements
   Reveal Solution Discussion Select an answer first
5. 4foundation · medium

   A library conflict occurs because a library installed at the cluster level is overridden by a library installed at the notebook level. Which configuration should be used to ensure the cluster-level library version is used consistently across all notebooks?

   • AInstall the library in every notebook individually
   • BSet a Spark configuration to ignore notebook libraries
   • CUse the Spark UI to force the cluster library version
   • DRemove the notebook-scoped library and rely on the cluster library
   Reveal Solution Discussion Select an answer first
6. 5application · medium

   A data engineering team has a cluster that fails to start. The cluster configuration page shows the state as 'Terminated' with the message 'Cloud provider launch failure: Insufficient instance capacity.' The team has verified that the instance profile and VPC settings are correct. What should the team do first to resolve this issue?

   • AModify the cluster's Spark configuration to reduce the executor memory overhead.
   • BReview the driver and executor logs in the cluster event log for a library dependency error.
   • CDisable autoscaling and set a fixed number of workers to reduce resource demand.
   • DRetry the cluster creation with a different availability zone or instance type.
   Reveal Solution Discussion Select an answer first