Explain the Apache Spark™ Architecture Execution Hierarchy.. ASSOCIATE-DEVELOPER-APACHE-SPARK Practice Questions (Page 1)

Questions 1–5

1. 1foundation · easy

   Which of the following correctly orders the Apache Spark execution hierarchy from the largest unit of work to the smallest?

   • AStage → Job → Task
   • BJob → Task → Stage
   • CTask → Stage → Job
   • DJob → Stage → Task
   Reveal Solution Discussion Select an answer first
2. 2application · easy

   A new data engineer is learning Spark and asks: 'What is the relationship between jobs, stages, and tasks?' Which explanation is correct?

   • AA job is triggered by an action and is divided into stages, each of which consists of tasks that run on partitions.
   • BTasks are the largest unit of work, and jobs are made up of tasks directly.
   • CA job is a single task that runs on one executor, and stages are collections of jobs.
   • DStages are triggered by actions, and each stage contains multiple jobs.
   Reveal Solution Discussion Select an answer first
4. 3application · medium

   A developer submits a Spark application that performs a series of transformations and then calls count() on the result. The developer wants to understand the order of execution. Which sequence correctly describes what happens after count() is called?

   • AThe task scheduler creates stages, then the DAG scheduler launches tasks, and finally the job is created.
   • BThe action is executed immediately on the driver, and then the results are distributed to executors.
   • CThe DAG scheduler creates a job, splits it into stages, and then the task scheduler launches tasks on executors.
   • DThe driver sends the transformations to each executor, which then builds the DAG and runs the job.
   Reveal Solution Discussion Select an answer first
5. 4application · medium

   A data engineer is analyzing the Spark UI for a job that performs a map, then a reduceByKey, then a filter. How many stages will the job have?

   • ATwo stages: one for the map and reduceByKey, and one for the filter.
   • BThree stages: one for each transformation.
   • COne stage, because all transformations are narrow.
   • DFour stages, because the map and filter are also wide dependencies.
   Reveal Solution Discussion Select an answer first
6. 5application · medium

   A data engineer is optimizing a Spark job that joins two large DataFrames. The current execution plan shows that the join causes a shuffle, and the job is split into multiple stages. The engineer wants to reduce the number of stages. Which approach would directly reduce the number of stages caused by the shuffle?

   • ACache the larger DataFrame before the join.
   • BIncrease the number of partitions for the shuffle using spark.sql.shuffle.partitions.
   • CUse a sort-merge join instead of a hash join.
   • DUse a broadcast join hint if one of the DataFrames is small enough to fit in memory on each executor.
   Reveal Solution Discussion Select an answer first