What is a task in Apache Spark?

A logical plan of transformations that Spark optimizes

The smallest unit of work that runs on a single partition of data on an executor

A unit of work that runs on the driver node

A collection of operations applied to an entire dataset

A data engineer is troubleshooting a Spark application that processes a large dataset. The Spark UI shows that a job is split into three stages. The first stage reads from Parquet files and performs a filter, the second stage performs a groupBy, and the third stage writes the result. The engineer notices that the second stage has many more tasks than the first. What is the most likely reason for the increase in task count?

The write operation in the third stage forced the second stage to use more tasks to prepare the output.

The filter operation in the first stage removed too many records, so the second stage had to compensate with more tasks.

The second stage was created because of a narrow dependency, which always results in more tasks.

The groupBy operation introduced a shuffle, causing the data to be repartitioned across a different number of partitions.

In the Spark execution hierarchy, where does a task run?

On the driver, coordinating stage execution

On an executor, processing one partition of data

On the storage layer, reading data from disk

On the cluster manager, allocating resources

DatabricksCertified Associate Developer for Apache Spark

Domain 1Objective 4

Explain the Apache Spark™ Architecture Execution Hierarchy.. ASSOCIATE-DEVELOPER-APACHE-SPARK Practice Questions (Page 3)

Part of the Apache Spark Architecture and Components domain, which accounts for 20% of the ASSOCIATE-DEVELOPER-APACHE-SPARK exam. Databricks does not publish an official question count, but from its 90-minute exam (~35–60 total, ~7–12 in this domain), expect 1–2 from this objective — we provide 27 practice questions to prepare you well beyond it. (estimate)

27questions here

6free pages

7concepts

20%of the exam

Questions 11–15

11application · medium
A Spark job is reading from a directory of 200 small Parquet files. The default parallelism is set to 100. After reading, the DataFrame is repartitioned to 50 partitions, then a wide transformation is applied. How many tasks will be in the stage that contains the wide transformation?
Select an answer first
12foundation · easy
What is a task in Apache Spark?
Select an answer first
13application · medium
A data engineer is troubleshooting a Spark application that processes a large dataset. The Spark UI shows that a job is split into three stages. The first stage reads from Parquet files and performs a filter, the second stage performs a groupBy, and the third stage writes the result. The engineer notices that the second stage has many more tasks than the first. What is the most likely reason for the increase in task count?
Select an answer first
14foundation · easy
In the Spark execution hierarchy, where does a task run?
Select an answer first
15application · medium
A Spark application is running on a cluster with 4 executors, each with 4 cores. The current stage has 16 tasks. What is the maximum number of tasks that can run concurrently in this stage?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to ASSOCIATE-DEVELOPER-APACHE-SPARK

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by Databricks. “ASSOCIATE-DEVELOPER-APACHE-SPARK” is a trademark of its owner, used for identification only.

Explain the Apache Spark™ Architecture Execution Hierarchy.. ASSOCIATE-DEVELOPER-APACHE-SPARK Practice Questions (Page 3)

Questions 11–15

A Spark job is reading from a directory of 200 small Parquet files. The default parallelism is set to 100. After reading, the DataFrame is repartitioned to 50 partitions, then a wide transformation is applied. How many tasks will be in the stage that contains the wide transformation?

What is a task in Apache Spark?

In the Spark execution hierarchy, where does a task run?

A Spark application is running on a cluster with 4 executors, each with 4 cores. The current stage has 16 tasks. What is the maximum number of tasks that can run concurrently in this stage?

Keep studying — Apache Spark Architecture and Components