Describe Adaptive Query Execution (AQE) and Its Benefits. ASSOCIATE-DEVELOPER-APACHE-SPARK Practice Questions (Page 4)

Questions 16–20

1. 16application · medium

   A company stores sales data in a partitioned Delta table by date. A daily report joins this table with a small table of active store IDs. The report only needs data for the last 7 days. The team notices that Spark scans all partitions of the sales table even though the filter on the date column should limit the scan. They have already verified that the filter is pushed down correctly. Which AQE feature would most likely reduce the amount of data scanned in this scenario?

   • ADynamic join strategy selection
   • BDynamic skew join optimization
   • COptimized data layout
   • DDynamic partition pruning
   Reveal Solution Discussion Select an answer first
2. 17application · medium

   A Spark developer is troubleshooting a slow query. They suspect that AQE is not being applied to the query plan. They have already confirmed that spark.sql.adaptive.enabled is set to true. What is the most direct way to verify whether AQE is actually being applied to the query?

   • ACheck the Spark driver logs for the string 'AQE is enabled'
   • BCheck the physical query plan in the Spark UI for nodes like 'AdaptiveSparkPlan'
   • CVerify that the query runs successfully without any errors
   • DCheck the number of shuffle partitions in the Spark UI
   Reveal Solution Discussion Select an answer first
4. 18foundation · easy

   When does dynamic partition pruning make its pruning decisions?

   • AAfter the query completes, for logging purposes
   • BAt query compilation time
   • COnly when the query is explicitly cached
   • DDuring query execution, based on runtime statistics
   Reveal Solution Discussion Select an answer first
5. 19application · medium

   A data engineering team runs a nightly batch job that joins a large fact table with a small dimension table. The dimension table size varies significantly day-to-day. On some days it is under 10 MB, and on other days it grows to over 200 MB. The team wants to avoid manually changing the join strategy each day. Which configuration should they enable to let Spark automatically choose the most efficient join strategy at runtime?

   • Aspark.sql.adaptive.enabled=true
   • Bspark.sql.adaptive.coalescePartitions.enabled=true
   • Cspark.sql.autoBroadcastJoinThreshold=10MB
   • Dspark.sql.shuffle.partitions=200
   Reveal Solution Discussion Select an answer first
6. 20foundation · easy

   What does dynamic partition pruning in AQE primarily reduce?

   • AThe size of the broadcast variable
   • BThe number of tasks in a stage
   • CThe number of shuffle partitions
   • DThe amount of data scanned during query execution
   Reveal Solution Discussion Select an answer first