Shared Variables and Broadcast Joins ASSOCIATE-DEVELOPER-APACHE-SPARK Practice Questions (Page 3)

Questions 11–15

1. 11application · medium

   A data engineering team runs a daily Spark job that joins a large fact table (2 TB) with a small lookup table (50 MB). The lookup table is static and used only for enrichment. The team notices the job takes 30 minutes and generates heavy shuffle traffic. They want to minimize shuffle and improve performance without changing the cluster size. Which approach should they use?

   • AUse the broadcast function on the lookup DataFrame before the join.
   • BConvert the lookup table to a broadcast variable and access it inside a UDF.
   • CIncrease spark.sql.shuffle.partitions to 2000.
   • DRepartition the fact table by the join key before the join.
   Reveal Solution Discussion Select an answer first
2. 12application · medium

   A company runs a Spark SQL query that joins a large orders table with a small currency conversion table. The currency table is 20 MB and changes daily. They want to ensure the join uses a broadcast join without modifying the query. Which configuration should they set?

   • Aspark.sql.broadcastTimeout=300
   • Bspark.sql.autoBroadcastJoinThreshold=-1
   • Cspark.sql.shuffle.partitions=200
   • Dspark.sql.autoBroadcastJoinThreshold=20971520
   Reveal Solution Discussion Select an answer first
4. 13application · easy

   A team runs a join between a large fact table (1 TB) and a small dimension table (10 MB). They want to minimize shuffle and improve performance. Which join strategy should they choose?

   • ABroadcast join
   • BCartesian join
   • CSort-merge join
   • DShuffled hash join
   Reveal Solution Discussion Select an answer first
5. 14foundation · easy

   Which problem does a broadcast variable solve in a Spark application?

   • APersisting a DataFrame to disk for fault tolerance
   • BAllowing multiple tasks to update a shared counter
   • CAvoiding the repeated transfer of a large read-only lookup table to each task
   • DEnabling tasks to write results back to the driver efficiently
   Reveal Solution Discussion Select an answer first
6. 15foundation · easy

   Which two types of shared variables does Apache Spark provide for use in distributed tasks?

   • ADistributed caches and counters
   • BPartition variables and reducers
   • CBroadcast variables and accumulators
   • DShared dataframes and broadcast datasets
   Reveal Solution Discussion Select an answer first