Data Aggregation and Joins ASSOCIATE-DEVELOPER-APACHE-SPARK Practice Questions (Page 3)

Questions 11–15

1. 11foundation · easy

   What is the primary benefit of using a broadcast join hint?

   • AIt sorts the data before joining to improve performance
   • BIt ensures the join result is written to disk for fault tolerance
   • CIt increases the number of partitions for better parallelism
   • DIt reduces data shuffling by sending the small DataFrame to all executors
   Reveal Solution Discussion Select an answer first
2. 12application · medium

   A data engineer has two DataFrames, `logs1` and `logs2`, with identical schemas. They need to combine them into a single DataFrame for further analysis, but they must preserve every log entry, even if the same log appears in both DataFrames. Which method should they use?

   • AUse `unionAll`
   • BUse `distinct` before `union`
   • CUse `union`
   • DUse `join` on a common column
   Reveal Solution Discussion Select an answer first
4. 13application · medium

   A company has two DataFrames, `df1` and `df2`, both with the same schema containing `id` and `value`. They need to combine them into a single DataFrame, but they want to remove any duplicate rows that appear in both DataFrames. Which method should they use?

   • AUse `crossJoin`
   • BUse `unionAll`
   • CUse `union`
   • DUse `join` on all columns
   Reveal Solution Discussion Select an answer first
5. 14foundation · easy

   Which DataFrame method returns the total number of rows in the DataFrame as a Long?

   • AnumRows()
   • Bcount()
   • Clength()
   • Dsize()
   Reveal Solution Discussion Select an answer first
6. 15foundation · easy

   Which DataFrame method generates summary statistics including count, mean, stddev, min, max, and percentiles for numeric columns?

   • Adescribe()
   • Baggregate()
   • Csummary()
   • Dstats()
   Reveal Solution Discussion Select an answer first