Create and Invoke Pandas UDF. ASSOCIATE-DEVELOPER-APACHE-SPARK Practice Questions (Page 3)

Questions 11–12

1. 11application · medium

   A data engineer needs to compute the median of a 'price' column for each 'category' group in a Spark DataFrame, and then join the result back to the original rows. They want to use a Pandas UDF. Which function type should they define?

   • AA GROUPED_AGG Pandas UDF that returns a scalar per group, used in groupBy().agg()
   • BA SCALAR Pandas UDF that takes the 'price' column and returns the median for the entire column
   • CA GROUPED_MAP Pandas UDF that returns a DataFrame with one row per input group
   • DA SCALAR Pandas UDF used inside groupBy().agg() to compute the median per group
   Reveal Solution Discussion Select an answer first
2. 12foundation · easy

   What is a primary performance benefit of Pandas UDFs compared to standard Python UDFs?

   • AThey avoid serialization overhead by using Java objects.
   • BThey run on the driver node instead of executors.
   • CThey process data in batches using vectorized operations.
   • DThey eliminate the need for data serialization altogether.
   Reveal Solution Discussion Select an answer first