User-Defined Functions and Stateful Operations ASSOCIATE-DEVELOPER-APACHE-SPARK Practice Questions (Page 5)

Questions 21–25

1. 21foundation · easy

   In flatMapGroupsWithState, how does the function indicate that a group should produce no output for the current batch?

   • AReturn an empty Iterator.
   • BThrow an exception.
   • CReturn null.
   • DReturn a single-element Iterator containing null.
   Reveal Solution Discussion Select an answer first
2. 22expert · hard

   A streaming application uses mapGroupsWithState to maintain a running count per key. The team wants to minimize the amount of state stored while still supporting accurate counts. Which approach is best?

   • AStore the count and the last event timestamp.
   • BStore only the count as an integer in the state.
   • CUse a windowed aggregation instead.
   • DStore the entire list of events for each key.
   Reveal Solution Discussion Select an answer first
4. 23foundation · easy

   What is the primary purpose of mapGroupsWithState in Spark Structured Streaming?

   • ATo aggregate streaming data into a single global state without keys.
   • BTo maintain and update state per key across streaming batches, producing exactly one output row per group per batch.
   • CTo perform a stateless map operation on each row in a streaming DataFrame.
   • DTo produce zero or more output rows per group, with the ability to emit multiple rows per batch.
   Reveal Solution Discussion Select an answer first
5. 24application · medium

   A streaming application uses flatMapGroupsWithState with a processing-time timeout of 5 minutes. The team wants to ensure that state is cleaned up even if no data arrives for a key. What is required?

   • AHandle the timeout in the update function by calling groupState.remove() or setting a final output.
   • BSet the timeout duration to 0.
   • CUse mapGroupsWithState instead.
   • DCall spark.conf.set('spark.sql.streaming.stateStore.cleanup', 'true').
   Reveal Solution Discussion Select an answer first
6. 25foundation · easy

   Which configuration setting controls the location where state data is stored for recovery?

   • Aspark.sql.shuffle.partitions
   • Bspark.streaming.receiver.writeAheadLog.enable
   • Cspark.sql.adaptive.enabled
   • Dspark.sql.streaming.checkpointLocation
   Reveal Solution Discussion Select an answer first