User-Defined Functions and Stateful Operations ASSOCIATE-DEVELOPER-APACHE-SPARK Practice Questions (Page 3)

Questions 11–15

1. 11application · medium

   A Spark job uses a Python UDF to convert a string column to a timestamp. The UDF is called on a column that contains many distinct string values. The job is slow. Which optimization is most effective?

   • AReplace the UDF with the built-in to_timestamp function.
   • BIncrease the number of executor cores.
   • CCache the DataFrame before applying the UDF.
   • DUse a UDF with a return type of TimestampType.
   Reveal Solution Discussion Select an answer first
2. 12expert · hard

   A team is building a streaming application that must track user sessions. For each user, they need to output a session summary when the session ends (after 30 minutes of inactivity). They also need to output intermediate updates for active sessions. Which API should they use?

   • AflatMapGroupsWithState, because it can emit zero or more rows per group and handle timeouts.
   • BA UDF with a state variable.
   • CA windowed aggregation with a 30-minute slide.
   • DmapGroupsWithState, because it is simpler and can emit one row per key.
   Reveal Solution Discussion Select an answer first
4. 13expert · hard

   A streaming application uses mapGroupsWithState and needs to recover from a failure. The team is considering using the RocksDB state store provider. What is a key benefit of this choice?

   • AIt speeds up all state operations compared to HDFS.
   • BIt eliminates the need for checkpointing.
   • CIt reduces memory usage by storing state on disk.
   • DIt automatically cleans up state without timeouts.
   Reveal Solution Discussion Select an answer first
5. 14application · medium

   A streaming application uses flatMapGroupsWithState to generate output rows. The team wants to ensure that the state is checkpointed for fault tolerance. What is required?

   • AUse mapGroupsWithState instead.
   • BCall groupState.setTimeoutDuration() for each key.
   • CEnable the RocksDB state store provider.
   • DSet a checkpoint location in the streaming query.
   Reveal Solution Discussion Select an answer first
6. 15foundation · easy

   What happens when a state timeout is triggered in a stateful streaming operation?

   • AThe streaming query fails with a timeout exception.
   • BThe state is automatically deleted without any function invocation.
   • CThe state is kept indefinitely until the next batch arrives.
   • DThe stateful function is called with the state marked as timed out, allowing the function to clean up or emit a final output.
   Reveal Solution Discussion Select an answer first