Explain the Structured Streaming Engine in Spark, Including Its Functions, Programming Model, Micro-Batch Processing, Exactly-Once Semantics, and Fault Tolerance Mechanisms. ASSOCIATE-DEVELOPER-APACHE-SPARK Practice Questions (Page 3)

Questions 11–15

1. 11foundation · easy

   Which component of Spark does Structured Streaming leverage to provide a unified API for both batch and streaming data?

   • ASpark Core's RDD API
   • BSpark Streaming's DStream API
   • CMLlib's streaming algorithms
   • DSpark SQL engine
   Reveal Solution Discussion Select an answer first
2. 12application · medium

   A streaming query uses a mapGroupsWithState operation to maintain session state. The team wants to ensure that the state is not lost on failure. Which mechanism provides this guarantee?

   • AThe state is stored in memory only and rebuilt from the source on restart.
   • BThe state is written to the sink with each batch.
   • CThe state is stored in a global temp view.
   • DThe state store is checkpointed as part of the streaming query's checkpoint.
   Reveal Solution Discussion Select an answer first
4. 13foundation · easy

   Which of the following is a valid output mode for a Structured Streaming query?

   • AOverwrite
   • BMerge
   • CInsert
   • DAppend
   Reveal Solution Discussion Select an answer first
5. 14expert · hard

   A streaming job writes to a custom sink that is not idempotent. The team wants to achieve exactly-once semantics. What is the recommended approach?

   • AUse the complete output mode to overwrite the entire sink each batch.
   • BSwitch to a sink that supports at-least-once semantics.
   • CUse the foreachBatch sink to write the output of each micro-batch in a transaction with the sink.
   • DRely on the checkpoint to prevent duplicate writes to the sink.
   Reveal Solution Discussion Select an answer first
6. 15application · medium

   A team is choosing between Structured Streaming and a custom batch job that runs every 15 minutes. Their data volume is high, but they need results within 1 minute of arrival. Which consideration should drive their decision?

   • AA custom batch job is always faster because it processes all data at once.
   • BStructured Streaming is not suitable for high data volumes because micro-batches add overhead.
   • CStructured Streaming's micro-batch model can achieve sub-minute latency with a trigger interval, while a 15-minute batch job cannot.
   • DA custom batch job provides exactly-once semantics, while Structured Streaming only provides at-least-once.
   Reveal Solution Discussion Select an answer first