Explain the Structured Streaming Engine in Spark, Including Its Functions, Programming Model, Micro-Batch Processing, Exactly-Once Semantics, and Fault Tolerance Mechanisms. ASSOCIATE-DEVELOPER-APACHE-SPARK Practice Questions (Page 2)

Questions 6–10

1. 6application · medium

   A streaming query reads from a file source and writes aggregated counts to memory. The team wants to see only the new counts for each micro-batch, not the cumulative total. Which output mode should they use?

   • AAppend mode
   • BUpdate mode
   • CComplete mode
   • DFinal mode
   Reveal Solution Discussion Select an answer first
2. 7foundation · easy

   What is a primary trade-off of the micro-batch processing model in Structured Streaming?

   • AHigher latency and lower throughput compared to continuous processing
   • BHigher latency compared to continuous processing, but higher throughput and fault tolerance
   • CLower throughput compared to continuous processing, but lower latency
   • DNo trade-off; it provides both lower latency and higher throughput than continuous processing
   Reveal Solution Discussion Select an answer first
4. 8application · medium

   A company wants to build a streaming pipeline that can handle late-arriving data and update previous aggregations. They plan to use Structured Streaming. Which combination of features supports this requirement?

   • AComplete mode output and no watermarking
   • BA custom UDF that buffers all events and a batch sink
   • CProcessing-time triggers and append mode output
   • DEvent-time watermarking and update mode output
   Reveal Solution Discussion Select an answer first
5. 9foundation · easy

   What is the primary purpose of Structured Streaming in Apache Spark?

   • ATo enable real-time stream processing with exactly-once semantics and fault tolerance using the Spark SQL engine
   • BTo provide a message queue service for ingesting streaming data
   • CTo provide a low-level API for managing distributed state in streaming applications
   • DTo replace batch processing with a purely event-driven processing model
   Reveal Solution Discussion Select an answer first
6. 10expert · hard

   A streaming job reads from Kafka and writes to a JDBC sink that does not support idempotent writes. The team wants exactly-once semantics. Which approach should they use?

   • AUse the complete output mode to overwrite the entire table each batch.
   • BUse the append output mode and rely on the checkpoint to prevent duplicates.
   • CUse foreachBatch to write each micro-batch within a transaction that is committed only after the batch is fully processed.
   • DUse a custom sink that buffers writes and flushes only on successful batch commit.
   Reveal Solution Discussion Select an answer first