Explain the Structured Streaming Engine in Spark, Including Its Functions, Programming Model, Micro-Batch Processing, Exactly-Once Semantics, and Fault Tolerance Mechanisms. ASSOCIATE-DEVELOPER-APACHE-SPARK Practice Questions (Page 4)

Questions 16–20

1. 16application · medium

   A developer is new to Structured Streaming and wants to understand how a streaming query is similar to a batch query. Which statement correctly describes the programming model?

   • AA streaming query can only use SQL, not DataFrame transformations.
   • BA streaming query is expressed as a DataFrame that operates on an unbounded table, using the same transformations as batch.
   • CA streaming query processes data record-by-record, unlike batch which processes in sets.
   • DA streaming query requires a different API, such as DStreams, to handle continuous data.
   Reveal Solution Discussion Select an answer first
2. 17foundation · easy

   In Structured Streaming's programming model, what does the concept of an 'unbounded input table' represent?

   • AA table that is limited to a fixed number of rows based on the batch interval
   • BA table that is stored in memory only and cannot be persisted
   • CA table that grows indefinitely as new data arrives from the streaming source
   • DA table that contains only the most recent micro-batch of data
   Reveal Solution Discussion Select an answer first
4. 18expert · hard

   A streaming query uses a Kafka source and writes to a Delta table. The team notices that after a restart, some records are reprocessed even though the checkpoint exists. What is the most likely cause?

   • AThe trigger interval was changed, causing the query to restart from the beginning.
   • BThe checkpoint location was changed or deleted between runs.
   • CThe Delta sink is not idempotent, causing duplicates.
   • DThe Kafka source does not support replay, so offsets are lost.
   Reveal Solution Discussion Select an answer first
5. 19foundation · easy

   Which two mechanisms are essential for Structured Streaming to provide exactly-once processing guarantees?

   • ACheckpointing and write-ahead logs
   • BWindow functions and watermarking
   • CIn-memory caching and broadcast variables
   • DData partitioning and shuffling
   Reveal Solution Discussion Select an answer first
6. 20expert · hard

   A streaming job uses a Kafka source and a file sink. After a node failure, the job restarts and reprocesses a batch that was partially written to the file sink. How does Structured Streaming prevent duplicate records in the output?

   • AThe checkpoint stores the exact records written, and the file sink skips any record already present.
   • BThe Kafka source automatically deletes consumed records, so reprocessing is impossible.
   • CThe file sink uses a unique file name per batch, and the job deletes the old file before rewriting.
   • DThe write-ahead log records the batch's output, and the file sink writes to a temporary location that is committed atomically.
   Reveal Solution Discussion Select an answer first