Describe Apache Spark - Streaming Processing and Architecture DATA-ENGINEERING-OPTIMIZE Practice Questions (Page 1)

Questions 1–5

1. 1foundation · easy

   How does Spark Streaming integrate with Spark SQL?

   • ABy using Spark SQL to create receivers
   • BBy using Spark SQL to manage the streaming context
   • CBy converting DStreams to DataFrames for SQL queries
   • DBy replacing DStreams with SQL tables
   Reveal Solution Discussion Select an answer first
2. 2expert · hard

   A streaming application uses Spark Streaming with a Kafka direct stream. The team wants to achieve exactly-once processing. Which combination of settings is required?

   • ACheckpointing, idempotent sink, and Kafka offset management.
   • BA high batch interval and checkpointing.
   • CWrite-ahead logging and a non-idempotent sink.
   • DMultiple receivers and checkpointing.
   Reveal Solution Discussion Select an answer first
4. 3foundation · easy

   What is the primary purpose of Spark Streaming in the Apache Spark ecosystem?

   • ATo process data in real time as it arrives, using a micro-batch model
   • BTo store large volumes of streaming data in a distributed file system
   • CTo manage cluster resources and schedule tasks across nodes
   • DTo provide a SQL interface for querying static data stored in HDFS
   Reveal Solution Discussion Select an answer first
5. 4expert · hard

   A team is deciding between Spark Streaming (DStreams) and Structured Streaming for a new project. The project requires event-time windowing, exactly-once processing from Kafka, and the ability to query streaming data with SQL. Which choice best satisfies all requirements?

   • AStructured Streaming with Kafka source and an idempotent sink.
   • BDStreams with window operations and SQL queries.
   • CStructured Streaming with a receiver-based Kafka source.
   • DDStreams with Kafka direct stream and a non-idempotent sink.
   Reveal Solution Discussion Select an answer first
6. 5expert · hard

   A company currently uses Spark Streaming (DStreams) to process IoT sensor data. They want to perform stateful operations like sessionization and also use SQL queries on the streaming data. They are considering migrating to Structured Streaming. Which migration consideration is most important?

   • ADStreams and Structured Streaming are interchangeable without code changes.
   • BStructured Streaming uses a different API and execution model, so existing DStream transformations must be rewritten.
   • CStructured Streaming does not support stateful operations, so sessionization is impossible.
   • DStructured Streaming cannot integrate with Spark SQL.
   Reveal Solution Discussion Select an answer first