Spark DATA-SCIENCE-OPTIMIZE Practice Questions (Page 5)

Questions 21–25

1. 21foundation · easy

   What is the basic unit of data in Spark Streaming (DStream)?

   • AA key-value pair
   • BA single RDD
   • CA DataFrame
   • DA sequence of RDDs over time
   Reveal Solution Discussion Select an answer first
2. 22foundation · easy

   In a Spark application, which component is responsible for breaking the user's program into tasks and scheduling those tasks across the cluster's worker nodes?

   • AThe driver program
   • BThe cluster manager
   • CThe shuffle service
   • DThe executor
   Reveal Solution Discussion Select an answer first
4. 23foundation · easy

   What is the purpose of a window operation in Spark Streaming?

   • ATo filter out irrelevant events from the stream.
   • BTo store the stream data in a persistent database.
   • CTo convert a batch RDD into a streaming DataFrame.
   • DTo aggregate data over a sliding time interval, such as counting events in the last 5 minutes.
   Reveal Solution Discussion Select an answer first
5. 24application · medium

   A team is migrating a legacy Spark job that uses RDDs to use DataFrames for better performance. The job currently performs a series of map and filter operations on a large dataset. What is the primary benefit they should expect from this migration?

   • ADataFrames provide a higher-level API that is easier to use than RDDs
   • BDataFrames automatically handle schema inference without any manual effort
   • CDataFrames benefit from Catalyst optimizer and Tungsten execution for better performance
   • DDataFrames automatically persist data in memory across all operations
   Reveal Solution Discussion Select an answer first
6. 25expert · hard

   A team is building a data pipeline that ingests JSON logs, performs schema validation, and then runs SQL analytics. They are deciding between using RDDs or DataFrames. The team values type safety and the ability to use SQL, but they also need to handle malformed records gracefully. Which approach should they take?

   • AUse RDDs for schema validation and DataFrames for SQL analytics
   • BUse Spark SQL with a temporary view and a UDF for schema validation
   • CUse RDDs for the entire pipeline to have maximum control over error handling
   • DUse DataFrames with a custom schema and the 'PERMISSIVE' mode to handle malformed records
   Reveal Solution Discussion Select an answer first