Describe the Architecture of Apache Spark™, Including DataFrame and Dataset Concepts, SparkSession Lifecycle, Caching, Storage Levels, and Garbage Collection. ASSOCIATE-DEVELOPER-APACHE-SPARK Practice Questions (Page 3)

Questions 11–15

1. 11expert · hard

   A Spark job uses a large cached DataFrame and performs many iterative operations. The team notices that the job becomes slower after each iteration and suspects that garbage collection is causing the slowdown. They have already tried increasing executor memory. What is the next best step to reduce GC overhead?

   • AUse DISK_ONLY to avoid memory pressure entirely
   • BUse MEMORY_ONLY_SER to store cached data in a serialized format, reducing memory footprint and GC pressure
   • CIncrease the number of partitions to spread the data across more executors
   • DDisable caching and recompute the data each iteration
   Reveal Solution Discussion Select an answer first
2. 12foundation · easy

   In a Spark application, which component is responsible for executing tasks and storing data for cached RDDs or DataFrames?

   • ASparkSession
   • BCluster Manager
   • CExecutor
   • DDriver
   Reveal Solution Discussion Select an answer first
4. 13foundation · easy

   Which strategy can help reduce garbage collection pressure in Spark?

   • AUsing serialized data formats like MEMORY_ONLY_SER for caching.
   • BUsing the default storage level for all cached data.
   • CDisabling garbage collection entirely.
   • DIncreasing the number of partitions to create more tasks.
   Reveal Solution Discussion Select an answer first
5. 14application · medium

   A developer is writing a new Spark application and needs to read a CSV file, perform a SQL query, and then write the results to Parquet. They want to minimize boilerplate and use the recommended API. What is the correct way to set up the application?

   • ACreate a SparkSession with SparkSession.builder().appName("app").getOrCreate() and use it for reading, SQL, and writing
   • BUse SparkContext to read the CSV and then convert to a DataFrame manually
   • CCreate a new SparkSession for each operation to keep them isolated
   • DCreate a SparkContext for reading and a SQLContext for SQL, then use SparkContext for writing
   Reveal Solution Discussion Select an answer first
6. 15foundation · easy

   In Spark 2.0 and later, which of the following is true about SparkSession?

   • AIt is only used for streaming applications.
   • BIt must be manually created for each DataFrame operation.
   • CIt replaces SparkContext entirely, and SparkContext is no longer available.
   • DIt unifies SparkContext, SQLContext, and HiveContext into a single entry point.
   Reveal Solution Discussion Select an answer first