Describe the Architecture of Apache Spark™, Including DataFrame and Dataset Concepts, SparkSession Lifecycle, Caching, Storage Levels, and Garbage Collection. ASSOCIATE-DEVELOPER-APACHE-SPARK Practice Questions (Page 4)

Questions 16–20

1. 16foundation · easy

   Which method allows you to specify a storage level when persisting a DataFrame, while cache() uses a default storage level?

   • Apersist()
   • Bcheckpoint()
   • Cunpersist()
   • Dcache()
   Reveal Solution Discussion Select an answer first
2. 17application · medium

   A Python developer needs to process a dataset with a known schema. They want to use Spark's optimizations like Catalyst and Tungsten, but they do not need compile-time type safety. Which API should they use?

   • ASQLContext
   • BDataset[T]
   • CRDD[Row]
   • DDataFrame
   Reveal Solution Discussion Select an answer first
4. 18foundation · easy

   Which Spark API provides compile-time type safety and allows you to work with strongly-typed JVM objects, while also offering the performance benefits of the Catalyst optimizer?

   • ADataset
   • BSparkContext
   • CDataFrame
   • DRDD
   Reveal Solution Discussion Select an answer first
5. 19expert · hard

   A Spark application is running on a YARN cluster. The application is failing with an OutOfMemoryError on the executors. The team has already increased the executor memory, but the issue persists. They suspect that the overhead of storing cached data in the JVM heap is the problem. Which configuration change is most likely to help?

   • AIncrease spark.executor.cores to allow more tasks per executor
   • BDisable caching entirely to free up heap space
   • CEnable off-heap storage by setting spark.memory.offHeap.enabled=true and spark.memory.offHeap.size
   • DSet spark.memory.fraction to 1.0 to allow more memory for execution and storage
   Reveal Solution Discussion Select an answer first
6. 20foundation · easy

   What is the primary difference between a DataFrame and a Dataset in Spark?

   • ADatasets are only available in Python, while DataFrames are only in Scala.
   • BDataFrames are distributed, while Datasets are not.
   • CDataFrames are untyped, while Datasets are strongly typed.
   • DDataFrames use Catalyst, while Datasets do not.
   Reveal Solution Discussion Select an answer first