Spark DATA-SCIENCE-OPTIMIZE Practice Questions (Page 1)

Questions 1–5

1. 1foundation · easy

   Which of the following is a common machine learning algorithm available in Spark MLlib?

   • ASQL JOIN
   • BHDFS replication
   • CMapReduce
   • DRandom Forest
   Reveal Solution Discussion Select an answer first
2. 2foundation · easy

   When would you choose an RDD over a DataFrame in Spark?

   • AWhen you need to use SQL to query the data.
   • BWhen you want automatic query optimization for better performance.
   • CWhen you want to leverage the Catalyst optimizer for complex joins.
   • DWhen you need low-level control over the data and operations, such as custom partitioning.
   Reveal Solution Discussion Select an answer first
4. 3foundation · easy

   What is a key advantage of using a DataFrame over an RDD in Spark?

   • ADataFrames do not require a schema, making them more flexible than RDDs.
   • BDataFrames automatically optimize queries through the Catalyst optimizer.
   • CDataFrames are always stored in memory, while RDDs are stored on disk.
   • DDataFrames are immutable, while RDDs are mutable.
   Reveal Solution Discussion Select an answer first
5. 4application · medium

   A data science team is working on a classification problem where they need to predict customer churn. They have a large dataset with millions of rows and many features. They want to use Spark MLlib to train a model. Which algorithm is most appropriate for this binary classification task?

   • AFP-Growth
   • BALS
   • CLogistic regression
   • DK-means clustering
   Reveal Solution Discussion Select an answer first
6. 5expert · hard

   A team is designing a Spark pipeline that processes both structured and semi-structured data. They need to perform complex aggregations and also run ad-hoc SQL queries. They are considering whether to use RDDs or DataFrames. The team values performance and ease of use, but they also need to handle a custom transformation that is not easily expressed in SQL. What should they do?

   • AUse DataFrames for the SQL and aggregation parts, and convert to RDDs only for the custom transformation
   • BUse DataFrames for everything and implement the custom transformation as a UDF
   • CUse RDDs for the entire pipeline to have full control
   • DUse Spark SQL for everything and avoid custom transformations
   Reveal Solution Discussion Select an answer first