Using Spark ML MACHINE-LEARNING-PROFESSIONAL Practice Questions (Page 3)

Questions 11–15

1. 11foundation · easy

   Which SparkML evaluator class would you use to compute the area under the ROC curve for a binary classification model?

   • AMulticlassClassificationEvaluator
   • BBinaryClassificationEvaluator
   • CClusteringEvaluator
   • DRegressionEvaluator
   Reveal Solution Discussion Select an answer first
2. 12application · medium

   A fintech startup needs to serve a fraud detection model. They have a small dataset (50 MB) and require predictions with sub-50ms latency for a REST API. The model is a gradient-boosted tree. What is the best inference approach?

   • AUse Structured Streaming to score events in real-time as they arrive.
   • BServe the model using a Spark cluster with a custom REST endpoint.
   • CTrain the model again on a larger dataset to improve accuracy before serving.
   • DExport the model to a single-node format (e.g., MLflow pyfunc or ONNX) and serve it with a lightweight web server.
   Reveal Solution Discussion Select an answer first
4. 13foundation · easy

   A data scientist is planning to train a logistic regression model on a dataset of 500 million rows and 200 features. Which factor most strongly indicates that SparkML is the appropriate choice?

   • AThe model must be deployed as a REST API with sub-10ms latency.
   • BThe dataset is small (e.g., 10,000 rows) and the model is a simple linear regression.
   • CThe dataset is too large to fit in the memory of a single machine.
   • DThe model must be trained on a single GPU without distributed processing.
   Reveal Solution Discussion Select an answer first
5. 14application · medium

   A data engineer is building a pipeline that includes a StringIndexer, a VectorAssembler, and a RandomForestClassifier. They want to tune the `numTrees` parameter using CrossValidator. What is the correct way to set up the tuning?

   • ACreate a ParamGridBuilder with `numTrees` values, pass the pipeline and grid to CrossValidator, and fit it on the training data.
   • BManually loop over `numTrees` values, train a new model for each, and pick the best based on training accuracy.
   • CUse TrainValidationSplit instead of CrossValidator because it is always more accurate.
   • DUse CrossValidator but only pass the RandomForestClassifier, not the full pipeline.
   Reveal Solution Discussion Select an answer first
6. 15application · medium

   A company has a trained SparkML RandomForestRegressionModel saved to a Delta table. They need to score a batch of 10 million new records stored in a Delta table every night. The scoring job must complete within 30 minutes. What is the most appropriate approach?

   • ALoad the model and use the `transform` method on the new DataFrame in a batch Spark job.
   • BUse Structured Streaming to read the Delta table and score records as they arrive.
   • CRe-train the model on the new data before scoring.
   • DLoad the model into a single-node Python process and use pandas to score the data.
   Reveal Solution Discussion Select an answer first