Scaling and Tuning MACHINE-LEARNING-PROFESSIONAL Practice Questions (Page 5)

Questions 21–23

1. 21application · medium

   A team is using Ray Tune to tune a PyTorch model. They want to use early stopping to save resources. Which scheduler should they use?

   • AHyperBandScheduler
   • BFIFOScheduler
   • CMedianStoppingRule
   • DASHAScheduler
   Reveal Solution Discussion Select an answer first
2. 22foundation · easy

   When using the Pandas Function API for inference on a large Spark DataFrame, what is the key benefit compared to using a standard UDF?

   • AIt guarantees that the inference results are deterministic and reproducible.
   • BIt eliminates the need to specify an output schema.
   • CIt allows vectorized operations on batches of rows, improving performance and reducing Python overhead.
   • DIt automatically caches the entire DataFrame in memory on the driver.
   Reveal Solution Discussion Select an answer first
3. 23application · medium

   A retail company trains a separate churn model for each of its 50 store regions. Each region has its own DataFrame with customer features. The data for all regions is stored in a single Delta table with a 'region_id' column. The data science team wants to train 50 logistic regression models in parallel using Spark's distributed processing, while keeping the training code simple and avoiding manual partitioning. Which approach should they use?

   • AUse repartition(50) on the DataFrame and then call fit() on each partition separately.
   • BUse the Pandas Function API with groupBy('region_id').applyInPandas() to train a model per group.
   • CUse SparkML's LogisticRegression on the entire dataset with a 'region_id' feature column.
   • DUse a UDF that iterates over each region_id and trains a model sequentially within a single executor.
   Reveal Solution Discussion Select an answer first