Scaling and Tuning MACHINE-LEARNING-PROFESSIONAL Practice Questions (Page 3)

Questions 11–15

1. 11foundation · easy

   A machine learning workload is running on a Databricks cluster and is hitting a memory bottleneck on the driver node. The team is considering switching to a larger driver node type. What type of scaling is this?

   • AData parallelism
   • BHorizontal scaling
   • CVertical scaling
   • DModel parallelism
   Reveal Solution Discussion Select an answer first
2. 12foundation · easy

   A team needs to perform distributed training of a custom PyTorch model that requires fine-grained control over the training loop and communication between workers. Which framework is better suited for this task compared to Spark?

   • APandas UDF
   • BSpark MLlib
   • CDatabricks SQL
   • DRay
   Reveal Solution Discussion Select an answer first
4. 13foundation · easy

   In Ray Tune, what is the purpose of a 'search algorithm' (e.g., BayesOptSearch, HyperOptSearch)?

   • AIt stops trials early if they are performing poorly.
   • BIt defines the range of values for each hyperparameter.
   • CIt is responsible for distributing the trials across the cluster.
   • DIt intelligently suggests the next set of hyperparameters to try, based on the results of previous trials.
   Reveal Solution Discussion Select an answer first
5. 14foundation · easy

   A team needs to train a separate model for each of 500 product categories using a large dataset. They want to leverage Spark's distributed execution without writing custom RDD code. Which API is designed for this exact use case?

   • AThe MLlib Pipeline API
   • BRDD.map() with a custom function
   • CPandas Function API (applyInPandas / mapInPandas)
   • DSpark SQL UDFs
   Reveal Solution Discussion Select an answer first
6. 15expert · hard

   A team is using Optuna to tune a model on a Databricks cluster. They have a large search space and want to run 100 trials. They have a cluster with 10 worker nodes, each with 8 cores. They want to minimize the wall-clock time. They are considering two approaches: (1) run Optuna on the driver with n_jobs=8, or (2) use Optuna's distributed mode with a MySQL database and run 10 workers (one per node) each with n_jobs=8. What is the primary trade-off?

   • AApproach 2 is always better because it uses more resources.
   • BApproach 1 is faster because it avoids network overhead; approach 2 is slower due to database communication.
   • CApproach 1 is simpler but uses only the driver's resources; approach 2 uses all nodes but requires setting up a database.
   • DBoth approaches are equivalent in terms of speed.
   Reveal Solution Discussion Select an answer first