Model Iteration DY0-001 Practice Questions (Page 1)

Questions 1–5

1. 1foundation · easy

   A data scientist has trained a model and wants to ensure it generalizes well to unseen data. Which validation method is most appropriate to estimate the model's performance on new data?

   • AUsing the model's confidence scores
   • BEvaluating the model on the training data
   • CChecking the model's complexity
   • DUsing a separate test set that was not used during training
   Reveal Solution Discussion Select an answer first
2. 2foundation · easy

   When designing a data model for a fraud detection system, the data scientist must ensure that the model can identify rare fraudulent transactions without overwhelming the team with false alarms. Which design principle is most important to address?

   • AUsing a simple model to guarantee low false positives
   • BMaximizing the number of features to capture all patterns
   • CPrioritizing recall over all other metrics
   • DHandling class imbalance and tuning the decision threshold
   Reveal Solution Discussion Select an answer first
4. 3application · medium

   A data science team is building a model to predict customer churn. They have a dataset with 50,000 records and 30 features. The team has split the data into training (60%), validation (20%), and test (20%) sets. After training a model, they evaluate it on the validation set and achieve an AUC of 0.82. They then use the validation set to tune hyperparameters and achieve an AUC of 0.85. Finally, they evaluate the model on the test set and achieve an AUC of 0.80. What is the most likely explanation for the drop in performance from validation to test?

   • AThe model is underfitting the training data
   • BThe test set is too small to provide a reliable estimate
   • CThe model is overfitting the validation set due to hyperparameter tuning
   • DThe validation set is not representative of the test set
   Reveal Solution Discussion Select an answer first
5. 4application · medium

   A data science team is developing a fraud detection model. They split the data into training (70%), validation (15%), and test (15%) sets. After several iterations, the model achieves 99.8% accuracy on the training set and 99.5% on the validation set. However, on the test set, the accuracy drops to 97.2%. The team is considering deploying the model. What should the team do first?

   • ADeploy the model, since 97.2% test accuracy is still high and the model is production-ready
   • BUse the validation set as the test set, since it is more representative of the model's performance
   • CInvestigate the cause of the performance drop, such as overfitting or data distribution shift between validation and test sets
   • DRe-split the data and retrain the model, since the test set may not be representative
   Reveal Solution Discussion Select an answer first
6. 5foundation · easy

   A data scientist must choose between a linear regression model and a random forest model for a dataset with 10,000 samples and 50 features. The primary objective is to maximize predictive accuracy, and the dataset contains both linear and non-linear relationships. Which criterion is most important for selecting the model?

   • AThe model's ability to capture non-linear relationships
   • BThe number of features in the dataset
   • CThe interpretability of the model
   • DThe training time of the model
   Reveal Solution Discussion Select an answer first