Hands-On Exercise: Identify Training and Test Data and Create an ML Model CT-AI Practice Questions (Page 1)

Questions 1–5

1. 1application · medium

   A machine learning engineer is building a model to predict credit risk. The dataset has 100,000 loan applications. He splits the data into training (80%) and test (20%) sets. He trains a model and evaluates it on the test set, achieving an accuracy of 95%. However, he notices that the model predicts 'low risk' for almost all applicants. Which of the following is the most likely explanation?

   • AThe test set is too small to be representative.
   • BThe dataset is imbalanced, with most applicants being 'low risk'.
   • CThe model is overfitting the training data.
   • DThe model is using too few features.
   Reveal Solution Discussion Select an answer first
2. 2application · medium

   A data analyst is building a model to predict employee attrition. The dataset includes features like age, years at company, and job satisfaction score. The target is 'attrition' (yes/no). She splits the data into training and test sets. She trains a decision tree and gets 90% accuracy on the training set and 65% accuracy on the test set. Which of the following actions is most likely to improve the model's test set performance?

   • AUse a smaller training set to reduce overfitting.
   • BRemove the test set and train on all data to improve accuracy.
   • CPrune the decision tree to reduce its complexity.
   • DIncrease the depth of the decision tree to capture more patterns.
   Reveal Solution Discussion Select an answer first
4. 3expert · hard

   A data scientist is working on a time-series dataset of daily website traffic over two years. She needs to build a model to predict future traffic. She randomly splits the data into 80% training and 20% test sets. The model performs well on the test set, but when deployed, it performs poorly on new data. Which of the following is the most likely cause of this problem?

   • AThe model is overfitting to the test set because it was evaluated multiple times.
   • BThe model is underfitting because the training set is too small.
   • CThe test set is too large, reducing the amount of data available for training.
   • DThe random split caused data leakage, where the test set contained future information that the model used during training.
   Reveal Solution Discussion Select an answer first
5. 4application · medium

   A data scientist is building a model to classify images of animals into 'cat', 'dog', or 'bird'. He has a dataset of 3,000 images. He splits the data into 70% training, 15% validation, and 15% test sets. He trains a convolutional neural network (CNN) and uses the validation set to tune hyperparameters. After tuning, he evaluates the final model on the test set. What is the primary purpose of the test set in this workflow?

   • ATo train the model's weights and biases.
   • BTo augment the training data and improve generalization.
   • CTo provide a final, unbiased evaluation of the model's performance on unseen data.
   • DTo tune the hyperparameters of the model during training.
   Reveal Solution Discussion Select an answer first
6. 5expert · hard

   A data scientist is building a model to predict house prices. The dataset contains a column 'price' with some missing values. She decides to fill the missing values with the mean price of the training set. She then splits the data into training and test sets. Which of the following is a potential problem with this approach?

   • AThe test set should be imputed with the median instead of the mean.
   • BThe mean is not a valid imputation method for numerical data.
   • CUsing the mean of the training set to fill test set missing values can leak information from the training set into the test set.
   • DThe model will be unable to handle missing values in the test set.
   Reveal Solution Discussion Select an answer first