AI Model Training, Testing, and Validation AAIR Practice Questions (Page 4)

Questions 16–20

1. 16expert · hard

   A marketing firm's customer-churn model shows excellent performance on the training set but poor performance on the validation set. The team increased model complexity to capture more patterns, but validation performance worsened. Which combination of issues is most likely occurring?

   • AOverfitting due to high variance, worsened by increasing model complexity
   • BData leakage causing inflated training performance
   • CUnderfitting due to high bias, worsened by increasing model complexity
   • DConcept drift between the training and validation periods
   Reveal Solution Discussion Select an answer first
2. 17application · medium

   A healthcare startup has only 4,000 labeled patient records for training a sepsis-prediction model. The team needs a reliable estimate of model performance to present to regulators, but they also want to use as much data as possible for training. Which approach best satisfies both requirements?

   • AUse a single 80/20 split but repeat it 10 times with different random seeds and average the results
   • BSplit the data 50/50 into training and test sets and report test performance
   • CPerform 5-fold cross-validation and report the mean and variance of the chosen metric across folds
   • DTrain on all 4,000 records and report the training accuracy as the expected performance
   Reveal Solution Discussion Select an answer first
4. 18expert · hard

   A fraud-detection model is trained on transaction data from January to June. The team evaluates it on July data and achieves 99.5% accuracy. However, after deployment in August, performance drops sharply. The risk team suspects data leakage. Which evidence would most strongly confirm this suspicion?

   • AThe model was trained using a random 80/20 split of the January-to-June data
   • BThe August data contains a new type of fraud not present in the training data
   • CThe July test data contains transactions that were manually reviewed and labeled after the model was trained
   • DThe model's training accuracy was 99.8%, slightly higher than the July test accuracy
   Reveal Solution Discussion Select an answer first
5. 19expert · hard

   A self-driving car company is testing its pedestrian-detection model. The model performs well on standard test sets but fails to detect pedestrians in foggy conditions. The safety team wants a testing strategy that identifies such weaknesses before deployment. Which approach is most effective?

   • AUse cross-validation to ensure the model performs consistently across random data splits
   • BReport only the overall accuracy on the combined test set to show high performance
   • CInclude edge-case and adversarial test sets with fog, rain, and low-light conditions
   • DIncrease the size of the standard test set to include more clear-weather images
   Reveal Solution Discussion Select an answer first
6. 20foundation · easy

   What is the purpose of adversarial testing in AI model evaluation?

   • ATo measure the model's performance on the training set
   • BTo test the model with a random sample of normal inputs
   • CTo evaluate the model's resilience against intentionally crafted inputs designed to cause errors
   • DTo compare the model's accuracy with that of a simpler baseline
   Reveal Solution Discussion Select an answer first