Testing Techniques for AI Solutions AAIA Practice Questions (Page 2)

Questions 6–10

1. 6expert · hard

   A data science team has developed a churn prediction model. The model's accuracy on the test set is 92%, but the team suspects the model may be overfitting because the test set was used during hyperparameter tuning. They need to confirm that the model's performance metrics are reliable and reproducible. Which approach best addresses this concern?

   • AUse a separate holdout dataset that was not used in any training or tuning step to evaluate the final model.
   • BRe-run the model on the same test set multiple times and average the accuracy scores.
   • CIncrease the model's complexity to improve accuracy on the test set.
   • DDocument the model's architecture and training process in a report.
   Reveal Solution Discussion Select an answer first
2. 7foundation · easy

   What does performance testing assess in an AI solution?

   • AThe responsiveness, throughput, and resource utilization under expected and peak loads
   • BThe correctness of the model's predictions
   • CThe fairness of the model across demographic groups
   • DThe model's ability to handle invalid or unexpected inputs
   Reveal Solution Discussion Select an answer first
4. 8expert · hard

   A real-time trading platform uses an AI model to execute trades. The model must respond within 50 ms to market changes, and the platform is integrated with multiple external data feeds. The team is planning to test the system before a major market event. They need to ensure the model can handle the expected increase in trading volume without slowing down. Which testing approach is most appropriate?

   • APerform security testing to protect against adversarial attacks on the model.
   • BReview the model's training data for biases that could affect trading decisions.
   • CConduct functional testing to verify the model's trade recommendations are correct.
   • DRun a performance test that simulates the expected peak trading volume and measures response times and throughput.
   Reveal Solution Discussion Select an answer first
5. 9foundation · easy

   Which testing technique would you use to determine if an AI solution can handle a sudden increase in user requests without degrading response time?

   • ARobustness testing
   • BPerformance testing
   • CFunctional testing
   • DSecurity testing
   Reveal Solution Discussion Select an answer first
6. 10expert · hard

   A financial institution is deploying an AI model for fraud detection. The security team is concerned about attackers who might manipulate the training data to hide fraudulent transactions. The team also needs to document all security testing activities for regulatory compliance. Which combination of actions best addresses these concerns?

   • ADeploy the model and monitor its performance in production for anomalies.
   • BConduct security testing that includes data poisoning simulations and adversarial attack scenarios, and document the findings in a test report.
   • CPerform functional testing to verify the model correctly identifies known fraud patterns.
   • DUse a larger training dataset to improve model accuracy, and document the data sources.
   Reveal Solution Discussion Select an answer first