Challenges Testing Complex AI-Based Systems CT-AI Practice Questions (Page 4)

Questions 16–20

1. 16foundation · easy

   What is a consequence of the 'black box' nature of complex AI models for testing?

   • ATesters can always predict the output for any input.
   • BTesting can be done without any test data.
   • CThe model is automatically transparent and explainable.
   • DTesters cannot easily understand why a particular output was produced, making it hard to identify the cause of failures.
   Reveal Solution Discussion Select an answer first
2. 17application · medium

   A company is transitioning from testing traditional software to testing an AI-based system. The existing QA team has strong skills in test automation and requirements-based testing. They are now asked to test a deep learning model that classifies customer feedback. What is the most significant gap in the team's current skill set?

   • AUsing version control to manage test scripts.
   • BUnderstanding how to validate the model's performance using appropriate metrics and statistical methods.
   • CWriting test cases that cover all possible input combinations.
   • DSetting up a test environment that mirrors production.
   Reveal Solution Discussion Select an answer first
4. 18application · medium

   A team is testing a deep neural network that detects fraudulent transactions. The model has millions of parameters and its internal decision-making process is not easily interpretable. During testing, the team finds a specific transaction that is misclassified. What is the primary challenge the team faces when trying to debug this failure?

   • AThe model may be non-deterministic, so the failure may not reproduce.
   • BThe team lacks a clear specification of expected behavior.
   • CThe test data may contain noise that caused the misclassification.
   • DThe model's complexity makes it difficult to trace which input features contributed to the misclassification.
   Reveal Solution Discussion Select an answer first
5. 19application · medium

   A team is testing a speech recognition system that transcribes customer service calls. The system sometimes produces different transcriptions for the same audio clip when run multiple times. The team is unsure whether this is a bug or expected behavior. What is the most appropriate first step to address this challenge?

   • AReport the issue as a bug and ask developers to fix it.
   • BSwitch to a different speech recognition system that is deterministic.
   • CIgnore the variations and only test the most common transcription.
   • DDetermine whether the system is designed to be non-deterministic and, if so, test using statistical methods over multiple runs.
   Reveal Solution Discussion Select an answer first
6. 20expert · hard

   A team is testing a deep learning model that predicts equipment failures in a manufacturing plant. The model is retrained weekly with new sensor data. During testing, the team observes a spike in false positives after one retraining cycle. They need to identify the cause, but the model is a black box. Which combination of actions is most effective for debugging this issue?

   • AAnalyze the training data used in the latest retraining cycle for anomalies, and use explainability tools to understand the model's predictions.
   • BSwitch to a simpler model that is easier to debug.
   • CRevert to the previous model version and assume the new data is faulty.
   • DIncrease the test set size to reduce the impact of the false positives.
   Reveal Solution Discussion Select an answer first