Specialized Testing Considerations CT-AI Practice Questions (Page 7)

Questions 31–35

1. 31foundation · easy

   Which testing strategy is most effective for detecting automation bias in an AI-based system?

   • ATesting only with data that the AI model has already seen during training.
   • BIncluding test cases where the AI output conflicts with other available information.
   • CEnsuring the AI system is tested only by automated tools.
   • DRemoving all human involvement from the testing process.
   Reveal Solution Discussion Select an answer first
2. 32foundation · easy

   How does documentation support traceability in AI-based systems?

   • ABy making the AI system run faster.
   • BBy linking requirements to test cases and model versions.
   • CBy reducing the need for human testers.
   • DBy automatically fixing defects in the AI model.
   Reveal Solution Discussion Select an answer first
4. 33application · medium

   An AI system for credit scoring has been in production for a year. The test team detects concept drift and needs to update the system to maintain its accuracy. Which strategy is most appropriate for handling concept drift in this context?

   • ARetrain the model using recent data that reflects the current concept, and update the test suite with new test cases based on recent data.
   • BRevert to a previous version of the model that performed well in the past.
   • CKeep the current model and adjust the decision threshold to improve accuracy.
   • DIncrease the amount of data collected from the current production environment before making any changes.
   Reveal Solution Discussion Select an answer first
5. 34foundation · easy

   What is the primary goal of testing for the impact of concept drift on an AI system?

   • ATo determine how much the model's performance degrades when the underlying data relationship changes.
   • BTo increase the model's accuracy on the training data.
   • CTo reduce the size of the test suite.
   • DTo prove that concept drift never occurs in production.
   Reveal Solution Discussion Select an answer first
6. 35application · medium

   A team is testing an AI-based system that automatically classifies support tickets. The testers are instructed to verify the AI's classification before proceeding. However, they often skip this verification because the AI has been correct in most cases. What is the most appropriate way to describe this behavior?

   • AOverfitting, where the AI model has memorized the training data too well.
   • BConfirmation bias, where testers only look for evidence that supports the AI's decisions.
   • CAutomation bias, where testers over-trust the AI's output and fail to perform independent verification.
   • DSampling bias, where the test data is not representative of the production data.
   Reveal Solution Discussion Select an answer first