Challenges Testing Complex AI-Based Systems CT-AI Practice Questions (Page 5)

Questions 21–25

1. 21foundation · easy

   Which of the following is a unique challenge when testing AI-based systems, as opposed to traditional software?

   • AThe system's user interface must be tested for usability.
   • BThe system may produce different outputs for the same input due to non-deterministic behavior.
   • CThe system requires a database to store test data.
   • DThe system must be tested on multiple operating systems.
   Reveal Solution Discussion Select an answer first
2. 22foundation · easy

   How does continuous learning in an AI system affect debugging?

   • AIt ensures that all failures are caused by the same bug.
   • BIt makes the system more stable and easier to debug.
   • CIt has no effect on debugging.
   • DThe model's behavior changes over time, so a failure observed today may not be reproducible tomorrow.
   Reveal Solution Discussion Select an answer first
4. 23foundation · easy

   How does the subjective nature of some AI tasks, such as image captioning, complicate the test oracle?

   • AThe test oracle is always a simple comparison to a reference output.
   • BThe system's output is always deterministic.
   • CDifferent human judges may disagree on what constitutes a correct caption.
   • DThe system cannot generate any captions.
   Reveal Solution Discussion Select an answer first
5. 24application · medium

   A team is testing an AI system that generates customer support responses. The system produces slightly different phrasings for the same input due to its probabilistic nature. The team needs to verify that responses are appropriate and helpful, but there is no single correct answer. Which approach best addresses the challenge of defining expected outcomes for this system?

   • ARun the system multiple times and only test the most frequent output.
   • BSkip testing for correctness and focus only on performance metrics like response time.
   • CDefine a set of acceptable output characteristics and use human evaluators to judge responses against them.
   • DUse a reference-based metric that compares each response to a single golden answer.
   Reveal Solution Discussion Select an answer first
6. 25application · medium

   A team is testing an AI system that assigns sentiment scores to product reviews. The system outputs a score from 1 to 5, and the team is using a test set with human-labeled sentiment scores. The team notices that human annotators often disagree on the correct score for the same review. What is the most significant implication for testing?

   • AThe test oracle is unreliable because there is no single correct sentiment score.
   • BThe test results are invalid because the system is non-deterministic.
   • CThe test results should be based on the majority label only.
   • DThe test results are valid because human labels are always correct.
   Reveal Solution Discussion Select an answer first