Challenges Testing Complex AI-Based Systems CT-AI Practice Questions (Page 2)

Questions 6–10

1. 6foundation · easy

   Why is defining expected outcomes difficult for AI systems that produce probabilistic outputs?

   • ABecause the system's output is not observable.
   • BBecause the system always produces the same output for the same input.
   • CBecause the system's output is always incorrect.
   • DBecause the output is a probability distribution, so there may be no single correct answer.
   Reveal Solution Discussion Select an answer first
2. 7application · medium

   A team is testing a natural language processing system that summarizes legal documents. The system's outputs are generally accurate, but occasionally it omits important details. The team is unsure how to define a 'correct' summary because different legal experts might summarize the same document differently. What is the primary challenge the team faces?

   • ALack of a clear specification for what constitutes a correct output.
   • BThe system's model is too complex to understand.
   • CThe test data is not large enough to cover all legal document types.
   • DThe system is non-deterministic, so outputs vary between runs.
   Reveal Solution Discussion Select an answer first
4. 8application · medium

   A team is testing a deep learning model that detects tumors in medical images. The model has high accuracy on the test set, but the team is concerned about its reliability in clinical use. What is the most important additional testing activity to address the model's complexity?

   • AUse explainability techniques to understand which image features the model relies on for its predictions.
   • BIncrease the size of the test set to reduce the confidence interval.
   • CCompare the model's output to a simple rule-based system.
   • DRun the model on a GPU to speed up testing.
   Reveal Solution Discussion Select an answer first
5. 9foundation · easy

   Why is reproducing a failure in an AI system often difficult?

   • AThe system always fails in the same way for the same input.
   • BThe system's failures are always caused by hardware issues.
   • CThe system may be non-deterministic, so the same input may not produce the same failure.
   • DThe system's failures are always visible in the logs.
   Reveal Solution Discussion Select an answer first
6. 10foundation · easy

   Why is achieving adequate test coverage for an AI system challenging?

   • ATest data is always perfectly representative of the real world.
   • BCoverage is not a concern for AI systems.
   • CThe input space is often vast and high-dimensional, making it impossible to test all possible inputs.
   • DThe system only accepts a small number of discrete inputs.
   Reveal Solution Discussion Select an answer first