Challenges Testing Complex AI-Based Systems CT-AI Practice Questions (Page 3)

Questions 11–15

1. 11foundation · easy

   Why do traditional testing tools often fall short when testing AI systems?

   • AThey are too expensive to use for AI testing.
   • BThey are not designed to handle probabilistic outputs or large-scale data-driven validation.
   • CThey are only compatible with a single programming language.
   • DThey cannot run test cases automatically.
   Reveal Solution Discussion Select an answer first
2. 12application · medium

   A team is testing a machine learning model that predicts customer churn. The model is a gradient boosting model with hundreds of features. The team is using traditional test coverage metrics like statement and branch coverage on the model's code. Why is this approach insufficient for testing the model's behavior?

   • ACode coverage metrics are only applicable to neural networks, not gradient boosting.
   • BCode coverage metrics do not measure the quality of the model's predictions or the diversity of the test data.
   • CCode coverage metrics are not automated, making them impractical.
   • DCode coverage metrics are too expensive to compute for large models.
   Reveal Solution Discussion Select an answer first
4. 13expert · hard

   A team is testing an AI system that generates poetry. The system is non-deterministic and produces different poems each time. A tester finds a poem that is offensive and reports it as a bug. The team is unsure how to handle this because the system's output is subjective and there is no specification for what is offensive. What is the most appropriate course of action?

   • AIgnore the offensive poem because the system is non-deterministic and it may not happen again.
   • BFix the system to be deterministic so that the same poem is always generated.
   • CRemove the poem from the test results and continue testing.
   • DDefine a set of content policies and use human reviewers to evaluate outputs against them.
   Reveal Solution Discussion Select an answer first
5. 14foundation · easy

   How can bias in training data affect the testing of an AI system?

   • AIt can cause the system to perform poorly on certain groups, making test results misleading if the test data is similarly biased.
   • BIt has no effect on testing because testing uses separate data.
   • CIt only affects the training phase, not the testing phase.
   • DIt makes the system more deterministic, simplifying testing.
   Reveal Solution Discussion Select an answer first
6. 15foundation · easy

   Which skill is particularly important for a tester working on AI-based systems?

   • AExpertise in manual exploratory testing only.
   • BUnderstanding of data quality and statistical analysis.
   • CAbility to write SQL queries for database testing.
   • DKnowledge of network protocols for security testing.
   Reveal Solution Discussion Select an answer first