Challenges Testing Self-Learning Systems CT-AI Practice Questions (Page 3)

Questions 11–15

1. 11expert · hard

   A team is testing a self-learning system for autonomous drones. They need to validate the system's ability to navigate in a city environment. They have a high-fidelity simulator, but the 'sim-to-real' gap is a concern. The team wants to ensure the system performs well in the real world. What is the most effective testing strategy?

   • ATest exclusively in the real world with safety protocols, as it is the only way to get true results
   • BUse a combination of simulation for broad coverage and real-world testing for critical scenarios
   • CUse simulation to train the system and then deploy it without further testing
   • DTest exclusively in the simulator, as it is the only safe and scalable option
   Reveal Solution Discussion Select an answer first
2. 12application · medium

   A team is testing a recommendation engine that retrains itself nightly on new user interaction data. A regression test suite that passed on Monday fails on Tuesday, even though no code was changed. The test team suspects the model's behavior changed. What is the most likely root cause of this test failure?

   • AThe model adapted to new training data, altering its decision boundary and causing previously passing test cases to fail
   • BThe test environment's hardware configuration changed overnight, causing a difference in floating-point calculations
   • CThe test data was corrupted during the nightly batch job, leading to invalid test inputs
   • DThe model's non-deterministic training process produced a different model from the same input data
   Reveal Solution Discussion Select an answer first
4. 13foundation · easy

   What is a primary challenge that self-learning systems pose to traditional testing approaches?

   • AThe expected behavior of the system may change over time, making it difficult to define stable test oracles.
   • BThe system cannot be tested because it is too complex.
   • CThe system does not produce any output that can be verified.
   • DThe system is immune to errors because it learns from data.
   Reveal Solution Discussion Select an answer first
5. 14foundation · easy

   Which of the following is a suitable test oracle for verifying the behavior of a self-learning system?

   • AA fixed set of expected outputs that never changes.
   • BA random number generator to determine pass/fail.
   • CA human expert who can judge whether the system's output is correct in context.
   • DThe system's own previous outputs, regardless of their correctness.
   Reveal Solution Discussion Select an answer first
6. 15foundation · easy

   Which characteristic is most typical of a self-learning system?

   • AIts behavior is independent of the training data used.
   • BIts behavior evolves based on data it encounters during operation.
   • CIts behavior is fixed after initial deployment.
   • DIts behavior is fully specified in a requirements document.
   Reveal Solution Discussion Select an answer first