How can monitoring be used to validate a self-learning system in production?

By continuously checking the system's outputs and performance metrics to detect anomalies or degradation.

By automatically fixing any defects found in the system.

By preventing the system from learning in production.

By replacing the need for any testing before deployment.

A team is testing a self-learning system for a chatbot that handles customer service. The system uses reinforcement learning to improve its responses. The team wants to test the system's ability to handle a new, unexpected type of customer query. What is the most effective testing approach?

Test the system with a set of predefined queries that are similar to the expected new type

Wait for the new query type to appear in production and then analyze the system's response

Rely on the system's self-learning capabilities to handle the new query without explicit testing

Use a simulation environment to generate a wide variety of new queries and observe the system's responses

How does data drift affect the testing of a self-learning system?

It only affects the training phase, not the testing phase.

It makes the system more deterministic.

It can cause the system's behavior to change even if the underlying code is unchanged.

It has no impact because the system is self-contained.

Why is safety a critical consideration when testing self-learning systems?

Because the system is always safe if it passes initial tests.

Because the system's behavior can change in unpredictable ways that may lead to unsafe actions.

Because safety is not a concern for AI systems.

Because safety testing is only required for systems that interact with humans.

Which traditional testing practice is most directly undermined by the adaptive nature of self-learning systems?

Performance testing with fixed load profiles.

Regression testing with a fixed set of expected outcomes.

Certified Tester AI Testing

Domain 8Objective 1

Challenges Testing Self-Learning Systems CT-AI Practice Questions (Page 1)

Part of the Domain 8: Testing AI-Specific Quality Characteristics domain, which makes up ~13% of our current practice bank. ISTQB does not publish an official question count, but from its 60-minute exam (~25–40 total, ~3–5 in this domain), expect 1–1 from this objective — we provide 25 practice questions to prepare you well beyond it. (estimate)

25questions here

5free pages

9concepts

Questions 1–5

1foundation · easy
How can monitoring be used to validate a self-learning system in production?
Select an answer first
2expert · hard
A team is testing a self-learning system for a chatbot that handles customer service. The system uses reinforcement learning to improve its responses. The team wants to test the system's ability to handle a new, unexpected type of customer query. What is the most effective testing approach?
Select an answer first
3foundation · easy
How does data drift affect the testing of a self-learning system?
Select an answer first
4foundation · easy
Why is safety a critical consideration when testing self-learning systems?
Select an answer first
5foundation · easy
Which traditional testing practice is most directly undermined by the adaptive nature of self-learning systems?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to CT-AI

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by ISTQB. “CT-AI” is a trademark of its owner, used for identification only.