Challenges Testing Probabilistic and Non-Deterministic AI-Based Systems CT-AI Practice Questions (Page 4)

Questions 16–20

1. 16foundation · easy

   When reporting test results for a probabilistic AI system, why is it important to include confidence intervals?

   • ATo guarantee that the system will perform as expected in production.
   • BTo make the results appear more scientific.
   • CTo communicate the uncertainty associated with the measured metrics.
   • DTo hide the fact that the system is non-deterministic.
   Reveal Solution Discussion Select an answer first
2. 17foundation · easy

   Which of the following is an example of a statistical test oracle for a non-deterministic AI system?

   • AChecking that the output is a valid data type, such as a string or integer.
   • BChecking that the system's output is exactly equal to a predefined value.
   • CChecking that the average output over multiple runs falls within a specified confidence interval.
   • DChecking that the system's output is always the same across multiple runs.
   Reveal Solution Discussion Select an answer first
4. 18application · medium

   A team is testing a reinforcement-learning agent that plays a game. The agent's behavior varies between test runs even when the test environment is identical. Which of the following is the most likely source of this non-determinism?

   • AThe agent's neural network weights are fixed after training, so the behavior should be deterministic.
   • BThe agent uses an epsilon-greedy policy that randomly selects exploratory actions.
   • CThe test environment is deterministic, so the agent should behave identically every time.
   • DThe test environment has a fixed random seed, which eliminates all sources of non-determinism.
   Reveal Solution Discussion Select an answer first
5. 19foundation · easy

   What should be communicated to stakeholders when reporting the results of testing a non-deterministic AI system?

   • AThe statistical significance and uncertainty of the results.
   • BThe exact output values for each test case.
   • CThe internal implementation details of the AI model.
   • DOnly the final pass/fail status of the tests.
   Reveal Solution Discussion Select an answer first
6. 20foundation · easy

   Which of the following is an example of environmental variability that can cause non-determinism in an AI system?

   • AThe AI model uses a fixed learning rate throughout training.
   • BThe AI model's weights are initialized with the same random seed.
   • CThe AI system runs on different hardware with different floating-point precision.
   • DThe AI system always receives inputs in the same order.
   Reveal Solution Discussion Select an answer first