Challenges Testing Probabilistic and Non-Deterministic AI-Based Systems CT-AI Practice Questions (Page 5)

Questions 21–25

1. 21foundation · easy

   Which of the following is an example of a non-deterministic AI system?

   • AA reinforcement learning agent that uses an epsilon-greedy policy to choose actions.
   • BA linear regression model that computes predictions using a fixed formula.
   • CA rule-based expert system that uses a fixed set of if-then rules to make decisions.
   • DA database query that returns the same result every time it is executed with the same parameters.
   Reveal Solution Discussion Select an answer first
2. 22foundation · easy

   What is the purpose of using a confidence interval when testing a probabilistic AI system?

   • ATo determine the exact number of test cases needed for the system.
   • BTo eliminate the need for hypothesis testing.
   • CTo estimate a range of values within which the true metric (e.g., accuracy) is likely to fall.
   • DTo guarantee that the system's output is always correct.
   Reveal Solution Discussion Select an answer first
4. 23foundation · easy

   What is a 'flaky test' in the context of testing AI systems?

   • AA test that fails consistently due to a bug in the code.
   • BA test that passes and fails intermittently without code changes.
   • CA test that only runs on certain days of the week.
   • DA test that is too slow to run in a continuous integration pipeline.
   Reveal Solution Discussion Select an answer first
5. 24foundation · easy

   Which of the following is a common source of non-determinism in AI systems?

   • AStochastic gradient descent with random batch selection.
   • BDeterministic algorithms like quicksort with a fixed pivot selection.
   • CA system that always uses the same hardware and software environment.
   • DUsing a fixed random seed for all random operations.
   Reveal Solution Discussion Select an answer first
6. 25foundation · easy

   Which quality characteristic is concerned with how an AI system performs under adversarial or unexpected inputs?

   • ARobustness
   • BExplainability
   • CReliability
   • DFairness
   Reveal Solution Discussion Select an answer first