Back-to-Back Testing CT-AI Practice Questions (Page 6)

Questions 26–29

1. 26application · medium

   A company has two teams that independently implemented the same AI-based sentiment-analysis feature using different libraries. They want to ensure both implementations behave consistently before offering them to customers. They have no labeled dataset. What is the most appropriate testing approach?

   • ARun both implementations on the same set of diverse text inputs and compare their sentiment outputs.
   • BAsk each team to write unit tests for their own implementation and ensure all tests pass.
   • CDeploy both implementations and compare customer satisfaction scores.
   • DManually label a small dataset and evaluate both implementations on it.
   Reveal Solution Discussion Select an answer first
2. 27foundation · easy

   Why might multiple runs be necessary when back-to-back testing an AI system with non-deterministic behavior?

   • ATo ensure the system produces the same output every time
   • BTo increase the number of test cases
   • CTo reduce the execution time of the tests
   • DTo obtain a more reliable basis for comparing outputs
   Reveal Solution Discussion Select an answer first
4. 28expert · hard

   A team is back-to-back testing a new version of a text classification model. The output is a probability distribution over classes. The old and new models produce different probability distributions for the same input. The team needs to decide if the differences are acceptable. Which approach is most appropriate?

   • ACompare only the predicted class (argmax) and ignore the probability values.
   • BCompare the probability distributions using a divergence metric (e.g., KL divergence) and investigate cases where divergence exceeds a threshold.
   • CRun each model once and compare the distributions visually.
   • DRequire the probability distributions to be exactly identical.
   Reveal Solution Discussion Select an answer first
5. 29application · medium

   A team has two versions of a chatbot: one rule-based and one neural. They want to verify that the neural version does not produce offensive responses on inputs the rule-based version handled safely. They have no labeled data for what constitutes 'offensive'. What is the most appropriate way to use back-to-back testing?

   • ARun both chatbots on a diverse set of inputs and flag cases where the neural version's response differs significantly from the rule-based version's response.
   • BRun only the neural chatbot on a large corpus and manually review all responses for offensiveness.
   • CUse a sentiment analysis model to score the neural chatbot's responses and flag negative ones.
   • DDeploy the neural chatbot and monitor user complaints.
   Reveal Solution Discussion Select an answer first