Back-to-Back Testing CT-AI Practice Questions (Page 1)

Questions 1–5

1. 1foundation · easy

   After executing multiple versions with the same inputs, what is the next step in back-to-back testing?

   • ARewriting the test cases
   • BDeploying the new version to production
   • CDeleting the test inputs
   • DComparing the outputs to identify discrepancies
   Reveal Solution Discussion Select an answer first
2. 2foundation · easy

   Which purpose does back-to-back testing serve when a new version of an AI model is developed?

   • ATo validate that the new version produces outputs consistent with the previous version
   • BTo measure the accuracy of the new model against ground truth
   • CTo determine the computational efficiency of the new model
   • DTo check the security vulnerabilities of the new model
   Reveal Solution Discussion Select an answer first
4. 3foundation · easy

   What is a limitation of back-to-back testing?

   • AIt can only be used for deterministic systems
   • BIt requires a separate oracle for each version
   • CIt is only applicable to AI systems
   • DIt assumes that at least one of the versions is correct
   Reveal Solution Discussion Select an answer first
5. 4expert · hard

   A team is back-to-back testing a recommendation system. The output is a ranked list of 10 items. The old and new models produce different rankings for the same user. The team needs to decide whether the differences are acceptable. Which approach is most appropriate for comparing ranked outputs?

   • ACompare only the top-1 item and ignore the rest of the ranking.
   • BTreat any difference in ranking as a defect and require exact match.
   • CRun each model once and manually review all differences.
   • DUse a rank-based similarity metric (e.g., Kendall tau or rank-biased overlap) to quantify the difference, and investigate cases where similarity falls below a threshold.
   Reveal Solution Discussion Select an answer first
6. 5application · medium

   A team is back-to-back testing a generative model that produces images. The model uses dropout, so outputs are non-deterministic. The team wants to compare a new version against the old one. Which approach best handles the non-determinism?

   • ACompare only the images that are exactly identical and ignore the rest.
   • BDisable dropout during testing to make the models deterministic.
   • CGenerate multiple images per input with each model and compare the distributions using a perceptual similarity metric.
   • DGenerate one image per input with each model and compare them pixel-by-pixel.
   Reveal Solution Discussion Select an answer first