Back-to-Back Testing CT-AI Practice Questions (Page 2)

Questions 6–10

1. 6application · medium

   A team is back-to-back testing a new version of a translation model. They have a large corpus of text in multiple languages. The old and new models produce translations. Which set of test inputs would be most appropriate for this back-to-back test?

   • AA random sample of texts from the corpus without any filtering.
   • BOnly the texts that the old model translated correctly in previous testing.
   • COnly the shortest texts to minimize the time needed for comparison.
   • DA diverse set of texts covering different languages, topics, lengths, and difficulty levels.
   Reveal Solution Discussion Select an answer first
2. 7foundation · easy

   Which challenge is associated with back-to-back testing of AI systems?

   • ADefining what constitutes an acceptable difference between outputs
   • BObtaining a labeled dataset for each version
   • CGuaranteeing that the test inputs are diverse enough to cover all possible defects
   • DEnsuring that all versions are executed on different hardware
   Reveal Solution Discussion Select an answer first
4. 8expert · hard

   A team is back-to-back testing a new version of an object detection model. The old model was trained on images from North America, and the new model was trained on a more diverse global dataset. The team runs both models on a test set of images from Europe. They find that the models disagree on 20% of images, and the new model is more accurate on a small labeled subset. What is the most important limitation of this back-to-back test?

   • AThe disagreement rate of 20% is too high, so the new model should be rejected.
   • BThe test set may not be representative of all regions where the model will be deployed, so the disagreement rate may not generalize.
   • CThe old model is the reference, so any disagreement means the new model is wrong.
   • DThe new model's higher accuracy on the labeled subset is irrelevant because back-to-back testing does not use labels.
   Reveal Solution Discussion Select an answer first
5. 9application · easy

   A team is back-to-back testing a speech-recognition model. They have a large corpus of audio files. The old and new models produce text transcripts. Which sequence of steps best follows the back-to-back testing process?

   • ACompare the models' accuracy on a labeled test set and choose the one with higher accuracy.
   • BRun the new model on all audio files, then manually transcribe a random subset to create ground truth.
   • CSelect a diverse set of audio inputs, run both models on each input, then compare the transcripts for discrepancies.
   • DRun the old model first, then use its transcripts to train the new model.
   Reveal Solution Discussion Select an answer first
6. 10expert · hard

   A team is back-to-back testing a new fraud-detection model against the current production model. The new model is more accurate on historical labeled data. During testing, the two models disagree on 5% of cases. The team discovers that in most disagreements, the new model flags transactions the old model approved, and manual review shows the new model is correct. However, in a small subset, the new model approves transactions the old model flagged. What is the most important limitation the team must address before concluding the new model is safe to deploy?

   • AThe old model may be wrong in the cases where it approved transactions that the new model flags, so the new model could be correct there too.
   • BThe new model's higher accuracy on historical data is irrelevant because back-to-back testing does not use labels.
   • CThe team should deploy the new model immediately because it is more accurate on historical data.
   • DThe 5% disagreement rate is too low to be statistically significant, so the test is inconclusive.
   Reveal Solution Discussion Select an answer first