Test Environments for AI-Based Systems CT-AI Practice Questions (Page 2)

Questions 6–10

1. 6expert · hard

   A team is testing a real-time recommendation model. They notice that the model's response time is sometimes very high, but they cannot determine whether the issue is caused by the model itself, the data pipeline, or the serving infrastructure. They need to pinpoint the bottleneck. What should they do?

   • AUse a load testing tool to simulate traffic and measure overall response time.
   • BIncrease the logging level to debug for all components.
   • CMonitor only the model's inference time.
   • DImplement distributed tracing that tracks a request through the data pipeline, model, and serving layer, along with detailed logging.
   Reveal Solution Discussion Select an answer first
2. 7expert · hard

   A data science team is working on a model and needs to reproduce a specific test result from three months ago. They have version control for code, but the training data has been updated multiple times since then, and the model artifacts are not versioned. What is the most likely reason they cannot reproduce the result?

   • AThe code has changed, so it is impossible to reproduce the result.
   • BThe test environment has changed, so the result cannot be reproduced.
   • CThe training data and model artifacts were not versioned, so the exact versions used in the original test are unknown.
   • DThe model's random seed was not fixed.
   Reveal Solution Discussion Select an answer first
4. 8application · medium

   A financial services company is setting up a test environment for an AI-based fraud detection system. They need to validate the model against historical transaction data, serve the model to a test application, and capture performance metrics during the test. Which set of components is essential for this test environment?

   • AA production database and a load balancer to distribute traffic.
   • BData pipeline for loading test data, a model registry to store model versions, and a monitoring dashboard for metrics.
   • CA synthetic data generator and a container registry, but no logging.
   • DA code editor and a version control system only.
   Reveal Solution Discussion Select an answer first
5. 9foundation · easy

   Why is versioning of training, validation, and test data important in an AI test environment?

   • ATo increase the size of the dataset
   • BTo automatically improve model accuracy
   • CTo ensure that experiments are reproducible and traceable
   • DTo reduce the need for data cleaning
   Reveal Solution Discussion Select an answer first
6. 10foundation · easy

   Why is logging important in an AI test environment?

   • ATo automatically improve the model's accuracy
   • BTo replace the need for test data
   • CTo capture model inputs, outputs, and system events for analysis and debugging
   • DTo increase the speed of model training
   Reveal Solution Discussion Select an answer first