Test Environments for AI-Based Systems CT-AI Practice Questions (Page 7)

Questions 31–35

1. 31expert · hard

   A company is deploying a new version of a fraud detection model to a test environment. The model is served via a REST API. The team wants to compare the performance of the new model against the previous version using live traffic in the test environment. They also need to monitor the model's response times. What is the best approach?

   • AUse a batch process to run both models on the same data and compare outputs.
   • BDeploy the new model to a separate endpoint and manually compare results.
   • CDeploy both model versions behind the same API endpoint and use A/B testing to route traffic, while logging response times.
   • DDeploy only the new model and compare its performance to historical metrics.
   Reveal Solution Discussion Select an answer first
2. 32expert · hard · select all that apply

   A company is setting up a test environment for an AI-based recommendation system. They need to ensure that the environment supports automated testing, provides visibility into model performance, and allows for easy versioning of models. Which of the following components should they include? (Select all that apply.)

   • AA monitoring dashboard that displays model metrics and logs.
   • BA shared network drive for storing all test data without version control.
   • CA CI/CD pipeline that triggers tests on code changes.
   • DA spreadsheet to manually track test results.
   • EA model registry to store and version model artifacts.
   Check answers Discussion Select an answer first
4. 33application · medium

   An e-commerce company wants to deploy a new version of its product recommendation model to a test environment so that the QA team can evaluate it through a web interface. The model is packaged as a Docker container. What is the most efficient way to serve the model in the test environment?

   • AManually copy the model files to each QA tester's machine.
   • BUse a batch job that runs the model on a schedule and writes results to a file.
   • CEmbed the model directly into the web application code and redeploy the entire application.
   • DRun the container on a test server and expose it via a REST API endpoint that the web interface can call.
   Reveal Solution Discussion Select an answer first
5. 34foundation · easy

   Which of the following is a primary benefit of using a dedicated test environment for AI-based systems?

   • AIt allows testers to run experiments without affecting production systems or data
   • BIt guarantees that the model will perform perfectly in production
   • CIt automatically fixes all model defects found during testing
   • DIt removes the need for test data management
   Reveal Solution Discussion Select an answer first
6. 35expert · hard

   A team is testing an AI model for a self-driving car. They have a simulation environment, but they are concerned that the simulation may not accurately reflect real-world conditions. They also need to monitor the model's performance during testing. What is the best way to address this challenge?

   • AValidate the simulation against real-world data and use monitoring to compare simulation results with real-world performance.
   • BTrust the simulation results without validation.
   • CUse simulation only for edge cases and real-world for everything else.
   • DUse only real-world testing to ensure accuracy.
   Reveal Solution Discussion Select an answer first