LLMOps When Deploying and Managing LLMs for Software Testing CT-GENAI Practice Questions (Page 4)

Questions 16–20

1. 16application · medium

   A team is using an LLM to generate test data for a banking application. They are concerned about the model producing biased or unfair test cases that could lead to discriminatory outcomes. What is the most effective way to address this concern?

   • ARely on the LLM provider's ethical guidelines.
   • BImplement a governance policy that requires bias testing and fairness evaluation of the LLM outputs before they are used in test suites.
   • CAdd a disclaimer to the test reports that the data may be biased.
   • DUse a larger, more powerful LLM to reduce bias.
   Reveal Solution Discussion Select an answer first
2. 17foundation · easy

   Which security measure is essential when deploying an LLM that processes sensitive test data?

   • AEncrypting data in transit and at rest
   • BUsing a public API for easy access
   • CStoring test data in plain text for simplicity
   • DDisabling logging to prevent data leaks
   Reveal Solution Discussion Select an answer first
4. 18expert · hard

   An LLM is used to generate test cases for a critical financial application. The team needs to ensure that the generated test cases are not only accurate but also cover edge cases and boundary conditions. Which evaluation approach is most appropriate?

   • AUse a combination of automated metrics (e.g., coverage analysis) and human review.
   • BUse a single automated metric like BLEU to evaluate quality.
   • CCompare the generated test cases to a set of manually written test cases using string matching.
   • DRely solely on human review of all generated test cases.
   Reveal Solution Discussion Select an answer first
5. 19expert · hard

   An LLM-based test oracle is used to classify test results as 'pass' or 'fail'. The team notices that the model's accuracy on the validation set is high, but in production, the model often misclassifies results for a specific type of test case. What is the most likely cause and what should they do?

   • AThe model is biased; they should implement a fairness mitigation technique.
   • BThe model is underfitting; they should increase the model size.
   • CThe production data distribution differs from the validation set; they should monitor for data drift and retrain or fine-tune on production-like data.
   • DThe model is overfitting to the validation set; they should retrain with more data.
   Reveal Solution Discussion Select an answer first
6. 20foundation · easy

   Which step is typically included in a CI/CD pipeline for an LLM used in testing?

   • ARunning automated evaluation of the model against a validation set
   • BManually copying model files to production servers
   • CRemoving version control for the model
   • DDisabling all tests to speed up deployment
   Reveal Solution Discussion Select an answer first