Hands-On Exercise:The Use of AI in Testing CT-AI Practice Questions (Page 3)

Questions 11–15

1. 11application · medium

   In a hands-on exercise, a tester uses a clustering algorithm to group test cases from a large regression suite based on code coverage patterns. The goal is to identify redundant test cases. After running the algorithm, the tester observes that one cluster contains test cases with very similar coverage but different execution times. What is the most appropriate next step to evaluate the effectiveness of this AI application?

   • AImmediately delete all test cases in the cluster to reduce the suite size.
   • BRe-run the clustering algorithm with different parameters until all clusters have exactly the same number of test cases.
   • CManually review the test cases in the cluster to confirm they are truly redundant before considering removal.
   • DAdd more test cases to the cluster to make the redundancy analysis more statistically significant.
   Reveal Solution Discussion Select an answer first
2. 12application · medium

   In a hands-on exercise, an AI model is used to classify test results as 'pass' or 'fail' based on log files. After running the model on a set of logs, the team calculates that the model has a high accuracy of 95%. However, the team is concerned because the logs contain very few actual failures. Which additional metric should the team examine to better understand the model's performance?

   • AThe number of features extracted from the log files.
   • BRecall (sensitivity) for the 'fail' class.
   • CThe time it took to train the classification model.
   • DThe total number of log files processed by the model.
   Reveal Solution Discussion Select an answer first
4. 13expert · hard

   A team is setting up a hands-on exercise to compare the effectiveness of two AI techniques for test case prioritization: a supervised learning model and a search-based algorithm. The team has a limited budget and can only fully execute one technique. They have a historical dataset with defect information and a large test suite. What is the most important consideration when designing the evaluation to ensure a fair comparison?

   • AUse the same test suite and the same evaluation metric (e.g., APFD) for both techniques.
   • BEvaluate the supervised model on the historical data it was trained on and the search-based algorithm on new data.
   • CPrioritize the speed of execution over the accuracy of the results to save time.
   • DUse different test suites for each technique to see which one handles more complex tests.
   Reveal Solution Discussion Select an answer first
5. 14expert · hard

   A team is using an AI model to generate test data for a financial application. The model is trained on a dataset of valid transactions. The generated data passes all format validation checks, but a domain expert notices that some generated transactions violate complex business rules (e.g., a transaction amount exceeding a daily limit for a specific account type). What is the most likely root cause of this issue?

   • AThe format validation checks are too strict.
   • BThe training data did not include examples of transactions that violate business rules.
   • CThe AI model is overfitting to the training data.
   • DThe model is not complex enough to learn the business rules.
   Reveal Solution Discussion Select an answer first
6. 15foundation · easy

   In a hands-on exercise where AI is to be applied to testing, which of the following is the primary purpose of preparing a clean, labeled dataset before starting the exercise?

   • ATo reduce the need for a separate validation step during the exercise
   • BTo ensure the AI model can learn patterns effectively and be evaluated reliably
   • CTo guarantee that the AI model will achieve high accuracy on the test data
   • DTo eliminate the need for human oversight during the exercise
   Reveal Solution Discussion Select an answer first