AI Chatbots and LLM-Powered Testing Applications for Software Testing CT-GENAI Practice Questions (Page 4)

Questions 16–20

1. 16application · medium

   A team uses an LLM to generate test cases for a financial application. The LLM produces a test case that suggests transferring a negative amount of money. What is the most important action for the team to take?

   • AAsk the LLM to generate a different test case.
   • BIgnore the test case because the LLM is clearly unreliable.
   • CImmediately add the test case to the regression suite.
   • DReview the test case for correctness and ensure it aligns with the business rules before adding it to the suite.
   Reveal Solution Discussion Select an answer first
2. 17application · medium

   A QA team uses an AI chatbot to generate test data for a new e-commerce checkout flow. The chatbot produces realistic customer names, addresses, and credit card numbers. Before using this data in the test environment, what is the most important additional step the team must take?

   • AStore the generated data in a secure vault accessible only to the QA team.
   • BEnsure the chatbot was prompted with the exact database schema of the test environment.
   • CValidate that the generated data conforms to the expected input formats and boundary conditions of the checkout forms.
   • DVerify that the credit card numbers are synthetic and do not correspond to real, active accounts.
   Reveal Solution Discussion Select an answer first
4. 18expert · hard

   A team uses an LLM-powered tool to convert legacy manual test scripts into automated Selenium code. The tool works well for simple scripts but produces flaky and unreliable code for complex ones. The team has limited time and must improve reliability. Which approach is most effective?

   • AUse the LLM to generate the code and then have a senior automation engineer review and refactor the complex scripts.
   • BIncrease the LLM's temperature setting to generate more diverse code.
   • CRetrain the LLM on a larger dataset of Selenium code.
   • DUse the LLM only for simple scripts and manually write all complex ones from scratch.
   Reveal Solution Discussion Select an answer first
5. 19expert · hard

   A company wants to use an LLM-powered application to analyze user feedback from app store reviews and automatically generate a list of potential bugs. The team is concerned about the LLM's tendency to generate false positives. Which strategy is most effective in mitigating this?

   • AUse the LLM to summarize the feedback and then have a human analyst verify each potential bug before it is added to the backlog.
   • BAutomatically add all LLM-identified bugs to the backlog to ensure nothing is missed.
   • CConfigure the LLM to only flag feedback that contains the word 'bug'.
   • DIncrease the LLM's temperature to make it more creative in identifying bugs.
   Reveal Solution Discussion Select an answer first
6. 20expert · hard

   A company is developing a chatbot to help testers generate SQL queries for test data setup. The chatbot has access to the database schema. A tester asks it to 'create a query to find all users who have not logged in for 90 days'. The chatbot returns a query that runs but returns zero rows, even though the tester knows such users exist. What is the most likely reason?

   • AThe chatbot misunderstood the schema and used the wrong column name for the last login date.
   • BThe chatbot correctly interpreted the request, but the users were deleted.
   • CThe database is temporarily down.
   • DThe tester's account does not have permission to see those users.
   Reveal Solution Discussion Select an answer first