The Role of LLM-Powered Agents in Automating Test Processes CT-GENAI Practice Questions (Page 6)

Questions 26–30

1. 26application · medium

   A team has an LLM-powered agent that runs a suite of integration tests in a CI pipeline. The agent needs to run the tests in a specific order because some tests depend on data created by earlier tests. How should the agent be configured to handle this dependency?

   • AUse a test runner that supports ordered execution and configure the agent to invoke the tests in the defined sequence
   • BAsk the agent to rewrite the tests so they do not depend on each other
   • CConfigure the agent to run the tests in random order to ensure they are robust
   • DHave the agent run all tests in parallel to save time, ignoring the dependencies
   Reveal Solution Discussion Select an answer first
2. 27application · medium

   A QA team maintains a legacy banking application with sparse, outdated test documentation. The team wants an LLM-powered agent to generate new test cases from the existing codebase and recent pull requests. The agent must produce tests that align with the team's existing test framework and naming conventions. Which architecture component is MOST important to configure for this requirement?

   • AA retrieval-augmented generation (RAG) pipeline that indexes the codebase, pull requests, and existing test files as context for the generation prompt
   • BA standalone prompt template that instructs the model to output tests in a generic industry-standard format
   • CA rule-based script that converts each method signature directly into a single happy-path test case
   • DA fine-tuned model that has been trained exclusively on the team's historical bug reports and defect descriptions
   Reveal Solution Discussion Select an answer first
4. 28expert · hard

   A QA team uses an LLM-powered agent to analyze test failures. The agent is very good at identifying the failing assertion, but it frequently suggests root causes that are plausible-sounding yet incorrect. The team is spending more time correcting the agent than they would have spent analyzing the failures themselves. What is the MOST effective way to improve the collaboration between the team and the agent?

   • AReplace the LLM-powered agent with a simple rule-based script that matches error messages to known issues
   • BDisable the root-cause suggestion feature and have the agent only highlight the failing line of code
   • CRequire the agent to run the failing test 10 times and only report a root cause if it is consistent across all runs
   • DProvide the agent with more context, such as recent code changes and related logs, and instruct it to state its confidence level for each root-cause suggestion
   Reveal Solution Discussion Select an answer first
5. 29foundation · easy

   In the architecture of an LLM-powered testing agent, what is the primary role of the 'tool executor' component?

   • ATo decide which test cases to generate based on requirements
   • BTo generate natural language responses to user queries
   • CTo store and retrieve past test results for context
   • DTo run external tools or commands, such as executing a test script or querying a database
   Reveal Solution Discussion Select an answer first
6. 30application · medium

   An LLM-powered agent is analyzing a test failure. The test expected a 200 OK response but received a 500 Internal Server Error. The agent has access to the application logs. What is the MOST appropriate first step for the agent to take in its analysis?

   • AImmediately re-run the test to see if the failure is flaky
   • BSearch the application logs around the timestamp of the failure for stack traces or error messages
   • CModify the test to expect a 500 status code so it passes
   • DReport the failure to the developer without any additional context
   Reveal Solution Discussion Select an answer first