Identify Hallucinations, Reasoning Errors and Biases in LLM Output CT-GENAI Practice Questions (Page 4)

Questions 16–20

1. 16foundation · easy

   What is a 'flawed chain-of-thought' in the context of LLM reasoning errors?

   • AThe model's reasoning is correct but the facts are wrong.
   • BThe model provides a correct answer but with too much detail.
   • CThe model skips the reasoning process and gives an answer directly.
   • DThe model's intermediate reasoning steps contain errors that lead to a wrong conclusion.
   Reveal Solution Discussion Select an answer first
2. 17application · medium

   A test team uses an LLM to generate user personas for usability testing. The LLM consistently generates personas with predominantly male names and technical job titles, even when the prompt asks for a diverse set. The team is concerned about bias in the test data. What is the most likely impact of this bias on software testing?

   • AThe test data may not represent all user groups, leading to missed usability issues for underrepresented users
   • BThe test data will be less biased because the LLM is trained on diverse internet data
   • CThe test data will be more accurate because it focuses on the most common user profile
   • DThe test data will be more efficient because it reduces the number of personas needed
   Reveal Solution Discussion Select an answer first
4. 18application · medium

   A QA team uses an LLM to generate test scripts for a legacy system. The LLM frequently produces references to API endpoints that do not exist in the system's documentation. The team suspects the cause is related to the model's training data. Which cause is most likely?

   • AThe model was trained on outdated or incomplete documentation
   • BThe model intentionally fabricates endpoints to test the team's vigilance
   • CThe model's temperature setting is too high, causing random output
   • DThe model's architecture is too small to store all API details
   Reveal Solution Discussion Select an answer first
5. 19foundation · easy

   Which of the following is the best example of a hallucination in an LLM's output?

   • AThe model correctly states that Paris is the capital of France.
   • BThe model states that the Eiffel Tower was built in 1889, which is historically accurate.
   • CThe model claims that a well-known historical figure won a Nobel Prize that they never received.
   • DThe model says 'I don't know the answer to that question.'
   Reveal Solution Discussion Select an answer first
6. 20application · medium

   A QA team uses an LLM to generate test cases for a credit approval system. The team suspects the LLM's outputs may be biased against certain demographic groups. They want to detect this bias before using the test cases. Which technique is most appropriate?

   • AAsk the LLM to generate more test cases to increase diversity
   • BManually review a random sample of test cases for obvious bias
   • CUse a different LLM that is known to be unbiased
   • DRun fairness metrics on the generated test cases to measure demographic parity
   Reveal Solution Discussion Select an answer first