Evaluation Judges and Ground Truth GENERATIVE-AI-ENGINEER-ASSOCIATE Practice Questions (Page 1)

Questions 1–5

1. 1application · medium

   A team is evaluating a machine translation model. They have a set of source sentences and their human-translated reference translations. They want to measure how closely the model's translations match the references. Which evaluation judge should they use?

   • ASelf-evaluation by the model
   • BExact match
   • CBLEU
   • DReference-free LLM-as-judge
   Reveal Solution Discussion Select an answer first
2. 2application · easy

   A team is building a question-answering system for a medical FAQ. They have a set of questions with verified answers from medical experts. They want to evaluate the system's accuracy. What should they use as ground truth?

   • AThe system's previous responses
   • BThe verified answers from medical experts
   • CThe questions themselves
   • DA random sample of general medical articles
   Reveal Solution Discussion Select an answer first
4. 3application · easy

   A team is building a model that generates responses to customer feedback. They have a set of feedback items with expected responses from customer service experts. They want to evaluate the model's performance. What should they use as ground truth?

   • AThe expected responses from customer service experts
   • BThe model's previous responses
   • CA random sample of customer service transcripts
   • DThe feedback items themselves
   Reveal Solution Discussion Select an answer first
5. 4foundation · easy

   How does the availability of ground truth influence the choice of evaluation judge?

   • AIf ground truth is available, you can use reference-based judges like exact match or ROUGE; if not, you must use reference-free judges like self-evaluation
   • BGround truth is only needed for LLM-as-judge, not for other metrics
   • CGround truth availability does not affect judge selection; any judge can be used regardless
   • DGround truth is only needed for self-evaluation, not for other judges
   Reveal Solution Discussion Select an answer first
6. 5application · medium

   A company is developing a creative writing assistant that generates short stories. They have no reference stories to compare against. They want to evaluate the creativity and engagement of the generated stories. Which evaluation approach is most suitable?

   • AUse BLEU to compare the stories against a corpus of published stories.
   • BUse exact match to check if the stories match any known story.
   • CUse a reference-free LLM-as-judge to rate the stories on creativity and engagement.
   • DUse ROUGE to compare the stories against a set of classic short stories.
   Reveal Solution Discussion Select an answer first