Evaluation Metrics and Model Selection GENERATIVE-AI-ENGINEER-ASSOCIATE Practice Questions (Page 2)

Questions 6–10

1. 6application · medium

   A startup is building a real-time translation service for customer support tickets. The service must translate short messages (under 50 words) from English to Spanish with high accuracy and respond in under 500 milliseconds. The team has access to a small distilled model and a large foundation model. Which approach best balances quality and latency?

   • AUse the small distilled model for all translations, accepting lower accuracy
   • BUse the small model for common phrases and the large model for complex or rare sentences
   • CUse the large model but cache all translations to reduce average latency
   • DUse the large foundation model for all translations to maximize accuracy
   Reveal Solution Discussion Select an answer first
2. 7expert · hard

   A team is evaluating an LLM for a text generation task where the output is free-form and there is no single correct answer. The team wants a metric that measures how well the generated text matches the style and content of human-written examples. Which metric is most appropriate?

   • AROUGE
   • BHuman evaluation with a rubric
   • CPerplexity
   • DBLEU
   Reveal Solution Discussion Select an answer first
4. 8application · medium

   An e-commerce company is deploying an LLM-powered product recommendation system. The system generates personalized descriptions for each user. The team must choose between a large model that produces high-quality text but has high inference cost, and a smaller model that is cheaper and faster but produces lower-quality descriptions. The business goal is to maximize user engagement while keeping infrastructure costs within budget. Which approach is most appropriate?

   • AUse the small model for most users and the large model for high-value segments
   • BDeploy the large model for all users to maximize quality
   • CDeploy the small model for all users to minimize cost
   • DUse the large model but only during peak hours
   Reveal Solution Discussion Select an answer first
5. 9foundation · easy

   Which metric measures the average number of tokens a language model can generate per second during inference?

   • AF1 score
   • BPerplexity
   • CThroughput
   • DBLEU
   Reveal Solution Discussion Select an answer first
6. 10foundation · easy

   Which model architecture is designed to reduce inference cost by activating only a subset of parameters per token, making it suitable for large models with limited compute?

   • AMixture-of-Experts (MoE)
   • BDecoder-only model
   • CEncoder-only model
   • DDense transformer
   Reveal Solution Discussion Select an answer first