Task Statement 3.4: Describe Methods to Evaluate FM Performance. AIF-C01 Practice Questions (Page 5)

Questions 21–25

1. 21foundation · easy

   In the context of evaluating foundation model outputs, what is the primary role of human evaluators in a human-in-the-loop evaluation process?

   • ATo generate the training data for the foundation model
   • BTo replace the need for automated evaluation metrics entirely
   • CTo automate the evaluation process using pre-defined algorithms
   • DTo assess outputs for quality, safety, and relevance based on human judgment
   Reveal Solution Discussion Select an answer first
2. 22foundation · easy

   How is task completion rate defined?

   • AThe percentage of user tasks successfully completed by the AI system
   • BThe number of errors made during task execution
   • CThe number of tasks attempted by the AI system
   • DThe time taken to complete a single task
   Reveal Solution Discussion Select an answer first
4. 23foundation · easy

   What does 'cost per interaction' measure in the context of AI applications?

   • AThe total cost of training the model
   • BThe time taken for each interaction
   • CThe number of interactions per user
   • DThe financial cost incurred for each user interaction with the AI application
   Reveal Solution Discussion Select an answer first
5. 24foundation · easy

   Which metric would be most relevant to evaluate if a business goal is to increase user engagement with an AI application?

   • AUser retention rate
   • BROUGE score
   • CPerplexity
   • DModel latency
   Reveal Solution Discussion Select an answer first
6. 25foundation · easy

   Which of the following can be used with Amazon Bedrock Model Evaluation to measure model performance?

   • ACustom metrics only
   • BOnly human evaluation
   • CBoth built-in and custom metrics
   • DBuilt-in metrics only
   Reveal Solution Discussion Select an answer first