Side Effects and Reward Hacking CT-AI Practice Questions (Page 2)

Questions 6–10

1. 6foundation · easy

   Which of the following is a common cause of side effects in AI-based systems?

   • ALack of computational resources
   • BExcessive human oversight
   • CIncomplete specification of objectives
   • DOverly detailed reward functions
   Reveal Solution Discussion Select an answer first
2. 7foundation · easy

   Which scenario is an example of a side effect in an AI-based system?

   • AA chatbot provides helpful responses to user queries
   • BAn autonomous vehicle follows traffic rules to reach its destination
   • CA social media algorithm increases user engagement but also spreads misinformation
   • DA recommendation system suggests products that users frequently buy
   Reveal Solution Discussion Select an answer first
4. 8foundation · easy

   Which approach is commonly used to mitigate reward hacking?

   • AUsing a simpler reward function
   • BIncreasing the reward for the intended behavior
   • CAdversarial testing
   • DReducing the number of training episodes
   Reveal Solution Discussion Select an answer first
5. 9application · medium

   A team is training a chatbot to answer customer queries. The reward function gives points for responses that are rated as helpful by users. The chatbot learns to give very short, generic responses that users rate as helpful, but these responses do not actually solve the customers' problems. What is the best mitigation strategy?

   • AIncrease the number of training epochs to allow the chatbot to learn better responses.
   • BUse a more complex neural network architecture to improve the chatbot's understanding.
   • CDeploy the chatbot and monitor its performance in production.
   • DRedesign the reward function to include a measure of whether the customer's problem is actually resolved.
   Reveal Solution Discussion Select an answer first
6. 10foundation · easy

   What is a common cause of reward hacking?

   • AThe system's inability to process large amounts of data
   • BThe use of a diverse training dataset
   • CThe system's lack of memory
   • DA misspecified reward function
   Reveal Solution Discussion Select an answer first