Side Effects and Reward Hacking CT-AI Practice Questions (Page 3)

Questions 11–15

1. 11application · medium

   A team is training an AI agent to play a strategy game. The reward function gives points for capturing enemy units. The agent learns to exploit a bug in the game that allows it to capture units without actually defeating them, achieving high scores. What is the best mitigation strategy?

   • AFix the bug in the game and retrain the agent.
   • BAdd more training data to teach the agent the correct strategy.
   • CIncrease the reward for capturing units to encourage the agent to play better.
   • DUse a different game engine to avoid the bug.
   Reveal Solution Discussion Select an answer first
2. 12foundation · easy

   Which of the following is a mitigation strategy for side effects?

   • ARemoving all constraints
   • BSafe exploration techniques
   • CIncreasing the reward signal magnitude
   • DUsing a more complex model
   Reveal Solution Discussion Select an answer first
4. 13foundation · easy

   Which of the following is a real-world example of a side effect?

   • AAn AI system used for credit scoring inadvertently discriminates against a demographic group
   • BAn AI system designed to optimize energy consumption also reduces costs
   • CAn AI system for image recognition correctly identifies objects in photos
   • DAn AI system for language translation accurately translates documents
   Reveal Solution Discussion Select an answer first
5. 14foundation · easy

   How can side effects degrade the reliability of an AI-based system?

   • ABy making the system more robust to environmental changes
   • BBy causing the system to produce unexpected and potentially harmful outputs
   • CBy reducing the system's need for human oversight
   • DBy improving the system's performance on the intended task
   Reveal Solution Discussion Select an answer first
6. 15foundation · easy

   What is reward hacking in the context of AI systems?

   • AThe use of adversarial examples to fool an AI system
   • BThe unintended side effects that occur during the training of an AI model
   • CThe exploitation of a reward function to achieve high scores without fulfilling the intended objective
   • DThe process of tuning a reward function to better align with the intended objective
   Reveal Solution Discussion Select an answer first