Side Effects and Reward Hacking CT-AI Practice Questions (Page 1)

Questions 1–5

1. 1foundation · easy

   Which of the following best characterizes a side effect in an AI-based system?

   • AA behavior that is the primary goal of the system
   • BA behavior that is explicitly programmed by the developers
   • CA behavior that is beneficial to the system's users
   • DA behavior that emerges unintentionally and may cause harm
   Reveal Solution Discussion Select an answer first
2. 2foundation · easy

   Which statement best describes reward hacking?

   • AAn AI system is unable to learn due to a poorly designed reward function
   • BAn AI system achieves high reward scores by fulfilling the intended objective
   • CAn AI system causes unintended harm while pursuing its goal
   • DAn AI system exploits the reward function to get high scores without achieving the true goal
   Reveal Solution Discussion Select an answer first
4. 3foundation · easy

   Which strategy is effective for mitigating side effects in AI-based systems?

   • ACareful specification of objectives
   • BReducing the amount of training data
   • CDisabling human oversight
   • DIncreasing the learning rate
   Reveal Solution Discussion Select an answer first
5. 4application · medium

   A company deploys an AI system that controls a robotic arm in a factory. The system is trained to pick up objects, but it sometimes knocks over nearby items, causing damage. The team wants to mitigate these side effects. Which approach is most effective?

   • ADeploy the system without further changes, as side effects are unavoidable.
   • BSwitch to a different AI model architecture to avoid side effects.
   • CIncrease the training data to include more examples of successful picks.
   • DAdd a penalty to the reward function for knocking over items, and implement safe exploration during training.
   Reveal Solution Discussion Select an answer first
6. 5foundation · easy

   How can you distinguish a side effect from reward hacking?

   • ASide effects are caused by the environment, while reward hacking is caused by the system's hardware
   • BSide effects are always harmful, while reward hacking is always beneficial
   • CSide effects are unintended consequences of goal-directed behavior, while reward hacking involves gaming the reward signal
   • DSide effects are predictable, while reward hacking is unpredictable
   Reveal Solution Discussion Select an answer first