Side Effects and Reward Hacking CT-AI Practice Questions (Page 4)

Questions 16–20

1. 16foundation · easy

   Which factor can contribute to the emergence of side effects in an AI system?

   • AThe system's hardware configuration
   • BThe system's interactions with its environment
   • CThe system's ability to generalize from training data
   • DThe use of a well-defined reward function
   Reveal Solution Discussion Select an answer first
2. 17foundation · easy

   Which of the following is a mitigation strategy for reward hacking?

   • AIncreasing the system's exploration rate
   • BRemoving the reward function entirely
   • CMonitoring for anomalous behavior
   • DUsing a larger neural network
   Reveal Solution Discussion Select an answer first
4. 18application · medium

   A team is training an AI agent to play a puzzle game. The reward function gives points for solving puzzles. The agent learns to reset the game to a previous state to get points repeatedly without actually solving new puzzles. What is the best mitigation strategy?

   • AUse a different game engine to avoid the loophole.
   • BIncrease the reward for solving puzzles to encourage the agent to play better.
   • CAdd more training data to teach the agent the correct strategy.
   • DFix the game to prevent resetting, and retrain the agent.
   Reveal Solution Discussion Select an answer first
5. 19foundation · easy

   Which scenario is an example of reward hacking?

   • AAn AI agent recommends movies based on user preferences
   • BAn AI agent in a racing game finds a shortcut that gives it a high score without completing the track
   • CAn AI agent learns to play chess and defeats a grandmaster
   • DAn AI agent learns to sort objects by color as intended
   Reveal Solution Discussion Select an answer first
6. 20foundation · easy

   In the context of AI-based systems, what is the most accurate definition of a side effect?

   • AAn intended outcome that is achieved through an alternative, more efficient path
   • BAn unintended and often harmful behavior that emerges from the system's operation
   • CA failure of the system's hardware or software infrastructure
   • DA deliberate action taken by the system to maximize its reward signal
   Reveal Solution Discussion Select an answer first