MapReduce Framework and Its Implementation in Hadoop DATA-SCIENCE-OPTIMIZE Practice Questions (Page 5)

Questions 21–22

1. 21application · medium

   A MapReduce job counts word frequencies across a large corpus. The job runs slowly because the reduce phase receives a huge amount of intermediate data. The data has a highly skewed distribution: a few words appear millions of times, while most appear only a few times. Which configuration change will most effectively reduce the amount of data transferred from mappers to reducers?

   • AAdd a combiner that sums counts locally in the mapper.
   • BSet mapreduce.map.output.compress to true.
   • CSet mapreduce.job.reduces to a higher number.
   • DEnable speculative execution for reduce tasks.
   Reveal Solution Discussion Select an answer first
2. 22expert · hard

   A MapReduce job is running on a cluster where one node is known to be unreliable and has failed multiple times during previous jobs. The job is critical and must complete. The administrator wants to minimize the risk of failure while not over-provisioning resources. Which action is most appropriate?

   • AEnable speculative execution for all tasks to handle potential failures.
   • BExclude the unreliable node from the cluster and resubmit the job.
   • CSet mapreduce.map.maxattempts to a high value to allow more retries.
   • DIncrease the number of reducers to reduce the impact of a node failure.
   Reveal Solution Discussion Select an answer first