MapReduce Framework and Its Implementation in Hadoop DATA-SCIENCE-OPTIMIZE Practice Questions (Page 1)

Questions 1–5

1. 1expert · hard

   A MapReduce job is running slower than expected. The job has a high number of map tasks, and each map task processes a small amount of data. The cluster has ample memory and CPU. The administrator suspects that the overhead of starting and stopping many map tasks is the bottleneck. Which configuration change is most likely to improve performance?

   • ASet mapreduce.input.fileinputformat.split.minsize to a larger value.
   • BEnable speculative execution for map tasks.
   • CIncrease the number of reducers to balance the load.
   • DSet mapreduce.map.memory.mb to a higher value.
   Reveal Solution Discussion Select an answer first
2. 2expert · hard

   A MapReduce job reads a large number of small files (e.g., thousands of files of a few KB each) from HDFS. The job is slow due to the overhead of processing many small splits. The administrator wants to improve performance without changing the data. Which approach is most effective?

   • ASet mapreduce.input.fileinputformat.split.minsize to a small value.
   • BUse CombineFileInputFormat to group small files into larger splits.
   • CIncrease the number of reducers to process more data in parallel.
   • DEnable speculative execution for map tasks.
   Reveal Solution Discussion Select an answer first
4. 3foundation · easy

   How does Hadoop MapReduce handle a task that fails due to a node crash?

   • AIt uses a backup copy of the task's output stored on HDFS.
   • BIt ignores the failure and continues without the task's output.
   • CIt re-executes the failed task on another available node.
   • DIt restarts the entire job from the beginning.
   Reveal Solution Discussion Select an answer first
5. 4expert · hard

   A MapReduce job performs a join between a large dataset and a small lookup table. The small table is 50 MB and is stored in HDFS. The job currently uses a reduce-side join, which is slow due to the shuffle of the large dataset. The administrator wants to reduce the shuffle data. The cluster has enough memory to hold the small table in each mapper's memory. Which approach is most effective?

   • AEnable compression for the map output.
   • BIncrease the number of reducers to reduce the data per reducer.
   • CUse a custom partitioner to send all keys for the small table to a single reducer.
   • DUse a map-side join by loading the small table into the distributed cache.
   Reveal Solution Discussion Select an answer first
6. 5application · medium

   A Hadoop cluster runs a long MapReduce job. One node in the cluster becomes slow due to a failing disk, causing a few map tasks to run much slower than others. The job is nearing its deadline. Which action should the administrator take to ensure the job completes on time?

   • ASet mapreduce.map.speculative to false to avoid wasting resources.
   • BKill the slow node and resubmit the job.
   • CEnable speculative execution for map tasks.
   • DIncrease the number of reducers.
   Reveal Solution Discussion Select an answer first