鍍金池/ 教程/ 大數(shù)據(jù)/ MapReduce - Mapper
YARN - Failover
MapReduce - Shuffle
Hadoop配置
MapReduce - 讀取數(shù)據(jù)
Hadoop - MapReduce
YARN - Container
Hadoop 測試
MapReduce - Mapper
Hadoop - 參考
YARN - ResourceManager
HDFS - 可靠性
HDFS - 讀文件
Hadoop - 簡介
YARN - ApplicationMaster
Hadoop - IO
Hadoop 監(jiān)控
MapReduce - 編程
Hadoop - YARN
Hadoop - HDFS
HDFS - 命令工具
HDFS - 寫文件
YARN - NodeManager
Hadoop安裝

MapReduce - Mapper

主要是讀取InputSplit的每一個Key,Value對并進行處理

public class Mapper<KEYIN, VALUEIN, KEYOUT, VALUEOUT> {
    /**
     * 預(yù)處理,僅在map task啟動時運行一次
     */
    protected void setup(Context context) throws  IOException, InterruptedException {
    }

    /**
     * 對于InputSplit中的每一對<key, value>都會運行一次
     */
    @SuppressWarnings("unchecked")
    protected void map(KEYIN key, VALUEIN value, Context context) throws IOException, InterruptedException {
        context.write((KEYOUT) key, (VALUEOUT) value);
    }

    /**
     * 掃尾工作,比如關(guān)閉流等
     */
    protected void cleanup(Context context) throws IOException, InterruptedException {
    }

    /**
     * map task的驅(qū)動器
     */
    public void run(Context context) throws IOException, InterruptedException {
        setup(context);
        while (context.nextKeyValue()) {
            map(context.getCurrentKey(), context.getCurrentValue(), context);
        }
        cleanup(context);
    }
}

public class MapContext<KEYIN, VALUEIN, KEYOUT, VALUEOUT> extends TaskInputOutputContext<KEYIN, VALUEIN, KEYOUT, VALUEOUT> {
    private RecordReader<KEYIN, VALUEIN> reader;
    private InputSplit split;

    /**
     * Get the input split for this map.
     */
    public InputSplit getInputSplit() {
        return split;
    }

    @Override
    public KEYIN getCurrentKey() throws IOException, InterruptedException {
        return reader.getCurrentKey();
    }

    @Override
    public VALUEIN getCurrentValue() throws IOException, InterruptedException {
        return reader.getCurrentValue();
    }

    @Override
    public boolean nextKeyValue() throws IOException, InterruptedException {
        return reader.nextKeyValue();
    }
}
下一篇:Hadoop配置