MapReduce - 編程

在線練習(xí)

http://cloudcomputing.ruc.edu.cn

處理

1. select：直接分析輸入數(shù)據(jù)，取出需要的字段數(shù)據(jù)即可
2. where: 也是對(duì)輸入數(shù)據(jù)處理的過程中進(jìn)行處理，判斷是否需要該數(shù)據(jù)
3. aggregation:min, max, sum
4. group by: 通過Reducer實(shí)現(xiàn)
5. sort
6. join: map join, reduce join

Third-Party Libraries

export LIBJARS=$MYLIB/commons-lang-2.3.jar, hadoop jar prohadoop-0.0.1-SNAPSHOT.jar org.aspress.prohadoop.c3. WordCountUsingToolRunner -libjars $LIBJARS

hadoop jar prohadoop-0.0.1-SNAPSHOT-jar-with-dependencies.jar org.aspress.prohadoop.c3. WordCountUsingToolRunner The dependent libraries are now included inside the application JAR file

一般還是上面的好，指定依賴可以利用Public Cache，如果是包含依賴，則每次都需要拷貝

參考書籍

MapReduce Design Patterns