使用hadoop做wordcount笔记

xiaoxiao2021-02-28  84

以前写的mapreduce的wordcount,都忘了怎么执行了

hadoop jar /home/dmc/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.6.0.jar -mapper mapper1.py -file mapper1.py -reducer reducer1.py -file reducer1.py -input /user/hujianqiu/wordcount/input_word/* -output output_word

其中,调用的jar包可不是随便找一个的,而是要用本机安装的hadoop-streaming的jar包,因而要找一下hadoop安装在哪里,可以使用如下命令:

locate *share/hadoop/tools/lib/hadoop-streaming*

这样就能很快地定位到hadoop的安装目录了

转载请注明原文地址: https://www.6miu.com/read-77418.html

最新回复(0)