Hadoop
Master/Slave model
HDFS namenode/datanode
hadoop fs -ls XXX / hls XXX
hadoop fs -rmr XXX
hadoop fs -cat XXX
hadoop fs -get XXX
hadoop job -list
hadoop job -kill XXX
Map/Reduce
jobtracker/tasktracker
Map => Shullfe => Reduce
Pipe
stdout => stdin
less XXX | ./map | ./reduce
Shullfe
copysortmerge
same key in same reduce task
Log Analysis
out of memory???
mapred.max.split.sizemapred.reduce.tasks
how to balance time of map and reduce???
