1. 启动hadoop集群
接上个实验01,启动hadoop集群,启动结束后使用jps命令列出守护进程验证安装是否成功。
#启动HDFS
[hadoop@master ~]$ start-dfs.sh
#启动Yarn
[hadoop@master ~]$ start-yarn.sh
也可以直接使用start-all.sh启动Hadoop
master主节点:
[hadoop@master ~]$ jps
3717 SecondaryNameNode
3855 ResourceManager
3539 NameNode
3903 JobHistoryServer
4169 Jps
#slave1节点
[hadoop@slave1 ~]$ jps
2969 Jps
2683 DataNode
2789 NodeManager
slave2 节点
[hadoop@slave2 ~]$ jps
2614 Jps
2363 DataNode
2470 NodeManager
发现JobHistoryServer没有启动,所以需要执行
[hadoop@master hadoop-2.6.0]$ sbin/mr-jobhistory-daemon.sh start historyserver
starting historyserver, logging to /home/hadoop/local/opt/hadoop-2.6.0/logs/mapred-hadoop-historyserver-master.out
2. HDFS 命令
1)新建文件夹。
hadoop fs -mkdir /user
hadoop fs -mkdir /user/root
1)新建文件夹。
hadoop fs -mkdir /user
hadoop fs -mkdir /user/root
2)查看文件夹权限。
hadoop fs -ls -d /user/root
drwxr-xr-x - root supergroup 0 2015-05-29 17:29 /user/root
3)上传文件。
拷贝02-上机实验/ds.txt到客户端机器,运行下面的命令和结果对照
用cp命令将ds.txt从mnt目录拷贝至hadoop-2.6.0,然后在hadoop-2.6.0目录下执行
hadoop fs -put ds.txt /user/root/ds.txt
hadoop fs -ls -R /user/root
-rw-r--r-- 3 root supergroup 9135 2015-05-29 19:07 /user/root/ds.txt
4)查看文件内容。
hadoop fs -cat /user/root/ds.txt
17.759065824032646,0.6708203932499373
20.787886563063058,0.7071067811865472
17.944905786933322,0.5852349955359809
......
5)复制/移动/删除文件。
hadoop fs -cp /user/root/ds.txt /user/root/ds_backup.txt
hadoop fs -ls /user/root
Found 2 items
-rw-r--r-- 3 root supergroup 9135 2015-05-29 19:07 /user/root/ds.txt
-rw-r--r-- 3 root supergroup 9135 2015-05-29 19:30 /user/root/ds_backup.tx
hadoop fs -mv /user/root/ds_backup.txt /user/root/ds_backup1.txt
hadoop fs -ls /user/root
Found 2 items
-rw-r--r-- 3 root supergroup 9135 2015-05-29 19:07 /user/root/ds.txt
-rw-r--r-- 3 root supergroup 9135 2015-05-29 19:30 /user/root/ds_backup1.txt
hadoop fs -rm -r /user/root/ds_backup1.txt
15/05/29 19:32:51 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 0 minutes, Emptier interval = 0 minutes.
Deleted /user/root/ds_backup1.txt
hadoop fs -ls /user/root
Found 1 items
-rw-r--r-- 3 root supergroup 9135 2015-05-29 19:07 /user/root/ds.txt