介绍:
- HDFS 文件系统提供了相当多的shell 操作命令,大大方便了程序员和系统管理人员查看、修改HDFS 上的文件。进一步,HDFS 的操作命令和Unix/Linux 的命令名称和格式相当一致,因而学习HDFS 命令的成本也大为缩小。
1.命令帮助
[hadoop@hadoop000 ~]$ hdfs dfs Usage: hadoop fs [generic options] [-appendToFile <localsrc> ... <dst>] [-cat [-ignoreCrc] <src> ...] [-checksum <src> ...] [-chgrp [-R] GROUP PATH...] [-chmod [-R] <MODE[,MODE]... | OCTALMODE> PATH...] [-chown [-R] [OWNER][:[GROUP]] PATH...] [-copyFromLocal [-f] [-p] [-l] [-d] <localsrc> ... <dst>] [-copyToLocal [-f] [-p] [-ignoreCrc] [-crc] <src> ... <localdst>] [-count [-q] [-h] [-v] [-t [<storage type>]] [-u] [-x] <path> ...] [-cp [-f] [-p | -p[topax]] [-d] <src> ... <dst>] [-createSnapshot <snapshotDir> [<snapshotName>]] [-deleteSnapshot <snapshotDir> <snapshotName>] [-df [-h] [<path> ...]] [-du [-s] [-h] [-x] <path> ...] [-expunge] [-find <path> ... <expression> ...] [-get [-f] [-p] [-ignoreCrc] [-crc] <src> ... <localdst>] [-getfacl [-R] <path>] [-getfattr [-R] {-n name | -d} [-e en] <path>] [-getmerge [-nl] [-skip-empty-file] <src> <localdst>] [-help [cmd ...]] [-ls [-C] [-d] [-h] [-q] [-R] [-t] [-S] [-r] [-u] [<path> ...]] [-mkdir [-p] <path> ...] [-moveFromLocal <localsrc> ... <dst>] [-moveToLocal <src> <localdst>] [-mv <src> ... <dst>] [-put [-f] [-p] [-l] [-d] <localsrc> ... <dst>] [-renameSnapshot <snapshotDir> <oldName> <newName>] [-rm [-f] [-r|-R] [-skipTrash] [-safely] <src> ...] [-rmdir [--ignore-fail-on-non-empty] <dir> ...] [-setfacl [-R] [{-b|-k} {-m|-x <acl_spec>} <path>]|[--set <acl_spec> <path>]] [-setfattr {-n name [-v value] | -x name} <path>] [-setrep [-R] [-w] <rep> <path> ...] [-stat [format] <path> ...] [-tail [-f] <file>] [-test -[defsz] <path>] [-text [-ignoreCrc] <src> ...] [-touchz <path> ...] [-truncate [-w] <length> <path> ...] [-usage [cmd ...]] # 注:hdfs dfs等同于hadoop fs
2.文件操作命令案例
hdfs dfs -copyFromLocal /local/data /hdfs/data:将本地文件上传到 hdfs 上(原路径只能是一个文件)
hdfs dfs -put /tmp/ /hdfs/ :和 copyFromLocal 区别是,put 原路径可以是文件夹等
hadoop fs -ls / :查看根目录文件
hadoop fs -ls /tmp/data:查看/tmp/data目录
hadoop fs -cat /tmp/a.txt :查看 a.txt,与 -text 一样
hadoop fs -mkdir dir:创建目录dir
hadoop fs -get /aaa/jdk.tar.gz :等同于copyToLocal,就是从hdfs下载文件到本地
hadoop fs -df -h / :统计文件系统的可用空间信息
hadoop fs -cp /aaa/jdk.tar.gz /bbb/jdk.tar.gz.2 :从hdfs的一个路径拷贝hdfs的另一个路径
hadoop fs -mv /aaa/jdk.tar.gz / :在hdfs目录中移动文件
hadoop fs -du -s -h /aaa/* :统计文件夹的大小信息
hadoop fs -rm -r /aaa/bbb/ :删除文件或文件夹
hadoop fs -rmdir /aaa/bbb/ccc :删除目录
hadoop fs -tail /weblog/access_log.1 :显示一个文件的末尾
3.管理命令
# 命令帮助 [hadoop@hadoop000 ~]$ hdfs dfsadmin Usage: hdfs dfsadmin Note: Administrative commands can only be run as the HDFS superuser. [-report [-live] [-dead] [-decommissioning]] [-safemode <enter | leave | get | wait>] [-saveNamespace] [-rollEdits] [-restoreFailedStorage true|false|check] [-refreshNodes] [-setQuota <quota> <dirname>...<dirname>] [-clrQuota <dirname>...<dirname>] [-setSpaceQuota <quota> [-storageType <storagetype>] <dirname>...<dirname>] [-clrSpaceQuota [-storageType <storagetype>] <dirname>...<dirname>] [-finalizeUpgrade] [-rollingUpgrade [<query|prepare|finalize>]] [-refreshServiceAcl] [-refreshUserToGroupsMappings] [-refreshSuperUserGroupsConfiguration] [-refreshCallQueue] [-refresh <host:ipc_port> <key> [arg1..argn] [-reconfig <datanode|...> <host:ipc_port> <start|status|properties>] [-printTopology] [-refreshNamenodes datanode_host:ipc_port] [-deleteBlockPool datanode_host:ipc_port blockpoolId [force]] [-setBalancerBandwidth <bandwidth in bytes per second>] [-getBalancerBandwidth <datanode_host:ipc_port>] [-fetchImage <local directory>] [-allowSnapshot <snapshotDir>] [-disallowSnapshot <snapshotDir>] [-shutdownDatanode <datanode_host:ipc_port> [upgrade]] [-evictWriters <datanode_host:ipc_port>] [-getDatanodeInfo <datanode_host:ipc_port>] [-metasave filename] [-triggerBlockReport [-incremental] <datanode_host:ipc_port>] [-help [cmd]]
hdfs dfsadmin -report :显示所有dataNode
hdfs dfsadmin -safemode leave :离开安全模式