Learnerslesson
   JAVA   
  SPRING  
  SPRINGBOOT  
 HIBERNATE 
  HADOOP  
   HIVE   
   ALGORITHMS   
   PYTHON   
   GO   
   KOTLIN   
   C#   
   RUBY   
   C++   




HDFS COMMANDS

There are a vast range of commands supported by HDFS. If you are familier with linux commands it would be much easier for you.


Local File System and HDFS(Hadoop Distributed File System)

Local file system is the file system of your own computer. Say if you are using windows, windows operating system will be having it's own way of managing files and folders. Same applies for linux.

We have also seen Hadoop has got it's own way of storing files and that is called HDFS. We have also seen, the data nodes are independent computers which gets the instruction from Name Node for storing the files. So if you think a little, a Name Node is actually using the Hadoop Distributed File System where as the Data Nodes uses it's own file system (i.e linux).


HDFS Command line

The two commands that helps us to interact with the HDFS are 'hadoop fs' and 'hdfs dfs'. The only difference is 'hdfs dfs' helps us to deal only with the HDFS file system and using 'hadoop fs' we can work with other file systems as well.



Creating a directory in HDFS

The 'mkdir' command is used to create a directory in HDFS.


Syntax :


hadoop fs -mkdir /<directory-name>



Example :

hadoop fs -mkdir /newTestDir

A directory named 'newTestDir' is created under the root directory.


Copying files from local file system to HDFS

To copy the files from local file system to HDFS 'copyFromLocal' command is used.


Syntax :


hadoop fs -copyFromLocal <local-file-path> <hdfs-file-path>


Example :

hadoop fs -copyFromLocal /user/hadoop/employee.csv /newTestDir

The above command copies employee.csv from your local file system to the newly created directory 'newTestDir' in HDFS.


Syntax :


hadoop fs -put <local-file-path> <hdfs-file-path>


Example :

hadoop fs -put /user/hadoop/employee.csv /newTestDir

The above command does the same thing. i.e. Copies employee.csv from your local file system to the newly created directory 'newTestDir' in HDFS.


Copying files from HDFS to local file system

To copy the files from HDFS to local file system 'get' command is used.


Syntax :


hadoop fs -get <hdfs-file-path> <local-file-path>


Example :

hadoop fs -get /newTestDir/employee.csv /user/hadoop/


The above command copies employee.csv from HDFS to your local file system.


Viewing a file in HDFS

To view a file in HDFS 'cat' command is used.


Syntax :


hadoop fs -cat <file-name>


Example :

hadoop fs -cat /newTestDir/employee.csv

The above command displays the contents of employee.csv in HDFS.


Displaying the contents of a directory in HDFS

To display the contents if a directory in HDFS 'ls' command is used.


Syntax :


hadoop fs -ls <file-name>


Example :

hadoop fs -ls /newTestDir

The above command displays the contents of the directory 'newTestDir' in HDFS.


Deleting all files from a directory in HDFS

To delete files from HDFS 'rm' command is used.


Syntax :


hadoop fs -rm /<directory-name>/*


Example :

hadoop fs -rm /newTestDir/*

The above command deletes all the files from HDFS.