02: Java to write from/to Local to HDFS File System

This extends Hadoop MapReduce Basic Tutorial and Apache Hadoop HDFS Tutorial. This could have have been done on the command-line as shown below after running “start-dfs.sh” to start the name and data nodes.

The focus of this tutorial is to do the same via Java and Hadoop APIs.

Step 1: The pom.xml file needs to have

Step 2: The HDFS name and data nodes are started as per Apache Hadoop HDFS Tutorial.

Step 3: The stand-alone Java class to copy a file from Local Unix file system to HDFS file system.

Step 4: You can verify of the file has been copied to the HDFS file system on the Unix command line with

Step 5: Now to write from HDFS File system to Unix Local File System.

Step 6: You can rename an HDFS File System file as shown below.

same as doing the folloing on the command-line.

Step 7: You can delete an HDFS file as

Step 8: Get the HDFS file status info.

Step 9: We know that in HDFS a single file may be stored in blocks at different nodes. To ge the file block info

Step 10: List hostnames of the data nodes.


800+ Java & Big Data Interview Q&As

Top