If you are not familiar with Docker get some hands-on experience at a series of step by step Docker tutorials with Java & Springboot examples.
Step 1: https://hub.docker.com/ is a docker repository from where you can pull & push images. You can search for the images. You can see some of the below steps at: https://hub.docker.com/r/cloudera/quickstart…
You can also install it on VMWare as illustrated on the ⏯ Getting started with BigData on Cloudera.
If you are not familiar with Docker get some hands-on experience at a series of step by step Docker tutorials with Java & Springboot examples.
This tutorial is based on 09: Docker Tutorial: Cloudera on Docker via DockerHub, where Cloudera Quickstart gets installed on Docker for learning purpose.…
There are a number of CLIs that you can run from the edge node, which is the gateway node to the Hadoop cluster consisting of master & slave (aka worker) nodes. Let’s look at the different CLIs (i.e Command Line Interfaces)
Most of the CLIs listed below are in /usr/bin
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
[root@quickstart /]# ls -ltr /usr/bin | grep 'hdfs\|impala-shell\|beeline\|hive\|spark-shell\|pyspark\|spark-submit' -rwxr-xr-x 1 root root 8144 Sep 22 2015 db_archive -rwxr-xr-x 1 root root 328 Mar 23 2016 hiveserver2 -rwxr-xr-x 1 root root 321 Mar 23 2016 hive -rwxr-xr-x 1 root root 324 Mar 23 2016 beeline -rwxr-xr-x 1 root root 504 Mar 23 2016 sqoop-create-hive-table -rwxr-xr-x 1 root root 1138056 Mar 23 2016 git-upload-archive -rwxr-xr-x 1 root root 141 Mar 23 2016 spark-submit -rwxr-xr-x 1 root root 140 Mar 23 2016 spark-shell -rwxr-xr-x 1 root root 185 Mar 23 2016 pyspark -rwxr-xr-x 1 root root 192 Mar 23 2016 hdfs -rwxr-xr-x 1 root root 1856 Mar 23 2016 impala-shell |
The following command will give you the commands you can use with “hdfs”
|
1 |
[root@quickstart /]# hdfs |
“dfs” – run a filesystem command on the file systems supported in Hadoop.…
The “etc” (i.e etcetera) folder is mainly for configuration files. It’s purpose is to host various configuration files. For instance, to add a new hard drive to your system and have Linux auto-mount it on boot, you’d have to edit /etc/fstab. Key “Hadoop Cluster Configuration” files are:
This extends Docker Tutorial: BigData on Cloudera quickstart via Docker. Step 1: Run the container on a command line.
|
1 2 3 4 |
~/projects/docker-hadoop]$ docker run --hostname=quickstart. |
This extends Docker Tutorial: BigData on Cloudera quickstart via Docker. Step 1: Run the container on a command line.
|
1 2 3 |
~/projects/docker-hadoop]$ docker run --hostname=quickstart. |
CSV is a row based storage, and Parquet is columnar in nature, and it is designed from the ground up for efficient storage, compression and encoding, which gives better performance….
Login Register 100+ Free Java interview FAQs 100+ Free Big Data interview FAQs
This extends Docker Tutorial: BigData on Cloudera quickstart via Docker. Step 1: Run the container on a command line.
|
1 2 3 |
~/projects/docker-hadoop]$ docker run --hostname=quickstart. |
This extends Docker Tutorial: BigData on Cloudera quickstart via Docker. Step 1: Run the container on a command line.
|
1 2 3 |
~/projects/docker-hadoop]$ docker run --hostname=quickstart. |
This extends Docker Tutorial: BigData on Cloudera quickstart via Docker. Step 1: Run the container on a command line.
|
1 2 3 |
~/projects/docker-hadoop]$ docker run --hostname=quickstart. |