TUT - Cloudera on Docker Archives | Page 2 of 2 | 800+ Java & Big Data Interview Q&As

Blog Archives

« 1 2

24: Docker Tutorial: HBase (i.e. NoSQL DB) Shell on Cloudera quickstart

This extends Docker Tutorial: BigData on Cloudera quickstart via Docker. Step 1: Run the container on a command line.

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \
--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \
--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 \
cloudera/quickstart /usr/bin/docker-quickstart

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \

--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \

--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 \

cloudera/quickstart /usr/bin/docker-quickstart

hbase cli We are going to focus on “shell” command to create a table, column-families, and columns, … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

25: Docker Tutorial: HBase (i.e. NoSQL) Java API on Cloudera quickstart

This extends Docker Tutorial: BigData on Cloudera quickstart via Docker. Step 1: Run the container on a command line.

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \
--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \
--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 \
cloudera/quickstart /usr/bin/docker-quickstart

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \

--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \

--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 \

cloudera/quickstart /usr/bin/docker-quickstart

Install maven Step 2: Download maven from “http://mirror.ventraip.net.au/apache/maven/maven-3/3.6.1/binaries/apache-maven-3.6.1-bin.tar.gz”

[root@quickstart /]# sudo curl -L -O http://mirror.ventraip.net.au/apache/maven/maven-3/3.6.1/binaries/apache-maven-3.6.1-bin.tar.gz

1 2	[root@quickstart /]# sudo curl -L -O http://mirror.ventraip.net.au/apache/maven/maven-3/3.6.1/binaries/apache-maven-3.6.1-bin.tar.gz

[root@quickstart /]# cp apache-maven-3.6.1-bin.tar.gz /opt
[root@quickstart /]# cd /opt
[root@quickstart /]# sudo tar xvfz apache-maven-3.6.1-bin.tar.gz
[root@quickstart /]# sudo rm -f apache-maven-3.6.1-bin.tar.gz

[root@quickstart /]# cp apache-maven-3.6.1-bin.tar.gz /opt

[root@quickstart /]# cd /opt

[root@quickstart /]# sudo tar xvfz apache-maven-3.6.1-bin.tar.gz

[root@quickstart /]# sudo rm -f apache-maven-3.6.1-bin.tar.gz

Step 3: Add it to your profile so that you can run the “mvn” … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

26: Docker Tutorial: Apache Kafka install, create topic & publish message on Cloudera quickstart

This extends Docker Tutorial: BigData on Cloudera quickstart via Docker. Step 1: Run the container on a command line.

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \
--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \
--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 \
cloudera/quickstart /usr/bin/docker-quickstart

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \

--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \

--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 \

cloudera/quickstart /usr/bin/docker-quickstart

Add new user & add to sudoers Step 2: “-m” creates a home directory “/home/kafka”, and the new user will be added to “ … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

27: Docker Tutorial: Apache Kafka with Java API on Cloudera quickstart

This requires 26: Docker Tutorial: Apache Kafka install, create topic & publish message on Cloudera quickstart. It is important that you continue from the previous tutorial. Add maven path to ~/.bash_profile Step 1: Maven is already in the quickstart/cloudera Docker image.

[kafka@quickstart /]$ vi ~/.bash_profile

1	[kafka@quickstart /]$ vi ~/.bash_profile

Modify so that “mvn” … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

28: Docker Tutorial: Apache Spark streaming with Kafka in Java on Cloudera quickstart

This extends 27: Docker Tutorial: Apache Kafka with Java API on Cloudera quickstart Install Java 8 Step 1: Install Java 8.

[kafka@quickstart my-app]$ sudo yum install java-1.8.0-openjdk-devel

1	[kafka@quickstart my-app]$ sudo yum install java-1.8.0-openjdk-devel

[kafka@quickstart my-app]$ java -version
openjdk version "1.8.0_212"
OpenJDK Runtime Environment (build 1.8.0_212-b04)
OpenJDK 64-Bit Server VM (build 25.212-b04, mixed mode)

[kafka@quickstart my-app]$ java -version

openjdk version "1.8.0_212"

OpenJDK Runtime Environment (build 1.8.0_212-b04)

OpenJDK 64-Bit Server VM (build 25.212-b04, mixed mode)

[kafka@quickstart my-app]$ update-alternatives --config java

There is 1 program that provides 'java'.

  Selection    Command
-----------------------------------------------
*+ 1           /usr/lib/jvm/jre-1.8.0-openjdk.x86_64/bin/java

[kafka@quickstart my-app]$ update-alternatives --config java

There is 1 program that provides 'java'.

Selection Command

-----------------------------------------------

*+ 1 /usr/lib/jvm/jre-1.8.0-openjdk.x86_64/bin/java

JAVA_HOME Step 2: Set JAVA_HOME to Java 8 so that Maven uses Java 8. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

29: Docker Tutorial: Apache Kafka with Python 3 on Cloudera quickstart

Prerequisite: Docker is installed on your Windows or Mac, and you have a basic understanding of Docker. Docker tutorials step by step | Hadoop, Hive, Impala & Spark on Cloudera quickstart on Docker tutorials Step 1: Pull the modified image “gdancik/cloudera” … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

30: Docker Tutorial: Apache Spark streaming in Python 3 with Apache Kafka on Cloudera quickstart

This extends Docker Tutorial: Apache Kafka with Python 3 on Cloudera quickstart Step 1: Create the pyspark streaming code in python. driver.py

(my-app_env) [root@quickstart my-app]# vi driver.py

1	(my-app_env) [root@quickstart my-app]# vi driver.py

from pyspark import SparkConf, SparkContext
from pyspark.streaming import StreamingContext
from mypackage import simple

if __name__ == "__main__":
    conf = SparkConf().setAppName("Simple App")
    conf = conf.setMaster("local[*]")
    sc = SparkContext(conf=conf)
    ssc = StreamingContext(sc, 10) # every 10 seconds
    simple.SimpleSpark().myfunc(ssc)

    ssc.start();
    ssc.awaitTermination();

from pyspark import SparkConf, SparkContext

from pyspark.streaming import StreamingContext

from mypackage import simple

if __name__ == "__main__":

conf = SparkConf().setAppName("Simple App")

conf = conf.setMaster("local[*]")

sc = SparkContext(conf=conf)

ssc = StreamingContext(sc, 10) # every 10 seconds

simple.SimpleSpark().myfunc(ssc)

ssc.start();

ssc.awaitTermination();

simple.py – receive messages from a topic

(my-app_env) [root@quickstart my-app]# vi mypackage/simple.py

1	(my-app_env) [root@quickstart my-app]# vi mypackage/simple.py

from pyspark.streaming import StreamingContext
from pyspark.streaming.kafka import KafkaUtils

class SimpleSpark:

    def myfunc(self, ssc):
        # 2181 is the zookeeper port & 9092 is the kafka broker port
        kafkaStream = KafkaUtils.createStream(ssc, 'localhost:2181', 'spark-streaming', {'MyTestTopic':1})
        lines = kafkaStream.map(lambda x : x[1])
        lines.pprint()

from pyspark.streaming import StreamingContext

from pyspark.streaming.kafka import KafkaUtils

class SimpleSpark:

def myfunc(self, ssc):

# 2181 is the zookeeper port & 9092 is the kafka broker port

kafkaStream = KafkaUtils.createStream(ssc, 'localhost:2181', 'spark-streaming', {'MyTestTopic':1})

lines = kafkaStream.map(lambda x : x[1])

lines.pprint()

Build the .egg file Step 2: Create setup.py to build the .egg file, … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

31: Docker Tutorial: Apache Spark streaming in Scala with Apache Kafka on Cloudera quickstart

This extends 27: Docker Tutorial: Apache Kafka with Java API on Cloudera quickstart. Pre-requisite: Java 8 & Kafka installed with the “MyTestTopic” as per the previous tutorials. Install scala & sbt We will install Scala 2.10.7 from “https://www.scala-lang.org/download/2.10.7.html” … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

« 1 2

Blog Archives

24: Docker Tutorial: HBase (i.e. NoSQL DB) Shell on Cloudera quickstart

25: Docker Tutorial: HBase (i.e. NoSQL) Java API on Cloudera quickstart

26: Docker Tutorial: Apache Kafka install, create topic & publish message on Cloudera quickstart

27: Docker Tutorial: Apache Kafka with Java API on Cloudera quickstart

28: Docker Tutorial: Apache Spark streaming with Kafka in Java on Cloudera quickstart

29: Docker Tutorial: Apache Kafka with Python 3 on Cloudera quickstart

30: Docker Tutorial: Apache Spark streaming in Python 3 with Apache Kafka on Cloudera quickstart

31: Docker Tutorial: Apache Spark streaming in Scala with Apache Kafka on Cloudera quickstart

300+ Java Interview Questions - FAQs

10+ Companion Techs Interview Q&As

300+ Big Data Interview Questions - FAQs

16+ Java Tech Key Areas Interview Q&As

Your Career Tips to go places

Tutorials on Java & Big Data

300+ Java Interview Questions Answered

Disclaimer