Tutorials - Big Data Archives | Page 11 of 13 | 800+ Java & Big Data Interview Q&As

Blog Archives

« 1 … 7 8 9 10 11 12 13 »

19: Docker Tutorial: Apache Spark SQL – on Cloudera quickstart

This extends Docker Tutorial: BigData on Cloudera quickstart via Docker.

Step 1: Run the container on a command line.

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \
--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \
--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 cloudera/quickstart /usr/bin/docker-quickstart

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \

--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \

--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 cloudera/quickstart /usr/bin/docker-quickstart

Create a .csv in HDFS

Step 2: Create a csv file “employee.csv” in the local file system.

…

2. Apache Pig: Regex (Regular expressions)

This extends the tutorial 1. Apache Pig Getting started.

Input Data

scores.xml in folder:/Users/arulk/projects representing marks of 4 students in 3 subjects:


<scores>
   <subject>
       <name>Science</name>
       <marks>
          <mark>80</mark>
      <mark>75</mark>
      <mark>89</mark>
      <mark>90</mark>
      </marks>
   </subject>
   <subject>
       <name>Maths</name>
       <marks>
          <mark>90</mark>
      <mark>87</mark>
      <mark>78</mark>
      <mark>92</mark>
      </marks>
   </subject>
   <subject>
       <name>English</name>
       <marks>
          <mark>78</mark>
      <mark>88</mark>
      <mark>65</mark>
      <mark>99</mark>
      </marks>
   </subject>
</scores>

<name>Science</name>

<marks>

80

75

89

90

</marks>

</subject>

<name>Maths</name>

<marks>

90

87

78

92

</marks>

</subject>

<name>English</name>

<marks>

78

88

65

99

</marks>

</subject>

</scores>

Step 1: Start pig in local file system mode.

…

2. Hadoop MapReduce Basic Tutorial

This extends the Part 1 tutorial 1. Hadoop MapReduce Basic Tutorial. The key difference of this tutorial is using a “TextInputFormat” instead of “KeyValueTextInputFormat“.

TextInputFormat reads

The key as line offset number starting from 0 and the values as “Science,

…

2. HBase Shell commands

This extends 1. HBase Tutorial Getting Started. You can learn HBase basics at HBase interview Questions & Answers.

Step 1: Unlike relational databases, the NoSQL databases are semi-structured, hence you can add new columns on the fly. In HBase, you can define the table name and the column family first and then new columns for a column family can be added programmatically on the fly.

…

20: Databricks – Spark ML K-Means

Prerequisite: Extends Databricks – Spark ML Random Forrest Classifier. Q. What is K-Means algorithm? A. K-Means is an unsupervised algorithm, hence it does only have “features“, … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

20: Docker Tutorial: Apache Spark (spark-submit) in Java on Cloudera quickstart

This extends Docker Tutorial: BigData on Cloudera quickstart via Docker. Step 1: Run the container on a command line.

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \
--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \
--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 cloudera/quickstart /usr/bin/docker-quickstart

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \

--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \

--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 cloudera/quickstart /usr/bin/docker-quickstart

Step 2: Java is already installed as part of the “cloudera/quickstart” docker image.

[root@quickstart /]# java -version
java version "1.7.0_67"
Java(TM) SE Runtime Environment (build 1.7.0_67-b01)
Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)

[root@quickstart /]# java -version

java version "1.7.0_67"

Java(TM) SE Runtime Environment (build 1.7.0_67-b01)

Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)

update-alternatives If you have multiple versions or installations of Java, … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

21: Databricks – Spark ML Naive Bayes

Prerequisite: Extends 16: Databricks – Spark ML Multiclass Logistic Regression & Pipeline. Problem statement: Predict the iris flower species based on its sepal & petal length & width. Independent Variables (aka features): sepal length in cm, … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

21: Docker Tutorial: Apache Spark (spark-submit) in Scala on Cloudera quickstart

Extends 20: Docker Tutorial: Apache Spark-submit in Java – on Cloudera quickstart, and Docker Tutorial: BigData on Cloudera quickstart via Docker. Step 1: Run the container on a command line.

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \
--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \
--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 cloudera/quickstart /usr/bin/docker-quickstart

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \

--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \

--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 cloudera/quickstart /usr/bin/docker-quickstart

Install Java 8 Step 2: Install Java 8. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

22: Docker Tutorial: Apache Spark (spark-submit) in Python 2.6 on Cloudera quickstart

Extends 20: Docker Tutorial: Apache Spark-submit in Java – on Cloudera quickstart, and Docker Tutorial: BigData on Cloudera quickstart via Docker. Step 1: Run the container on a command line.

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \
--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \
--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 cloudera/quickstart /usr/bin/docker-quickstart

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \

--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \

--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 cloudera/quickstart /usr/bin/docker-quickstart

Python version Step 2: The cloudera/quickstart comes with Python 2.6. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

22: Writing Tests in Databricks Notebooks in Python

Prerequisite: Extends 03: Databricks – Spark SCD Type 1. Note: databricks-connect let’s you connect your favourite IDE to your Databricks cluster. This allows you to lint, test, and package the code that you want to run on Databricks as explained in Enhance Your Databricks Workflow. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

23: Docker Tutorial: Apache Spark (spark-submit) in Python 3 with virtual env on Cloudera quickstart

Cloudera quickstart Docker with Python 3

Prerequisite: Docker is installed on your Windows or Mac, and you have a basic understanding of Docker. Docker tutorials step by step | Hadoop, Hive, Impala & Spark on Cloudera quickstart on Docker tutorials

Step 1: Pull the modified image “gdancik/cloudera”

…

23: PySpark Delta Lake on Databricks notebook

Prerequisite: Databricks – Spark SCD Type 1. Delta Lake bring features like ACID Transactions, Scalable metadata, time travel, etc to a Data Lake. Databricks notebook has delta-spark package preinstalled. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

24: Docker Tutorial: HBase (i.e. NoSQL DB) Shell on Cloudera quickstart

This extends Docker Tutorial: BigData on Cloudera quickstart via Docker. Step 1: Run the container on a command line.

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \
--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \
--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 \
cloudera/quickstart /usr/bin/docker-quickstart

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \

--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \

--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 \

cloudera/quickstart /usr/bin/docker-quickstart

hbase cli We are going to focus on “shell” command to create a table, column-families, and columns, … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

« 1 … 7 8 9 10 11 12 13 »

Blog Archives

19: Docker Tutorial: Apache Spark SQL – on Cloudera quickstart

2. Apache Pig: Regex (Regular expressions)

2. Hadoop MapReduce Basic Tutorial

2. HBase Shell commands

20: Databricks – Spark ML K-Means

20: Docker Tutorial: Apache Spark (spark-submit) in Java on Cloudera quickstart

21: Databricks – Spark ML Naive Bayes

21: Docker Tutorial: Apache Spark (spark-submit) in Scala on Cloudera quickstart

22: Docker Tutorial: Apache Spark (spark-submit) in Python 2.6 on Cloudera quickstart

22: Writing Tests in Databricks Notebooks in Python

23: Docker Tutorial: Apache Spark (spark-submit) in Python 3 with virtual env on Cloudera quickstart

23: PySpark Delta Lake on Databricks notebook

24: Docker Tutorial: HBase (i.e. NoSQL DB) Shell on Cloudera quickstart

300+ Java Interview Questions - FAQs

10+ Companion Techs Interview Q&As

300+ Big Data Interview Questions - FAQs

16+ Java Tech Key Areas Interview Q&As

Your Career Tips to go places

Tutorials on Java & Big Data

300+ Java Interview Questions Answered

Disclaimer