Tutorials - Big Data Archives | Page 10 of 13 | 800+ Java & Big Data Interview Q&As

Blog Archives

« 1 … 6 7 8 9 10 11 12 13 »

14: Databricks – Spark ML – StringIndexer & OneHotEncoder

Prerequisite: Extends Databricks – Spark ML – Linear Regression Categorical Features. Problem statement: Predict the house prices by land area in square feet, house condition as in “Bad”, “Average”, and “Good”, and house locality as in “Sydney”, … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

14: Docker Tutorial: Hive (via beeline) on Cloudera quickstart

This extends Docker Tutorial: BigData on Cloudera quickstart via Docker. Step 1: Run the container on a command line.

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \
--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \
--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 cloudera/quickstart /usr/bin/docker-quickstart

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \

--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \

--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 cloudera/quickstart /usr/bin/docker-quickstart

Step 2: Create a csv file “employee.csv” in the local file system.

[root@quickstart /]# touch employee.csv
[root@quickstart /]# vi employee.csv

[root@quickstart /]# touch employee.csv

[root@quickstart /]# vi employee.csv

John, Samuel, IT
Peter, Smith, Finance
Sean, Mendis, Marketing

John, Samuel, IT

Peter, Smith, Finance

Sean, Mendis, Marketing

Step 3: Copy this file onto HDFS file system. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

14: Spark joins with SQLContext & JavaPairRDD

This extends the last tutorial where Spark inner & outer joins in Java with JavaPairRDDs. In this tutorial let’s read the orders via a Hive table using SQLContext & Dataframe. RDD left outer join with filtering via JavaPairRDD Here is a left outer join with filtering to display all the...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

15: Databricks – Spark ML – Classification with Logistic Regression

Prerequisite: Extends Spark ML – StringIndexer & OneHotEncoder – LinearRegression. Q. What is a Classification type prediction? How does it differ from Linear Regression? A. Classification type predictions are 1) Is Email spam or not? … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

15: Docker Tutorial: Hive & parquet-tools – csv to parquet on Cloudera quickstart

CSV is a row based storage, and Parquet is columnar in nature, and it is designed from the ground up for efficient storage, compression and encoding, which gives better performance. Run the cloudera/quickstart This extends Docker Tutorial: BigData on Cloudera quickstart via Docker. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

15: Spark joins with Dataframes & SQLContext

Create LineItems Hive table Step 1: Create a file “line-item1.txt” on HDFS under “/user/cloudera/learn-hdfs/lineitems” as


123,Pen
456,Pencil

123,Pen

456,Pencil

Step 2: You create a Hive table “lineitems” in the database “learnhadoop”.


CREATE EXTERNAL TABLE IF NOT EXISTS learnhadoop.lineitems (
order_id int
,product_name string

)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
LOCATION 'hdfs:////user/cloudera/learn-hdfs/lineitems';

CREATE EXTERNAL TABLE IF NOT EXISTS learnhadoop.lineitems (

order_id int

,product_name string

)

ROW FORMAT DELIMITED FIELDS TERMINATED BY ','

LOCATION 'hdfs:////user/cloudera/learn-hdfs/lineitems';

Step 3: The “orders” table can be created in a similar manner as shown above. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

16: Databricks – Spark ML Multiclass Logistic Regression & Pipeline

Prerequisite: Extends 15: Databricks – Spark ML – Classification with Logistic Regression & Databricks – Spark ML – StringIndexer & OneHotEncoder. Problem statement: Predict the likelihood of leaning towards a political party based on age & … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

16: Docker Tutorial: Apache Spark (spark-shell) & parquet-tools – csv to parquet on Cloudera quickstart

This extends Docker Tutorial: BigData on Cloudera quickstart via Docker. Step 1: Run the container on a command line.

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \
--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \
--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 cloudera/quickstart /usr/bin/docker-quickstart

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \

--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \

--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 cloudera/quickstart /usr/bin/docker-quickstart

Get parquet-tools Step 2: Install wget. The “uname -a” gets you the info of the kernel. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

17: Databricks – Spark ML K-Folds Cross Validation

Prerequisite: Extends Databricks – Spark ML Multiclass Logistic Regression & Pipeline. Q: What is K-Folds Cross Validation in ML? How does it differ from the train-split validation? A: Once we are done with training our model, … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

17: Docker Tutorial: sqoop import – on Cloudera quickstart

This extends Docker Tutorial: BigData on Cloudera quickstart via Docker. Step 1: Run the container on a command line.

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \
--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \
--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 cloudera/quickstart /usr/bin/docker-quickstart

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \

--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \

--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 cloudera/quickstart /usr/bin/docker-quickstart

Connect to MySQL DB Step 2: Hive & Impala store the metadata in a database like MySQL. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

18: Databricks – Spark ML Decision Tree Classifier

Prerequisite: Extends Databricks – Spark ML Multiclass Logistic Regression & Pipeline. Q: What is a Decision Tree Classifier? A: A Decision Tree a Supervised Machine Learning where the data is continuously split into Root Node, … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

18: Docker Tutorial: sqoop export – on Cloudera quickstart

This extends Docker Tutorial: BigData on Cloudera quickstart via Docker. Step 1: Run the container on a command line.

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \
--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \
--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 cloudera/quickstart /usr/bin/docker-quickstart

~/projects/docker-hadoop]$ docker run --hostname=quickstart.cloudera \

--privileged=true -t -i -v /Users/arulkumarankumaraswamipillai/projects/docker-hadoop:/src \

--publish-all=true -p 8888:8888 -p 80:80 -p 7180:7180 cloudera/quickstart /usr/bin/docker-quickstart

Connect to MySQL DB Step 2: Hive & Impala store the metadata in a database like MySQL. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

19: Databricks – Spark ML Random Forrest Classifier

Prerequisite: Extends Databricks – Spark ML Decision Tree Classifier. Q. What is the difference between a decision tree classifier and a random forrest classifier? A. As is implied by the names “Tree” … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

« 1 … 6 7 8 9 10 11 12 13 »

Blog Archives

14: Databricks – Spark ML – StringIndexer & OneHotEncoder

14: Docker Tutorial: Hive (via beeline) on Cloudera quickstart

14: Spark joins with SQLContext & JavaPairRDD

15: Databricks – Spark ML – Classification with Logistic Regression

15: Docker Tutorial: Hive & parquet-tools – csv to parquet on Cloudera quickstart

15: Spark joins with Dataframes & SQLContext

16: Databricks – Spark ML Multiclass Logistic Regression & Pipeline

16: Docker Tutorial: Apache Spark (spark-shell) & parquet-tools – csv to parquet on Cloudera quickstart

17: Databricks – Spark ML K-Folds Cross Validation

17: Docker Tutorial: sqoop import – on Cloudera quickstart

18: Databricks – Spark ML Decision Tree Classifier

18: Docker Tutorial: sqoop export – on Cloudera quickstart

19: Databricks – Spark ML Random Forrest Classifier

300+ Java Interview Questions - FAQs

10+ Companion Techs Interview Q&As

300+ Big Data Interview Questions - FAQs

16+ Java Tech Key Areas Interview Q&As

Your Career Tips to go places

Tutorials on Java & Big Data

300+ Java Interview Questions Answered

Disclaimer