Tutorials - Big Data Archives | Page 13 of 13 | 800+ Java & Big Data Interview Q&As

Blog Archives

« 1 … 9 10 11 12 13

Before running a Spark job on a YARN cluster in Cloudera and about the Spark history server

Problem: When you run a Spark job via “spark-submit” command on a “YARN” cluster as shown below in a terminal,


-bash-4.1$ spark-submit --class com.mytutorial.SparkSimpleRdd --master yarn --deploy-mode cluster /home/cloudera/projects/simple-spark/target/simple-spark-1.0-SNAPSHOT.jar

-bash-4.1$ spark-submit --class com.mytutorial.SparkSimpleRdd --master yarn --deploy-mode cluster /home/cloudera/projects/simple-spark/target/simple-spark-1.0-SNAPSHOT.jar

It creates a folder and files in HDFS under


/user/spark/applicationHistory/

/user/spark/applicationHistory/

The files will be named with “application ids” like “application_1510971789108_0015”. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

MapReduce to HBase: Read from & write to

This tutorial extends 1. HBase Tutorial Getting Started & 1. Hadoop MapReduce Basic Tutorial to read an HBase table with data from the mapper and write the max marks for each subject to another HBase table from the reducer..

Input Data Marks secured by 4 students


Science, 80, 75, 89, 90
Maths,  90, 87, 78, 92
English, 78, 88, 65, 99

Science, 80, 75, 89, 90

Maths, 90, 87, 78, 92

English, 78, 88, 65, 99

Step 1: Let’s store this Data in HBase database as described in tutorial 1.

…

Understanding Cloudera Hadoop users

Step 1: A number of special users are created by default when installing and using CDH and Cloudera Manager. For example

Unix user id: hdfs
groups: hdfs hadoop

Unix user id: spark
groups: spark

Unix user id: hive
groups: hive

and so on.

…

Zookeeper example – getting started tutorial with Java & Docker

Zookeeper solves challenges in a distributed system. This extends Apache Zookeeper interview Q&As and Getting started with Zookeeper tutorial. In this Zookeeper tutorial let’s use Docker. Zookeeper image in Docker Hub Step 1: Zookeeper image can be found at Docker Hub. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

« 1 … 9 10 11 12 13

Blog Archives

Before running a Spark job on a YARN cluster in Cloudera and about the Spark history server

MapReduce to HBase: Read from & write to

Understanding Cloudera Hadoop users

Zookeeper example – getting started tutorial with Java & Docker

300+ Java Interview Questions - FAQs

10+ Companion Techs Interview Q&As

300+ Big Data Interview Questions - FAQs

16+ Java Tech Key Areas Interview Q&As

Your Career Tips to go places

Tutorials on Java & Big Data

300+ Java Interview Questions Answered

Disclaimer