Blog Archives

07: spark-xml to split & read very large XML files

Processing very large XML files can be a bit tricky as they cannot be processed line by line in parallel as you would do with CSV files. The xml file has to be intact whilst matching the start and end entity tags, and if the tags are distributed in parts…...

Members Only Content

This content is for the members with any one of the following paid subscriptions:

30-Day-Java-JEE-Career-Training, 90-Day-Java-JEE-Career-Training, 180-Day-Java-JEE-Career-Training, 365-Day-Java-JEE-Career-Training, 60-Day-Java-JEE-Career-Training and 2-Year-Java-JEE-Career-Training

Want to evaluate the quality of the contents to see if they will add value to you?

Click Here and check the contents with Try.

Log In | Register
Posted in member-paid, Spark Tutorials

06: Avro Schema evolution tutorial

Q1. What do you understand by the term “AVRO schema evolution“? A1. Schema evolution is the term used for how the store behaves when Avro schema is changed after data has been written to the store using an older version of that schema. Q2. When does the schema evolution take

Read more ›

Posted in Converting File Formats

01b: Convert XML file To Sequence File – writing & reading – Hadoop File System (i.e HDFS)

This extends Convert XML file To Sequence File – writing & reading – Local File System. Step 1: Upload “report.xml” onto HDFS. E.g using the Cloudera HUE on to path “/user/cloudera/report-data”. You need to create the “report-data” folder.

The uploaded file on Hue: Step 2: Change the code to…...

Members Only Content

This content is for the members with any one of the following paid subscriptions:

30-Day-Java-JEE-Career-Training, 90-Day-Java-JEE-Career-Training, 180-Day-Java-JEE-Career-Training, 365-Day-Java-JEE-Career-Training, 60-Day-Java-JEE-Career-Training and 2-Year-Java-JEE-Career-Training

Want to evaluate the quality of the contents to see if they will add value to you?

Click Here and check the contents with Try.

Log In | Register
Posted in Converting File Formats, member-paid

01B: Spark tutorial – writing to HDFS from Spark using Hadoop API

Step 1: The “pom.xml” that defines the dependencies for Spark & Hadoop APIs.

Step 2: The Spark job that writes numbers 1 to 10 to 10 different files on HDFS.

Step 3: Build the “jar” file.

Step 4: Run the “spark-submit” job.

Step 5: You can…...

Members Only Content

This content is for the members with any one of the following paid subscriptions:

30-Day-Java-JEE-Career-Training, 90-Day-Java-JEE-Career-Training, 180-Day-Java-JEE-Career-Training, 365-Day-Java-JEE-Career-Training, 60-Day-Java-JEE-Career-Training and 2-Year-Java-JEE-Career-Training

Want to evaluate the quality of the contents to see if they will add value to you?

Click Here and check the contents with Try.

Log In | Register
Posted in member-paid, Spark Tutorials

02: Spark RDD grouping with groupBy & cogroup in Scala tutorial

This Spark tutorial extends Spark RDD joins in Scala tutorial and Setting up Spark and Scala with Maven. Step 1: Let’s take a simple example of joining a student to department. This will be written in an SQL world as:

Step 2: Let’s create classes to represent Student and…...

Members Only Content

This content is for the members with any one of the following paid subscriptions:

30-Day-Java-JEE-Career-Training, 90-Day-Java-JEE-Career-Training, 180-Day-Java-JEE-Career-Training, 365-Day-Java-JEE-Career-Training, 60-Day-Java-JEE-Career-Training and 2-Year-Java-JEE-Career-Training

Want to evaluate the quality of the contents to see if they will add value to you?

Click Here and check the contents with Try.

Log In | Register
Posted in member-paid, Spark and Scala Tutorials

01: ♥ Spark RDD joins in Scala tutorial

This tutorial extends Setting up Spark and Scala with Maven. Step 1: Let’s take a simple example of joining a student to department. This will be written in an SQL world as:

Step 2: Let’s create classes to represent Student and Department data.

Step 3: The Spark

Read more ›

Posted in Spark and Scala Tutorials

01: Hive Basic Q&As and Tutorial

Q1. What is Hive? A1. Hive is used for accessing and analyzing data in Hadoop using SQL syntax. It is known as the HiveQL. Q2. What is the difference between Hive internal tables & external tables? A2. When you drop an internal table, it drops the data, and it also…...

Members Only Content

This content is for the members with any one of the following paid subscriptions:

30-Day-Java-JEE-Career-Training, 90-Day-Java-JEE-Career-Training, 180-Day-Java-JEE-Career-Training, 365-Day-Java-JEE-Career-Training, 60-Day-Java-JEE-Career-Training and 2-Year-Java-JEE-Career-Training

Want to evaluate the quality of the contents to see if they will add value to you?

Click Here and check the contents with Try.

Log In | Register
Posted in Hive Tutorial & Q&As, member-paid

06: Spark Streaming with Flume Avro Sink Tutorial

This extends Running a Simple Spark Job in local & cluster modes and Apache Flume with JMS source (Websphere MQ) and HDFS sink. In this tutorial a Flume sink will ingest the data from a source like JMS, HDFS, etc and pass it to an “Avro Sink” that pushes data…...

Members Only Content

This content is for the members with any one of the following paid subscriptions:

30-Day-Java-JEE-Career-Training, 90-Day-Java-JEE-Career-Training, 180-Day-Java-JEE-Career-Training, 365-Day-Java-JEE-Career-Training, 60-Day-Java-JEE-Career-Training and 2-Year-Java-JEE-Career-Training

Want to evaluate the quality of the contents to see if they will add value to you?

Click Here and check the contents with Try.

Log In | Register
Posted in member-paid, Spark Tutorials
Page 1 of 512345
1100+ paid subscribers. Reviews | Free Contents. Monthly 260K+ views & 40k+ visitors. 9 tips to earn more.

Java Developer Training – 800+ Q&As ♥Free|♦FAQ (Mouse Hover for Tooltip)

open all | close all

200+ Java Developer Job Interview FAQs

open all | close all

16 Java Programmer Key Areas to be a top-notch

open all | close all

80+ Java Tutorials – Step by step

open all | close all

100+ Java Developer Coding Exercises

open all | close all

How good are your …..Java job hunting & career fast-tracking skills?

open all | close all