Blog Archives

01. Setting up Scala & practicing the concepts via REPL the Scala way for Java developers

Scala runs on the JVM, so Java and Scala stacks can be freely mixed. You can call Java libraries from Scala. Having said this, it is very important that you learn to write code the Scala way, and Not Java way. Currently, Java programmers with Scala experience are paid more and in demand.

Read more ›



02. Setting up Scala IDE & using Maven plugins for Java developers

This extends Setting up Java, Maven & Eclipse and Setting up Scala & practicing the concepts via REPL the Scala way for Java developers. Step 1: Scala IDE is an eclipse plugin. Following is the Scala IDE update site URL for Eclipse 4.4 (Luna) and Eclipse 4.5 (Mars). … Read...



03. Setting up Scala and Spark with Maven

This assumes that you set up Scala IDE for eclipse as per Setting up Scala IDE & using Maven plugins. Step 1: Create a Scala project in Maven. Step 2: Import Maven project into eclipse. File -> … Read more ›...



04. Setting up & getting started with sbt

SBT (i.e. Simple Build Tool) for Scala requires Java & Scala. This tutorial is based on Java 8 (i.e. 1.8.0) and Scala 2.12.7. Scala runs on the JVM, so Java and Scala stacks can be freely mixed. You can call Java libraries from Scala. … Read more ›...



05: Setting up & getting started with Spark local mode with Sbt & Scala

This extends Setting up & getting started with sbt. Spark 2.3.2 requires Scala 2.11.x. You need to use the right Scala version. Step 1: On Eclipse set the Scala compiler to 2.11. Right mouse click on the project “sbt-tutorial” … Read more ›...



06: Setting up Spark-shell on Mac & getting started

This tutorial outlines the basic steps to get started with Spark on Mac OS. 1. Install Xcode Xcode can be installed via Apple appstore. Xcode is Apple’s Integrated Development Environment (IDE). Xcode is a large suite of software development tools and libraries from Apple. 2.… Read more ›...



07: Getting started with Spark-submit

This extends the tutorial Setting up & getting started with Spark local mode with Sbt & Scala and Setting up Spark standalone on Mac. Spark-submit is used to submit a Spark job. In this tutorial we will be using the “local” master in “client” … Read more ›...



08. Setting up & getting started with Hadoop on Mac

This tutorial outlines the basic steps to get started with Hadoop on Mac OS. This is a single-node cluster with YARN as the resource manager.

1. Install Xcode

Xcode can be installed via Apple appstore. Xcode is Apple’s Integrated Development Environment (IDE). Xcode is a large suite of software development tools and libraries from Apple.

Read more ›



09. Getting started with Spark & Hadoop – client mode on local & cluster mode on yarn

This extends Setting up & getting started with Spark local mode with Sbt & Scala and Setting up & getting started with Hadoop on Mac.

The Spark job written in Scala will be reading the data from HDFS.

1. CSV data in HDFS

2.

Read more ›



10. Setting up & getting started with Hive

Hive is popular SQL based tool to process Big Data. It stores & retrieves data to/from HDFS.

Prerequisite: Xcode & Hadoop are installed as outlined in Setting up & getting started with Hadoop on Mac

1. Install Hive

Let’s install hive-2.3.4 with hadoop 2.7.7.

Read more ›



800+ Java Q&As & tutorials

Top