Blog Archives
1 2

01: Docker tutorial with Java & Maven

Pre-requisite: Docker is installed on your machine for Mac OS X (E.g. $ brew cask install docker) or Windows 10. Docker interview Q&As.

Step 1: Create a Java project “docker-test” with “HelloDocker.java” file under “src/main/java”

Read more ›



02: Apache Spark – local mode on Docker tutorial with Java & Maven

This extends 01: Docker tutorial with Java & Maven. This runs Spark in local mode. You build the Spark code as a jar file and run it as a Java application in the docker container with the “java” command. You can either create a fat or uber jar with the...

Members Only Content
Log In Register Home


03: Apache Spark cluster – Docker compose tutorial with Java, Maven & spark-shell

This extends 01: Docker tutorial with Java & Maven. Step 1: Create a Docker network where all 3 containers – Spark master (i.e. Stand-alone cluster manager), Spark worker, and Spark driver will be deployed to. Step 2: The image docker file “ … Read more ›...

Members Only Content
Log In Register Home


04: Apache Spark cluster – Docker compose tutorial with Java, Maven & spark-submit

This extends the previous Docker tutorials. This is a stand-alone Spark cluster tutorial on Docker compose. Step 1: The image docker file “spark.dockerfile” in the folder “docker-test/docker/spark/”. Step 2: The pom.xml file. Step 3: The Spark code “ … Read more ›...

Members Only Content
Log In Register Home


05: Apache Spark local mode on Docker to read from AWS S3 bucket

Install minio, which is Amazon S3 compatible API Minio is an open source object storage server with Amazon S3 compatible API. Step 1: The “docker-compose.yml” with minio to emulate AWS S3. Step 2: You can start the minio service as a daemon (i.e with -d) by running Step 3: You...

Members Only Content
Log In Register Home


06: Apache Spark standalone cluster on Docker to read from AWS S3 bucket

This extends Apache Spark local mode read from AWS S3 bucket with Docker. Step 1: The “docker-compose.yml” with minio to emulate AWS S3, Spark master and Spark worker to form a cluster. Step 2: The “spark.dockerfile” … Read more ›...

Members Only Content
Log In Register Home


07: Apache Spark standalone cluster on Docker to read from & write to AWS S3 bucket

This extends Apache Spark local mode read from AWS S3 bucket with Docker. Step 1: The “docker-compose.yml” with minio to emulate AWS S3, Spark master and Spark worker to form a cluster. Step 2: The “spark.dockerfile” … Read more ›...

Members Only Content
Log In Register Home


1 2

800+ Java Interview Q&As Menu

Learn by categories on the go...
Learn by categories such as FAQs – Core Java, Key Area – Low Latency, Core Java – Java 8, JEE – Microservices, Big Data – NoSQL, Architecture – Distributed, Big Data – Spark, etc. Some posts belong to multiple categories.
Top