Blog Archives

01: Docker tutorial with Java & Maven

Pre-requisite: Docker is installed on your machine for Mac OS X (E.g. $ brew cask install docker) or Windows 10. Docker interview Q&As.

Step 1: Create a Java project “docker-test” with “HelloDocker.java” file under “src/main/java” in a package named “com/mypkg”, and “pom.xml” in the base project folder.…



02: Apache Spark – local mode on Docker tutorial with Java & Maven

This extends 01: Docker tutorial with Java & Maven. This runs Spark in local mode. You build the Spark code as a jar file and run it as a Java…



03: Apache Spark cluster – Docker compose tutorial with Java, Maven & spark-shell

This extends 01: Docker tutorial with Java & Maven. Step 1: Create a Docker network where all 3 containers – Spark master (i.e. Stand-alone cluster manager), Spark worker, and Spark…



04: Apache Spark cluster – Docker compose tutorial with Java, Maven & spark-submit

This extends the previous Docker tutorials. This is a stand-alone Spark cluster tutorial on Docker compose. Step 1: The image docker file “spark.dockerfile” in the folder “docker-test/docker/spark/”.



05: Apache Spark local mode on Docker to read from AWS S3 bucket

Install minio, which is Amazon S3 compatible API Minio is an open source object storage server with Amazon S3 compatible API. Step 1: The “docker-compose.yml” with minio to emulate AWS…



06: Apache Spark standalone cluster on Docker to read from AWS S3 bucket

This extends Apache Spark local mode read from AWS S3 bucket with Docker. Step 1: The “docker-compose.yml” with minio to emulate AWS S3, Spark master and Spark worker to form…



07: Apache Spark standalone cluster on Docker to read from & write to AWS S3 bucket

This extends Apache Spark local mode read from AWS S3 bucket with Docker. Step 1: The “docker-compose.yml” with minio to emulate AWS S3, Spark master and Spark worker to form…



08: Apache Spark standalone cluster on Docker to read from AWS S3 bucket & write to MySQL database

This extends Apache Spark local mode read from AWS S3 bucket with Docker. Step 1: The “docker-compose.yml” with minio to emulate AWS S3, MySQL DB, Spark master and Spark worker…



800+ Java & Big Data Interview Q&As

200+ Java & Big Data Tutorials

Top