Home › Big Data Engineers › 80+ Big Data Tutorials › Archive for BDT – Spark on Docker
Blog Archives

01: Docker tutorial with Java & Maven

Pre-requisite: Docker is installed on your machine for Mac OS X (E.g. $ brew cask install docker) or Windows 10. Docker interview Q&As.

Step 1: Create a Java project “…


02: Apache Spark – local mode on Docker tutorial with Java & Maven

This extends 01: Docker tutorial with Java & Maven. This runs Spark in local mode. You build the Spark code as a jar file and run it as a Java application …


03: Apache Spark cluster – Docker compose tutorial with Java, Maven & spark-shell

This extends 01: Docker tutorial with Java & Maven. Step 1: Create a Docker network where all 3 containers – Spark master (i.e. Stand-alone cluster manager), Spark worker, and Spark driver …


04: Apache Spark cluster – Docker compose tutorial with Java, Maven & spark-submit

This extends the previous Docker tutorials. This is a stand-alone Spark cluster tutorial on Docker compose. Step 1: The image docker file “spark.dockerfile” in the folder “docker-test/docker/spark/”.

…

05: Apache Spark local mode on Docker to read from AWS S3 bucket

Install minio, which is Amazon S3 compatible API Minio is an open source object storage server with Amazon S3 compatible API. Step 1: The “docker-compose.yml” with minio to emulate AWS S3. …


06: Apache Spark standalone cluster on Docker to read from AWS S3 bucket

This extends Apache Spark local mode read from AWS S3 bucket with Docker. Step 1: The “docker-compose.yml” with minio to emulate AWS S3, Spark master and Spark worker to form a …


07: Apache Spark standalone cluster on Docker to read from & write to AWS S3 bucket

This extends Apache Spark local mode read from AWS S3 bucket with Docker. Step 1: The “docker-compose.yml” with minio to emulate AWS S3, Spark master and Spark worker to form a …


08: Apache Spark standalone cluster on Docker to read from AWS S3 bucket & write to MySQL database

This extends Apache Spark local mode read from AWS S3 bucket with Docker. Step 1: The “docker-compose.yml” with minio to emulate AWS S3, MySQL DB, Spark master and Spark worker to …


800+ Java Interview Q&As Menu

250+ Java Interview FAQs 🔥
FAQs - Prepared?
FAQs - Ice Breaker
FAQs - Core Java
FAQs - JEE
FAQs - Web Services
FAQs - AWS
FAQs - Microservices
FAQs - Architecture
FAQs - Hibernate
FAQs - Spring
FAQs - Key Areas 🔑
FAQs - OOP & FP
FAQs - Code Quality
FAQs - UML
FAQs - SQL
FAQs - RDBMS
FAQs - Maven
FAQs - JSON & XML
FAQs - Unix
FAQs - Regex
FAQs - DevOps
FAQs - JMeter
FAQs - GIT
16+ Key Areas Q&As 🔑 Open-ended Q&As
Key Area - Going places 🚄
Key Area - Ice Breaker
Key Area - Judging Experience
Key Area - Best Practices
Key Area - Concurrency Mgmt
Key Area - Low Latency
Key Area - CI/CD & DevOps
Key Area - Debugging
Key Area - Debug Hibernate
Key Area - Debug Java
Key Area - Debug JEE
Key Area - Debug Spark
Key Area - Debug Spring
Key Area - Design Concepts
Key Area - Design Patterns
Key Area - GoF Design
Key Area - JEE Design
Key Area - Exception Handling
Key Area - Memory Mgmt
Key Area - Performance
Key Area - Scalability
Key Area - SDLC
Key Area - Security
Key Area - Productivity Tools
Key Area - Transaction Mgmt
Key Area - QoS & SLAs
300+ Core Java Interview Q&As
Core Java - Overview
Core Java - Data types
Core Java - Constructors
Core Java - Reserved Key Words
Core Java - Classes
Core Java - Objects
Core Java - OOP
Core Java - GC
Core Java - Generics
Core Java - FP
Core Java - OOP to FP
Core Java - I/O
Core Java - Multithreading
Core Java - Data Structures
Core Java - Annotations
Core Java - AOP
Core Java - Other
Core Java - Collection API
Core Java - Differences X & Y
Core Java - Event Driven
Core Java - Exceptions
Core Java - Java 7
Core Java - Java 8
Core Java - JVM
Core Java - Reactive Program
Core Java - Swing & AWT
Core Java - Understanding
300+ JEE Interview Q&As
JEE - Overview
JEE - Web basics
JEE - Web Services
JEE - JPA
JEE - JTA
JEE - API Design
JEE - JDBC
JEE - Microservices
JEE - JMS
JEE - JMX
JEE - JNDI and LDAP
JEE - JSF
JEE - Other
80+ Spring & Hibernate Q&As
Frameworks - Hibernate Q&As
Frameworks - Spring Core
Frameworks - Spring MVC
Frameworks - Spring Cloud
Frameworks - Spring boot
Frameworks - Spring IO
Frameworks - Spring Config
100+ Architecture FAQs 🔥
Architecture - Patterns
Architecture - How to
Architecture - AWS
Architecture - Distributed
Architecture - Microservices
Architecture - QoS
Architecture - ERD
Architecture - ETL
Architecture - Low Latency
Architecture - UML
80+ Companion Technologies
Companion - Angular JS
Companion - Domain Knowledge
Companion - FIX
Companion - Git & SVN
Companion - Groovy
Companion - JavaScript
Companion - JMeter
Companion - Maven
Companion - NoSQL
Companion - Regex
Companion - SQL
Companion - Unix
Companion - JSON
Companion - XML
Companion - XSD
Companion - YAML
40+ Code Quality - Q&As ✍
Code Quality - Automation
Code Quality - Code Coverage
Code Quality - FAQs
Code Quality - Profiling
Code Quality - Performance Test
Code Quality - Unit Testing
Code Quality - BDD
Code Quality – Testing DAOs
Code Quality – JUnit Spring
Code Quality – JUnit Web
Code Quality - Spring TXN
Tutorials Java, JEE, Spring
Tutorials - ⏯ Starting with Java
Tutorials - Diagnosis
Tutorials - Akka
Tutorials - Akka Scala
Tutorials - Akka Java
Tutorials - AWS EC2
Tutorials - CI/CD Jenkins
Tutorials - Core Java
Tutorials - Debugging
Tutorials - Java 8
Tutorials - JSON and Java
Tutorials - Java Beginner
Tutorials - XML and Java
Tutorials - CSV and Java
Tutorials - Docker
Tutorials - JEE
Tutorials - Web & Web Services
Tutorials - REST Client
Tutorials - JSF
Tutorials - JDBC
Tutorials - FIX and Java
Tutorials - JAXB
Tutorials - JMX
Tutorials - JPA
Tutorials - MySQL
Tutorials - Scala
Tutorials - Scala Actors
Tutorials - Spring & Hibernate
Tutorials - Hibernate
Tutorials - Spring Basics
Tutorials - Spring batch
Tutorials - Hibernate & Spring
Tutorials - Spring Boot
Tutorials - Spring IO
Tutorials - Spring ORM
Tutorials - Spring other
Tutorials - Spring OXM
Tutorials - Spring Cloud
Tutorials - Tools
Tutorials - JMeter
Tutorials - Maven
Tutorials - Maven Assembly
Tutorials - Maven Profiles
Tutorials - Productivity Tools
Tutorials - Kubernetes
Tutorials - ⏯ Videos
Tutorials - ⏯ Starting with Java
Tutorials - ⏯ Spring Core
Tutorials - ⏯ Spring MVC
Tutorials - ⏯ JDBC, Hibernate
Tutorials - ⏯ Spring Boot
Tutorials - ⏯ Angular with Spring
Tutorials - ⏯ Q&As
Tutorials - Other
Tutorials - SQL
Tutorials - Drools
Tutorials - POI
Tutorials - DOS
Tutorials - Dozer
Tutorials - ESB
Tutorials - Groovy
Tutorials - Jasper Reports
Tutorials - Logging and auditing
Tutorials - Metrics
Tutorials - Orika Mapping
Tutorials - Pentaho Kettle ETL
Tutorials - Unix
Tutorials - Webspeher MQ
Tutorials - YAML
150+ Coding Q&As ✍
Coding - Can you write code?
Coding - Complete the code
Coding - Converting from A to B
Coding - Designing your classes
Coding - Data Structures
Coding - Arrays
Coding - Big O notation
Coding - Graph Structure
Coding - LinkedList Structure
Coding - Map Structure
Coding - Pointer Algorithms
Coding - Queue Structure
Coding - Searching Algo
Coding - Sorting Algo
Coding - Stack Structure
Coding - Tree structure
Coding - Data Which & Why
Coding - Passing the unit tests
Coding - What is wrong?
Coding - Home Assignments
Coding - Written Test Core Java
Coding - Written Test JEE
Prepare to fast-track & go places
with multi-offers to choose from & increased earning potential. Expand your horizons along the way by taking the road less travelled.

Career Paths as a Developer

Learn by categories on the go...
Learn by categories such as FAQs – Core Java, Key Area – Low Latency, Core Java – Java 8, JEE – Microservices, Big Data – NoSQL, etc. Some posts belong to multiple categories.
Top