Home › Big Data Engineers › 80+ Big Data Tutorials › Archive for BDT – Hadoop HDFS
Blog Archives

01: Apache Hadoop HDFS Tutorial

Step 1: Download the latest version of “Apache Hadoop common” from http://apache.claz.org/hadoop using wget, curl or a browser. This tutorial uses “http://apache.claz.org/hadoop/core/hadoop-2.7.1/”.

Step 2: You can set Hadoop environment variables by appending the following commands to ~/.bashrc file.

You can run this in a Unix command prompt as

Step 3: You can verify if Hadoop has been setup properly with

Step 4: The Hadoop file in $HADOOP_HOME/etc/Hadoop/hadoop-env.sh has the JAVA_HOME setting.

…

Read more ›


02: Java to write from/to Local to HDFS File System

This extends Hadoop MapReduce Basic Tutorial and Apache Hadoop HDFS Tutorial. This could have have been done on the command-line as shown below after running “start-dfs.sh” to start the name and data nodes.

The focus of this tutorial is to do the same via Java and Hadoop APIs.

…

Read more ›


03: Create or append a file to HDFS – Hadoop API tutorial

Step 1: Create a simple maven project named “simple-hadoop“. Step 2: Import the “simple-hadoop” maven project into eclipse or IDE of your choice. Step 3: Modify the pom.xml file include 1) relevant Hadoop libraries 2) The shade plugin to create a single jar (i.e. … Read more ›...

Members Only Content
Log In Register Home

04: Create new or append to an existing AVRO file tutorial

This extends Create or append a file to HDFS – Hadoop API tutorial to write an AVRO file to HDFS. Step 1: Include the AVRO library files in the pom.xml file. Step 2: The AVRO files are schema based. … Read more ›...

Members Only Content
Log In Register Home

05: Create or append a Sequence file to HDFS – Hadoop API tutorial

The following tutorial extends Create or append a file to HDFS – Hadoop API tutorial, and Create or append an AVRO file to HDFS – Hadoop & AVRO API tutorial. In this tutorial we will write to a Sequence file, … Read more ›...

Members Only Content
Log In Register Home

800+ Java Interview Q&As

250+ Java Interview FAQs 🔥
FAQs - Prepared?
FAQs - Ice Breaker
FAQs - Core Java
FAQs - JEE
FAQs - Web Services
FAQs - AWS
FAQs - Microservices
FAQs - Architecture
FAQs - Hibernate
FAQs - Spring
FAQs - Key Areas
FAQs - OOP & FP
FAQs - Code Quality
FAQs - UML
FAQs - SQL
FAQs - RDBMS
FAQs - Maven
FAQs - JSON & XML
FAQs - Unix
FAQs - Regex
FAQs - DevOps
FAQs - JMeter
FAQs - GIT
16+ Tech Key Areas Q&As 🔑
Key Area - Going places 🚄
Key Area - Concurrency Mgmt
Key Area - Low Latency
Key Area - Best Practices
Key Area - CI/CD & DevOps
Key Area - Debugging
Key Area - Debug Hibernate
Key Area - Debug Java
Key Area - Debug JEE
Key Area - Debug Spark
Key Area - Debug Spring
Key Area - Design Concepts
Key Area - Design Patterns
Key Area - GoF Design
Key Area - JEE Design
Key Area - Exception Handling
Key Area - Memory Mgmt
Key Area - Performance
Key Area - Scalability
Key Area - SDLC
Key Area - Security
Key Area - Productivity Tools
Key Area - Transaction Mgmt
Key Area - QoS & SLAs
Key Area - Judging Experience
300+ Core Java Interview Q&As
Core Java - Overview
Core Java - Data types
Core Java - Constructors
Core Java - Reserved Key Words
Core Java - Classes
Core Java - Objects
Core Java - OOP
Core Java - GC
Core Java - Generics
Core Java - FP
Core Java - OOP to FP
Core Java - I/O
Core Java - Multithreading
Core Java - Data Structures
Core Java - Annotations
Core Java - AOP
Core Java - Other
Core Java - Collection API
Core Java - Differences X & Y
Core Java - Event Driven
Core Java - Exceptions
Core Java - Java 7
Core Java - Java 8
Core Java - JVM
Core Java - Reactive Program
Core Java - Swing & AWT
Core Java - Understanding
300+ JEE Interview Q&As
JEE - Overview
JEE - Web basics
JEE - Web Services
JEE - JPA
JEE - JTA
JEE - API Design
JEE - JDBC
JEE - Microservices
JEE - JMS
JEE - JMX
JEE - JNDI and LDAP
JEE - JSF
JEE - Other
80+ Spring & Hibernate Q&As
Frameworks - Hibernate Q&As
Frameworks - Spring Core
Frameworks - Spring MVC
Frameworks - Spring Cloud
Frameworks - Spring boot
Frameworks - Spring IO
Frameworks - Spring Config
100+ Architecture FAQs 🔥
Architecture - Patterns
Architecture - How to
Architecture - AWS
Architecture - Distributed
Architecture - Microservices
Architecture - QoS
Architecture - ERD
Architecture - ETL
Architecture - Low Latency
Architecture - UML
80+ Companion Technologies
Companion - Angular JS
Companion - Domain Knowledge
Companion - FIX
Companion - Git & SVN
Companion - Groovy
Companion - JavaScript
Companion - JMeter
Companion - Maven
Companion - NoSQL
Companion - Regex
Companion - SQL
Companion - Unix
Companion - JSON
Companion - XML
Companion - XSD
Companion - YAML
40+ Code Quality - Q&As ✍
Code Quality - Automation
Code Quality - Code Coverage
Code Quality - FAQs
Code Quality - Profiling
Code Quality - Performance Test
Code Quality - Unit Testing
Code Quality - BDD
Code Quality – Testing DAOs
Code Quality – JUnit Spring
Code Quality – JUnit Web
Code Quality - Spring TXN
Tutorials Java, JEE, Spring
Tutorials - ⏯ Starting with Java
Tutorials - Diagnosis
Tutorials - Akka
Tutorials - Akka Scala
Tutorials - Akka Java
Tutorials - AWS EC2
Tutorials - CI/CD Jenkins
Tutorials - Core Java
Tutorials - Debugging
Tutorials - Java 8
Tutorials - JSON and Java
Tutorials - Java Beginner
Tutorials - XML and Java
Tutorials - CSV and Java
Tutorials - Docker
Tutorials - JEE
Tutorials - Web & Web Services
Tutorials - REST Client
Tutorials - JSF
Tutorials - JDBC
Tutorials - FIX and Java
Tutorials - JAXB
Tutorials - JMX
Tutorials - JPA
Tutorials - MySQL
Tutorials - Scala
Tutorials - Scala Actors
Tutorials - Spring & Hibernate
Tutorials - Hibernate
Tutorials - Spring Basics
Tutorials - Spring batch
Tutorials - Hibernate & Spring
Tutorials - Spring Boot
Tutorials - Spring IO
Tutorials - Spring ORM
Tutorials - Spring other
Tutorials - Spring OXM
Tutorials - Spring Cloud
Tutorials - Tools
Tutorials - JMeter
Tutorials - Maven
Tutorials - Maven Assembly
Tutorials - Maven Profiles
Tutorials - Productivity Tools
Tutorials - Kubernetes
Tutorials - ⏯ Videos
Tutorials - ⏯ Starting with Java
Tutorials - ⏯ Spring Core
Tutorials - ⏯ Spring MVC
Tutorials - ⏯ JDBC, Hibernate
Tutorials - ⏯ Spring Boot
Tutorials - ⏯ Angular with Spring
Tutorials - ⏯ Q&As
Tutorials - Other
Tutorials - SQL
Tutorials - Drools
Tutorials - POI
Tutorials - DOS
Tutorials - Dozer
Tutorials - ESB
Tutorials - Groovy
Tutorials - Jasper Reports
Tutorials - Logging and auditing
Tutorials - Metrics
Tutorials - Orika Mapping
Tutorials - Pentaho Kettle ETL
Tutorials - Unix
Tutorials - Webspeher MQ
Tutorials - YAML
150+ Coding Q&As ✍
Coding - Can you write code?
Coding - Complete the code
Coding - Converting from A to B
Coding - Designing your classes
Coding - Data Structures
Coding - Arrays
Coding - Big O notation
Coding - Graph Structure
Coding - LinkedList Structure
Coding - Map Structure
Coding - Pointer Algorithms
Coding - Queue Structure
Coding - Searching Algo
Coding - Sorting Algo
Coding - Stack Structure
Coding - Tree structure
Coding - Data Which & Why
Coding - Passing the unit tests
Coding - What is wrong?
Coding - Home Assignments
Coding - Written Test Core Java
Coding - Written Test JEE
Prepare to fast-track & go places
with multi-offers to choose from & increased earning potential. Expand your horizons along the way by taking the road less travelled.
Learn by categories on the go...
Learn by categories such as FAQs – Core Java, Key Area – Low Latency, Core Java – Java 8, JEE – Microservices, Big Data – NoSQL, etc. Some posts belong to multiple categories.
Top