Q01. What is a data pipeline? A01. A data pipeline is a means of moving data from one place (i.e source systems e.g. OLTP) to a destination such as a data lake or warehouse by simultaneously optimising and transforming the data. … Read more ›...
800+ Java & Big Data Interview Q&As
130+ job offers to choose from as a self-taught Java & Big Data contractor for 22+ years. It pays to prepare.
Blog Archives
Debugging Spark applications written in Java locally by connecting to HDFS, Hive and HBase
This extends Remotely debugging Spark submit Jobs in Java. Running Spark in local mode When you run Spark in local mode, both the Driver and Executor will be running in the same JVM and is very handy to debug the logic of your transformations. … Read more ›...
This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only.
Register
80+ Free Java FAQs
80+ Free Big Data FAQs
Already a member? Log in here
Defining tables in postgresql with primary key, foreign key, unique constraint & data migration example
Uses DB-FIDDLE for the code example with PostgreSQL v17.
Q01 How will you go about defining tables with primary key, foreign keys and unique constraints?
A01 The main purpose of a primary key in a table is to uniquely identify each record.
…
Finding your way around YARN and Spark on Cloudera
What is Apache Hadoop YARN? Apache Hadoop YARN (Yet Another Resource Negotiator) is the prerequisite for Enterprise Hadoop for dynamic allocation oc the cluster resources. For example, when you run a Spark job on YARN as opposed to in standalone mode, you can will take advantage of categorizing, isolating, and...
This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only.
Register
80+ Free Java FAQs
80+ Free Big Data FAQs
Already a member? Log in here
How would you go about.. building an ETL pipeline to handle 3k TPS (i.e transactions per second)
Q1. How would you go about.. building an ETL pipeline to handle 3k TPS? A1. Building a data pipeline (i.e. aka ETL) for 3000 TPS (i.e. Transactions per second), which about ~260 million records per day requires distributed, highly scalable, … Read more ›...
This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only.
Register
80+ Free Java FAQs
80+ Free Big Data FAQs
Already a member? Log in here
InfluxDB interview questions & answers
Q01. What is InfluxDb?
A01. InfluxDB is an open-source time series database. It is a Big Data, NoSQL database that allows for massive scalability, high availability, fast write, and fast read. As a NoSQL, InfluxDB stores time-series data,
…
More HDFS & NameNode interview Q&As
Q1. What are the different ways to access files or data stored in HDFS? A1. You can access files or data stored in HDFS in many different ways. HDFS Command-Line Interface The HDFS command-line interface (CLI), called hdfs (or hadoop) shell, … Read more ›...
This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only.
Register
80+ Free Java FAQs
80+ Free Big Data FAQs
Already a member? Log in here
Multi-tenant Architecture Interview Q&As
Q1. What is a multi-tenant architecture? A1. A “tenant” in cloud computing is a single customer or to be more specific a group of users who share a common access with specific privileges to the software instance. … Read more ›...
This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only.
Register
80+ Free Java FAQs
80+ Free Big Data FAQs
Already a member? Log in here
MySQL import and export of data via command line using mysqldump & mysql
Q01 How do you migrate MySQL database between two servers
A01 This can be done using an SQL Dump Export/Import.
1. Export MySQL Database to a dump File
|
1 2 3 |
mysqldump -h myserver1.com -u root -p mydb1 > mydb1.sql |
There are variations to dump multiple databases:
|
1 2 3 |
mysqldump -h myserver1.com -u root -p --databases mydb1 mydb2 > dump.sql |
To backup specific tables:
|
1 2 3 |
mysqldump -h myserver1.com -u root -p mydb1 table1 table2 > dump.sql |
To backup data using a custom query:
|
1 2 3 |
mysqldump -h myserver1.com -u root -p mydb1 --where="mycolumn=myvalue" > dump.sql |
To copy only the schema but not the data:
|
1 2 3 |
mysqldump -h myserver1.com -u root -p mydb1 --no-data > dump.sql |
2.…
Q01 – Q07 Scenarios Based Hive Interview Q&As
Q1. Given you have a Hive table partitioned by ship_type where the ship_type can be DIRECT or PICK_UP. The last job run to populate the DIRECT shipment data for the month of “2021 Feb” has caused a data corruption. How will you go about rollback data load & … Read...
This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only.
Register
80+ Free Java FAQs
80+ Free Big Data FAQs
Already a member? Log in here
Q08- Q12 Scenarios Based Hive Query Language (i.e. HQL) Interview Q&As
This extends Q01 – Q07 Scenarios Based Hive Query Language (i.e. HQL) Interview Q&As. Q08. When you have a requirement to replicate all the data from a particular source table table to a target table, how will you go about it? A08. … Read more ›...
This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only.
Register
80+ Free Java FAQs
80+ Free Big Data FAQs
Already a member? Log in here
Snowflake Snowpark interview Questions & Answers
Both Apache Spark & Snowflake’s Snowpark are used for data engineering use cases like ETL data pipelines, data validations and create data lakes & warehouses. They also can be used in data science use cases like train, develop & provision machine learning models. They can also be used for data...
This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only.
Register
80+ Free Java FAQs
80+ Free Big Data FAQs
Already a member? Log in here
Spark interview Q&As with coding examples in Scala – part 01: Key basics
Some of these basic Apache Spark interview questions can make or break your chance to get an offer.
Q01. Why is “===” used in the below Dataframe join?
|
1 2 3 4 5 6 7 |
df1.join (df2, $"df1Key" === $"df2Key") df1.join (df2, $"df1Key" === $"df2Key", "inner") // inner join df1.join (df2, $"df1Key" === $"df2Key", "left") // left outer join df1.join(df2).where($"df1Key" === $"df2Key") df1.join(df2).filter($"df1Key" === $"df2Key") |
A01. Comparisons with == and != are universal. This has a problem as they compare any two values,
…
300+ Java Interview Questions - FAQs
10+ Companion Techs Interview Q&As
300+ Big Data Interview Questions - FAQs
16+ Java Tech Key Areas Interview Q&As
Your Career Tips to go places
Tutorials on Java & Big Data
300+ Java Interview Questions Answered
“
I have a good news, I got job offer. I like to share with you from my bottom of my heart feeling thankful to you.
Jithendra
via LinkedIn
Disclaimer
The contents in this Java-Success are copyrighted and from EmpoweringTech pty ltd. The EmpoweringTech pty ltd has the right to correct or enhance the current content without any prior notice. These are general advice only, and one needs to take his/her own circumstances into consideration. The EmpoweringTech pty ltd will not be held liable for any damages caused or alleged to be caused either directly or indirectly by these materials and resources. Any trademarked names or labels used in this blog remain the property of their respective trademark owners. Links to external sites do not imply endorsement of the linked-to sites. Privacy Policy
Responsive WordPress Theme
powered by
CyberChimps
Top
Scroll Up