Big Data Interview FAQs - 300+ Archives | Page 3 of 16 | 800+ Java & Big Data Interview Q&As

Blog Archives

« 1 2 3 4 5 6 7 … 16 »

02: Pyspark with schema & a simple join of two data frames example

This extends Installing & getting started with Pyspark on Mac tutorial, and assumes that pyspark has been properly set up. If you are using a Windows OS, please google for “installing pyspark on Windows” or “getting started with pyspark on windows”.

In the previous tutorial we have set up pyspark &

…

02: Q7 – Q15 Hadoop overview & architecture interview Q&As

This extends Q1 – Q6 Hadoop Overview & Architecture interview Q&As. Q7. What are the major machine roles in a Hadoop cluster? A7. The three major categories of machine roles in a Hadoop cluster are 1) Client machines. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

02: Snowflake interview questions & answers – caching & performance tuning

Q01. What is Query Caching in Snowflake? A01. Snowflake caches the results of queries to speed up repeated query executions. Running the same query twice will utilise the cached results if the underlying data has not changed to improve query performance & … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

02: SQL analytic functions interview questions – Part 1

Data Engineers & analysts must know the analytic functions applied over a window of rows is a powerful tool to analyse the data. This post compliments “Apache Spark window functions” by focusing on MySQL server. You can also apply this on Big Data SQL engines like Hive &

…

02: SQL analytic functions interview questions – Part 2

This extends SQL window function for analytical queries interview questions – Part 1 Analytic function construct


analytic_function_name([argument_list])
OVER (
[PARTITION BY partition_expression,…]
[ORDER BY sort_expression, … [ASC|DESC]])

analytic_function_name([argument_list])

OVER (

[PARTITION BY partition_expression,…]

[ORDER BY sort_expression, … [ASC|DESC]])

The analytic function names can be RANK(), DENSE_RANK(), ROW_NUMBER(), MIN(column), MAX(column), SUM(column), COUNT(), AVG(column), FIRST_VALUE(column), LAST_VALUE(column), NTH_VALUE(column, n), LEAD(return_value ,offset [,default]), LAG(return_value ,offset [,default]), NTILE(n), … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

02: SQL analytic functions interview questions – Part 3

This extends SQL analytic functions interview questions – Part 2 and SQL analytic functions interview questions – Part 1. The below example was created in IMPALA SQL engine with Common Table Expression (i.e CTE).

WITH WORK_ORDER
AS (
     SELECT 101 as order_id, TO_DATE('2020-01-01') as ORDER_DATE, TO_DATE('2020-01-15') as COMPLETED_DATE  
     UNION ALL
     SELECT 102 as order_id, TO_DATE('2020-06-01') as ORDER_DATE, TO_DATE('2020-06-08') as COMPLETED_DATE
     UNION ALL
     SELECT 103 as order_id, TO_DATE('2020-08-15') as ORDER_DATE, TO_DATE('2020-09-12') as COMPLETED_DATE
     UNION ALL
     SELECT 104 as order_id, TO_DATE('2020-09-13') as ORDER_DATE, TO_DATE('2020-09-18') as COMPLETED_DATE
     UNION ALL
     SELECT 105 as order_id, TO_DATE('2020-11-11') as ORDER_DATE, TO_DATE('2020-12-07') as COMPLETED_DATE
     UNION ALL
     SELECT 106 as order_id, TO_DATE('2021-02-13') as ORDER_DATE, TO_DATE('2021-02-28') as COMPLETED_DATE
     UNION ALL
     SELECT 107 as order_id, TO_DATE('2021-05-01') as ORDER_DATE, TO_DATE('2021-05-16') as COMPLETED_DATE
)
SELECT order_id, DATEDIFF(COMPLETED_DATE, ORDER_DATE) AS COMPLETION_DAYS
FROM WORK_ORDER
ORDER BY order_id
;

WITH WORK_ORDER

AS (

SELECT 101 as order_id, TO_DATE('2020-01-01') as ORDER_DATE, TO_DATE('2020-01-15') as COMPLETED_DATE

UNION ALL

SELECT 102 as order_id, TO_DATE('2020-06-01') as ORDER_DATE, TO_DATE('2020-06-08') as COMPLETED_DATE

UNION ALL

SELECT 103 as order_id, TO_DATE('2020-08-15') as ORDER_DATE, TO_DATE('2020-09-12') as COMPLETED_DATE

UNION ALL

SELECT 104 as order_id, TO_DATE('2020-09-13') as ORDER_DATE, TO_DATE('2020-09-18') as COMPLETED_DATE

UNION ALL

SELECT 105 as order_id, TO_DATE('2020-11-11') as ORDER_DATE, TO_DATE('2020-12-07') as COMPLETED_DATE

UNION ALL

SELECT 106 as order_id, TO_DATE('2021-02-13') as ORDER_DATE, TO_DATE('2021-02-28') as COMPLETED_DATE

UNION ALL

SELECT 107 as order_id, TO_DATE('2021-05-01') as ORDER_DATE, TO_DATE('2021-05-16') as COMPLETED_DATE

)

SELECT order_id, DATEDIFF(COMPLETED_DATE, ORDER_DATE) AS COMPLETION_DAYS

FROM WORK_ORDER

ORDER BY order_id

;

Output:

order_id	completion_days
------------------------------	
101	           14	
102	            7	
103	           28	
104	            5	
105	           26	
106	           15	
107	           15

order_id completion_days

------------------------------

101 14

102 7

103 28

104 5

105 26

106 15

107 15

CUME_DIST() analytic function The CUME_DIST( ) analytic function stands for cumulative distribution, …...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

03: 50+ SQL scenarios based interview Q&As on familiarising various features

This extends 50+ SQL scenarios based interview Q&As on joins, CDC and recursive CTE. Q01 In What order the SELECT SQL statements are executed? A01 Here is the the order in which the SQL statements are evaluated. 1) FROM (including JOIN, … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

03: Azure interview Q&As on storage account

Q01. What is an Azure Storage Account? A01 An Azure storage account contains all of your Azure Storage data objects blobs, files, queues, and tables. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

03: Databricks – Spark SCD Type 1

Prerequisite: Extends Databricks getting started – Spark, Shell, SQL.

What is SCD Type 1

SCD stands for Slowly Changing Dimension, and it was explained in 10 Data warehouse interview Q&As.

…

03: Databricks interview questions & answers – Azure Databricks

Q1. What is ADLS Gen2? A1. Azure Data Lake Storage Gen2 (ADLS) is a cloud-based repository for both structured and unstructured data. For example, you could use it to store everything from documents to images to social media streams. Data Lake Storage Gen2 is built on top of Azure Blob...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

03: Pyspark testing interview Q&As with unittest tutorial

This extends Pyspark with schema & a simple join of two data frames example. Q01: Why should you test the pyspark pipelines? A01: The data pipelines need to be tested as data pipelines evolve with new functionalities and enhancements to existing processes, … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

03: Q16 – Q26 Hadoop MapReduce interview questions & answers

This extends 02: Hadoop overview & architecture interview Q&As.

MapReduce is obsolete, and this post is mainly to understand the concepts that can be applied elsewhere in distributed computing.

Q16. What is MapReduce (i.e MR)?
A16. MapReduce is a parallel programming model used for processing large datasets across 10 to 1000 of servers across the Hadoop cluster.

…

03: Simple Linear Regression interview Q&As

Q01. What is a gradient? A01. In algebra we can represent a straight line with: y = mx + c A parabola is represented as: y = m1x2 + m2x + c, and so on. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

« 1 2 3 4 5 6 7 … 16 »

Blog Archives

02: Pyspark with schema & a simple join of two data frames example

02: Q7 – Q15 Hadoop overview & architecture interview Q&As

02: Snowflake interview questions & answers – caching & performance tuning

02: SQL analytic functions interview questions – Part 1

02: SQL analytic functions interview questions – Part 2

02: SQL analytic functions interview questions – Part 3

03: 50+ SQL scenarios based interview Q&As on familiarising various features

03: Azure interview Q&As on storage account

03: Databricks – Spark SCD Type 1

03: Databricks interview questions & answers – Azure Databricks

03: Pyspark testing interview Q&As with unittest tutorial

03: Q16 – Q26 Hadoop MapReduce interview questions & answers

03: Simple Linear Regression interview Q&As

300+ Java Interview Questions - FAQs

10+ Companion Techs Interview Q&As

300+ Big Data Interview Questions - FAQs

16+ Java Tech Key Areas Interview Q&As

Your Career Tips to go places

Tutorials on Java & Big Data

300+ Java Interview Questions Answered

Disclaimer