This extends 02: Hadoop overview & architecture interview Q&As.
MapReduce is obsolete, and this post is mainly to understand the concepts that can be applied elsewhere in distributed computing.
Q16. What is MapReduce (i.e MR)?
A16. MapReduce is a parallel programming model used for processing large datasets across 10 to 1000 of servers across the Hadoop cluster.
…
Q01. What is a gradient? A01. In algebra we can represent a straight line with: y = mx + c A parabola is represented as: y = m1x2 + m2x + c, and so on. … Read more ›...
Q01. What are stages in Snowflake? A01. Snowflake stages are locations where data files are temporarily stored (i.e. staged) for loading and unloading data from & to Snowflake tables. The stages can be internal or external Internal Stage Internal storages are used for loading data from local files or other...
This SQL scenarios based interview questions & answers focus on nth highest value or nth lowest value. These examples were done on MySQL v8.0 at www.db-fiddle.com. This will answer questions like:
Problem statements & hints
1) Who gets the 2nd highest salary?
…
Q01. What are the Compute options Azure provide? A01 Azure compute services can be IaaS, PaaS and Serverless. Azure provides: 1) Azure Virtual Machines (i.e. VMs) are IaaS allowing you to deploy and manage VMs inside a virtual network (VNet). … Read more ›...
Example #10: Null checks with “Option-Some-None”
Nulls in Scala is a code smell, and a better way to handle null values is with an “Option“, which has 2 sub classes “Some” that has a value and a “None”
…
Let’s see how we can wire up functions with & without currying, and then touch on functional composition. Without currying Here are simple functions example that add, add10, multiply and multiplyBy10 without currying.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
package com.sbthello object MyApp extends App { val input = 5 println("input=" + input) def add(x: Int, y: Int) = { x + y } def add10(x: Int) = add(x, 10) def multiply(x: Int, y: Int) = { x * y } def multiplyBy10(x: Int) = multiply(x, 10) println("add10=" + add10(input)) println("multiplyBy10=" + multiplyBy10(input)) //wiring def addAndThenMultiply(x: Int) = { multiplyBy10(add10(x)) } println("add10AndThenmultiplyBy10=" + addAndThenMultiply(input)); } |
Output:
|
1 2 3 4 5 6 |
input=5 add10=15 multiplyBy10=50 add10AndThenmultiplyBy10=150 |
With currying &… Read more ›...
Prerequisite: Extends 03: Databricks – Spark SCD Type 1. What is SCD Type 2 SCD stands for Slowly Changing Dimension, and it was explained in 10 Data warehouse interview Q&As. … Read more ›...
Q1. What is a medallion architecture? A1. A medallion architecture is a data design pattern used to organise data in a lakehouse with the view of incrementally and progressively improving the structure and quality of data as it flows through each layer of the architecture from Bronze ⇒ Silver ⇒...
Q. Are functions objects in Python? A. Yes. In Python, a function is an object of type function. A function being an object means it is possible to pass a function object around as argument or as return value or to assign another name to the function object. … Read...
This extends Pyspark testing interview Q&As with unittest tutorial. Q04: How would you go about writing init tests to test your pyspark transformations and functions? A04: Here is a complete example using a popular python third-party testing library pytest. … Read more ›...
This extends Q1 – Q5 Scala Functional Programming basics interview questions & answers Q6. What is a curried function in Scala? A6. Currying is the technique of transforming a function with multiple arguments into a function with just one argument, … Read more ›...
Q01. What do you understand by the terms mean, variance, and standard deviation of the sample Vs. the population? A01. Given that the following are the number of job applications sent by 6 individuals:
|
1 2 |
X = 17, 15, 23, 7, 9 and 13 |
Where X is the Sample. … Read more ›...
I have a good news, I got job offer. I like to share with you from my bottom of my heart feeling thankful to you.