Quick Prep Big Data Interview Archives | Page 6 of 23 | 800+ Java & Big Data Interview Q&As

Blog Archives

« 1 2 3 4 5 6 7 8 9 10 … 23 »

04: Snowflake interview questions & answers – security, monitoring & recovery

Q01. What is RBAC, which is Role Based Access Control in Snowflake? A01. RBAC is a security model that restricts access to data & resources based on the roles assigned to users. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

04a: Databricks – Spark SCD Type 1 with Merge

Prerequisite: Extends 03: Databricks – Spark SCD Type 1. What is SCD Type 1 SCD stands for Slowly Changing Dimension, and it was explained in 10 Data warehouse interview Q&As. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

04b: Databricks – Spark SCD Type 2 with Merge

Databricks SCD Type 2 with null mergeKey

Prerequisite: … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

05: 50+ SQL scenarios based interview Q&As on N items per group

This extends 15+ SQL scenarios based interview Q&As on Nth value – part 4. This SQL scenarios based interview questions & answers focus on N items per group. These examples were done on MySQL v8.0 at www.db-fiddle.com. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

05: Azure interview Q&As on Security

Q01. What is Azure Active Directory? A01 Azure Active Directory (i.e. AD) is a cloud-based identity and access management (i.e. IAM) solution that provides a centralised, secure, and scalable platform for managing identities, access control, … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

05: Coding Scala Way – FP concepts

Example #13: Explain the concepts

Can you explain the Scala concepts applied in the following code snippets?


package com.myapp

object Test2 extends App {

  trait Wrapper[C[_]] {
    def aFunction[T,R](f: T => R): C[T] => C[R]
  }

  implicit object ListWrapper extends Wrapper[List] {
    //returns a function: so this is a higher order function
    override def aFunction[T,R](f: T => R): List[T] => List[R]  = { l =>
      println("taking T => R & returning List[T] => List[R]")
      l.map(f)
    }
  }

  // A function that returns an anonymous Object
  // Anonymous class Test2$$anon$1 AnyRef is like java.lang.Object
  private implicit def fnGenAnonymousObj[C[_], T](input: C[T]) = new  { // or new AnyRef or new Object

    //takes a function as an arg: So, this is a higher order function
    def anotherFunction[R](f: T => R)(implicit wrap: Wrapper[C]): C[R] = {
      println("In here first.......")
      val aClosure: C[T] => C[R] = wrap.aFunction(f)         // C[T] => C[R]
      val result: C[R] = aClosure(input)                     // pass the list to the closure
      result
    }
  }

  //val result = fnGenAnonymousObj(List(1, 2, 3)).anotherFunction {x => x + 1}(ListWrapper) // without implicits
  val result = List(1, 2, 3).anotherFunction {x => x + 1}
  println(result)

  //val result2 = fnGenAnonymousObj(List("Java", "Scala")).anotherFunction {x => x.length}(ListWrapper) // without implicits
  val result2 = List("Java", "Scala").anotherFunction {x => x.length}   // with implicits
  println(result2)

}

package com.myapp

object Test2 extends App {

trait Wrapper[C[_]] {

def aFunction[T,R](f: T => R): C[T] => C[R]

}

implicit object ListWrapper extends Wrapper[List] {

//returns a function: so this is a higher order function

override def aFunction[T,R](f: T => R): List[T] => List[R] = { l =>

println("taking T => R & returning List[T] => List[R]")

l.map(f)

}

// A function that returns an anonymous Object

// Anonymous class Test2$$anon$1 AnyRef is like java.lang.Object

private implicit def fnGenAnonymousObj[C[_], T](input: C[T]) = new { // or new AnyRef or new Object

//takes a function as an arg: So, this is a higher order function

def anotherFunction[R](f: T => R)(implicit wrap: Wrapper[C]): C[R] = {

println("In here first.......")

val aClosure: C[T] => C[R] = wrap.aFunction(f) // C[T] => C[R]

val result: C[R] = aClosure(input) // pass the list to the closure

result

}

//val result = fnGenAnonymousObj(List(1, 2, 3)).anotherFunction {x => x + 1}(ListWrapper) // without implicits

val result = List(1, 2, 3).anotherFunction {x => x + 1}

println(result)

//val result2 = fnGenAnonymousObj(List("Java", "Scala")).anotherFunction {x => x.length}(ListWrapper) // without implicits

val result2 = List("Java", "Scala").anotherFunction {x => x.length} // with implicits

println(result2)

}

Output:


In here first.......
taking T => R & returning List[T] => List[R]
List(2, 3, 4)
In here first.......
taking T => R & returning List[T] => List[R]
List(4, 5)

In here first.......

taking T => R & returning List[T] => List[R]

List(2, 3, 4)

In here first.......

taking T => R & returning List[T] => List[R]

List(4, 5)

1. Higher Order Functions: “Line 12” is a higher order function as the function not only takes a function as an argument i.e.

…

05: Databricks interview questions & answers – parquet data vs Delta lake

Q1. How will you convert parquet data to data lake in Databricks? A1. A parquet data will have a list of *.snappy.parquet files under each partition like year=2023, month=02, day-25, etc.

%sql
CONVERT TO DELTA parquet.`abfss://mycontainer@mystorageaccount.dfs.core.windows.net/user_detail`
  PARTITIONED BY (year STRING, month STRING, day STRING)

%sql

CONVERT TO DELTA parquet.`abfss://mycontainer@mystorageaccount.dfs.core.windows.net/user_detail`

PARTITIONED BY (year STRING, month STRING, day STRING)

%python
from delta.tables import *

deltaTable = DeltaTable.convertToDelta(spark, "parquet.`abfss://mycontainer@mystorageaccount.dfs.core.windows.net/user_detail`", "year STRING, month STRING, day STRING")

%python

from delta.tables import *

deltaTable = DeltaTable.convertToDelta(spark, "parquet.`abfss://mycontainer@mystorageaccount.dfs.core.windows.net/user_detail`", "year STRING, month STRING, day STRING")

Q2. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

05: ETL & ELT architecture interview Q&As

Q1. What is an ETL process? A1. ETL is a architectural style, and it stands for Extract, Transform and Load. Extract does the process of reading data from an input data source like file, database, etc. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

05: Linear regression outputs, null hypothesis, t-test & p-value interview Q&As

Q1. How do you produce & interpret Linear Regression output? A1. Scatter plots can only detect obvious relationships between variables by looking at the graph, but we can use statistics to comment about the variable relationships as outlined below. The link 11A: Databricks – Spark ML – Pandas Dataframe &...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

05: Pyspark read from a file & add new columns interview Q&As with tutorial

This extends Pyspark with schema & a simple join of two data frames example.

Q08: How would you read csv data into a spark Dataframe?
A08: Here is the tutorial step by step that reads employee.csv file into the Dataframe.

…

05: Python debugging tutorial

Q. How would you go about debugging Python code? A. Firstly, import pdb

import pdb

1 2	import pdb

Secondly, use “pdb.set_trace()” where ever you want the execution to pause as shown below with “addition.py” file. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

05: Q13 – Q18 Scala FP combinators interview questions & answers

Q13. What are the different approaches in Scala to solve the following task?

Given numbers 1 to 6, and you need to extract out the odd numbers and double them, and finally print the result.

A13. There are 3 ways to solve the problem.

…

05: Q27 – Q32 Scala idioms Q&As on Option-Some-None, Try-Success-Failure and Either-Left-Right

Java has idioms like Try-Catch-Finally, Scala has idioms like 1) “Option-Some-None”, 2) Either-Left-Right 3) Try-Success-Failure. Q27. How does Scala try to solve the problem of getting rid of null values & the “NullPointerException”? A27. Scala provides its own type for representing optional values with “Option[A]”, … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

« 1 2 3 4 5 6 7 8 9 10 … 23 »

Blog Archives

04: Snowflake interview questions & answers – security, monitoring & recovery

04a: Databricks – Spark SCD Type 1 with Merge

04b: Databricks – Spark SCD Type 2 with Merge

05: 50+ SQL scenarios based interview Q&As on N items per group

05: Azure interview Q&As on Security

05: Coding Scala Way – FP concepts

05: Databricks interview questions & answers – parquet data vs Delta lake

05: ETL & ELT architecture interview Q&As

05: Linear regression outputs, null hypothesis, t-test & p-value interview Q&As

05: Pyspark read from a file & add new columns interview Q&As with tutorial

05: Python debugging tutorial

05: Q13 – Q18 Scala FP combinators interview questions & answers

05: Q27 – Q32 Scala idioms Q&As on Option-Some-None, Try-Success-Failure and Either-Left-Right

300+ Java Interview Questions - FAQs

10+ Companion Techs Interview Q&As

300+ Big Data Interview Questions - FAQs

16+ Java Tech Key Areas Interview Q&As

Your Career Tips to go places

Tutorials on Java & Big Data

300+ Java Interview Questions Answered

Disclaimer