Quick Prep Big Data Interview Archives | Page 15 of 23 | 800+ Java & Big Data Interview Q&As

Blog Archives

« 1 … 11 12 13 14 15 16 17 18 19 … 23 »

16B: Scala Monoids, Functors & Monads interview Q&As

Scala Monoids, Functors & Monads interview questions are very popular & form the basis for the functional programming. Q01. Given the below code, is it possible to implement the map(..) method with flatMap(…) within the Monad trait?

package com.monad

object MyMonad extends App {

  trait Monoid[A] {
    def identity: A

    def combine(a: A, b: A): A
  }

  trait Functor[F[_]] {
    def map[A, B](a: F[A])(f: A => B): F[B]
  }

  trait Monad[M[_]] extends Functor[M] /* with Monoid[_ => M[_]] */ {
    def pure[A](a: A): M[A]

    def flatMap[A, B](a: M[A])(f: A => M[B]): M[B]

  }
}

package com.monad

object MyMonad extends App {

trait Monoid[A] {

def identity: A

def combine(a: A, b: A): A

}

trait Functor[F[_]] {

def map[A, B](a: F[A])(f: A => B): F[B]

}

trait Monad[M[_]] extends Functor[M] /* with Monoid[_ => M[_]] */ {

def pure[A](a: A): M[A]

def flatMap[A, B](a: M[A])(f: A => M[B]): M[B]

}

A01. Yes. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

17: 50+ SQL scenarios based interview Q&As on teams & matches played

Let’s look at a standard teams interview question to understand your SQL knowledge in using a non-equi join. Problem Statement 1 Given the below 5 teams


team_id     team_name
------------------------
1           Team A
2           Team B
3           Team C
4           Team D
5           Team E

team_id team_name

------------------------

1 Team A

2 Team B

3 Team C

4 Team D

5 Team E

1) Write a query so that each team plays with each other only once. 2) Write a query so that each team plays...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

17: Coding Scala Way – What is wrong with this code?

Q. What is wrong with the below code? How will you fix it?


package com.myapp

object Test extends App {

  val map = Map (
     "1" -> "2015",
     "2" -> "2016",
     "3" -> "2017"
  )

  def isCorrectYearById(id: String, year: String) : Boolean = {
    map.get(id) == year
  }

  println(isCorrectYearById("2", "2016"))
}

package com.myapp

object Test extends App {

val map = Map (

"1" -> "2015",

"2" -> "2016",

"3" -> "2017"

)

def isCorrectYearById(id: String, year: String) : Boolean = {

map.get(id) == year

}

println(isCorrectYearById("2", "2016"))

}

A. The above code will always return “false”. Why? Because map.get(id) returns Some(2016), which is not == to the String value of “2016”. The “==”

…

17: PySpark problem solving Interview Q&As with tutorials – problem 1

This is based on the series of tutorials & interview Q&As at PySpark interview Q&As with tutorials

Problem: Convert the below table

+------+--------+---------+-------------+
|emp_id|emp_name| emp_city|emp_expertise|
+------+--------+---------+-------------+
|     1|    John|   Sydney|         Java|
|     2|   Peter|Melbourne|        Scala|
|     3|     Sam| Brisbane|       Python|
|     4|   David|Melbourne|       Python|
|     5|  Elliot|   Sydney|         Java|
+------+--------+---------+-------------+

+------+--------+---------+-------------+

+------+--------+---------+-------------+

| 3| Sam| Brisbane| Python|

+------+--------+---------+-------------+

+-------------+----------------------------------------------------------+
|Column       |Content                                                   |
+-------------+----------------------------------------------------------+
|emp_id       |[[1, 1], [3, 1], [5, 1], [4, 1], [2, 1]]                  |
|emp_name     |[[Elliot, 1], [John, 1], [Sam, 1], [Peter, 1], [David, 1]]|
|emp_city     |[[Sydney, 2], [Melbourne, 2], [Brisbane, 1]]              |
|emp_expertise|[[Python, 2], [Java, 2], [Scala, 1]]                      |
+-------------+----------------------------------------------------------+

+-------------+----------------------------------------------------------+

|Column |Content |

+-------------+----------------------------------------------------------+

|emp_id |[[1, 1], [3, 1], [5, 1], [4, 1], [2, 1]] |

|emp_name |[[Elliot, 1], [John, 1], [Sam, 1], [Peter, 1], [David, 1]]|

|emp_city |[[Sydney, 2], [Melbourne, 2], [Brisbane, 1]] |

|emp_expertise|[[Python, 2], [Java, 2], [Scala, 1]] |

+-------------+----------------------------------------------------------+

Where each column is counted for its occurrence. For example,

…

17: Python OOP and @property decorator tutorial

Members in a Python class are public by default All members in a Python class are public by default. Any member can be accessed from outside the class environment. Here is “person.py”

class Person:

    def __init__(self, name, age):
        self.name = name
        self.age = age

    def get_name(self):
        return self.name

    def get_age(self):
        return self.age

class Person:

def __init__(self, name, age):

self.name = name

self.age = age

def get_name(self):

return self.name

def get_age(self):

return self.age

$ python3
>>> from person import Person
>>> p = Person("John", 25)
>>> p.name
'John'
>>> p.age
25
>>>

$ python3

>>> from person import Person

>>> p = Person("John", 25)

>>> p.name

'John'

>>> p.age

>>>

Python’s convention to make an instance variable protected is to add a prefix _ (single underscore)...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

17: Q99 – Q103 Scala applicative functors interview Q&As

This extends Scala Functors Interview Q&As & Scala monads interview Q&As. You need to remember the following.


"T => R"      to return  "C[T] => C[R]"    is a  Functor
"T => C[R]"   to return  "C[T] => C[R]"    is a  Monad
"C[T => R]"   to return  "C[T] => C[R]"    is an Applicative Functor

"T => R" to return "C[T] => C[R]" is a Functor

"T => C[R]" to return "C[T] => C[R]" is a Monad

"C[T => R]" to return "C[T] => C[R]" is an Applicative Functor


"Int => Boolean"                to return "MyContainer[Int] => MyContainer[Boolean]"   is a  Functor
"Int => MyContainer[Boolean]"   to return "MyContainer[Int] => MyContainer[Boolean]"   is a  Monad
"MyContainer[Int => Boolean]"   to return "MyContainer[Int] => MyContainer[Boolean]"   is an Applicative Functor

"Int => Boolean" to return "MyContainer[Int] => MyContainer[Boolean]" is a Functor

"Int => MyContainer[Boolean]" to return "MyContainer[Int] => MyContainer[Boolean]" is a Monad

"MyContainer[Int => Boolean]" to return "MyContainer[Int] => MyContainer[Boolean]" is an Applicative Functor

Q99. Why do you need an applicative functor? A99. The “map” method in the functor works with a single input argument as explained in the Scala functors interview Q&As. … Read...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

17: Spark interview Q&As with coding examples in pyspark (i.e. python)

Q01. How will you create a Spark context? A01.

from pyspark.sql import SparkSession

 spark = SparkSession.builder.appName("my spark job")
 spark.master('local[*]')

 spark.config('spark.jars.packages', 'com.amazonaws:aws-java-sdk:1.11.297,org.apache.hadoop:hadoop-aws:2.8.3,mysql:mysql-connector-java:5.1.46')
              .config('spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version', 2)
              .config('spark.speculation', 'false')

from pyspark.sql import SparkSession

spark = SparkSession.builder.appName("my spark job")

spark.master('local[*]')

spark.config('spark.jars.packages', 'com.amazonaws:aws-java-sdk:1.11.297,org.apache.hadoop:hadoop-aws:2.8.3,mysql:mysql-connector-java:5.1.46')

.config('spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version', 2)

.config('spark.speculation', 'false')

Q02. How will you create a Dataframe by reading a file from AWS S3 bucket? A02.


      csvFileAsDataframe = spark.read.format("com.databricks.spark.csv") \
                         .option("header", "false") \
                         .option("inferSchema", "true") \
                         .load(s3://my-bucket/some-path/input-file.csv) \

csvFileAsDataframe = spark.read.format("com.databricks.spark.csv") \

.option("header", "false") \

.option("inferSchema", "true") \

.load(s3://my-bucket/some-path/input-file.csv) \

Q03. How will you create a Dataframe by reading a table in a database? … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

18 Amazon S3 interview Q&As

Q01. What is Amazon S3 storage? A01. S3 stands for Simple Storage Service, which provides storage through web service interfaces. Unlike the other storage systems like Unix file system, HDFS (i.e. Hadoop Distributed File System), etc which are based on having folders and files, the S3 is based on a...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

18: 50+ SQL scenarios based interview Q&As on account balance

Here is another SQL scenario based interview question to determine a candidate’s ability to translate a business requirement into an SQL query. Problem Statement Given the below data of running account balance, find the start_date and end_date where the balance changes.


id       balance         date
-------------------------------------
1        24000.00        2023-01-15
2        24000.00        2023-01-16
3        19000.00        2023-01-17
4        19000.00        2023-01-18
5        22000.00        2023-01-19
6        24000.00        2023-01-20
7        24000.00        2023-01-21
8        29000.00        2023-01-22
9        29000.00        2023-01-23
10       29000.00        2023-01-24

id balance date

-------------------------------------

1 24000.00 2023-01-15

2 24000.00 2023-01-16

3 19000.00 2023-01-17

4 19000.00 2023-01-18

5 22000.00 2023-01-19

6 24000.00 2023-01-20

7 24000.00 2023-01-21

8 29000.00 2023-01-22

9 29000.00 2023-01-23

10 29000.00 2023-01-24

Expected output:


balance          start_date         end_date
----------------------------------------------
24000.00         2023-01-15         2023-01-16
19000.00         2023-01-17         2023-01-18
22000.00         2023-01-19         2023-01-19
24000.00         2023-01-20         2023-01-21
29000.00         2023-01-22         2023-01-24

balance start_date end_date

----------------------------------------------

24000.00 2023-01-15 2023-01-16

19000.00 2023-01-17 2023-01-18

22000.00 2023-01-19 2023-01-19

24000.00 2023-01-20 2023-01-21

29000.00 2023-01-22 2023-01-24

Create sample data

CREATE TABLE IF NOT EXISTS account_balance
   (id INT,
   balance DECIMAL(10,2),
   date DATE
);  

INSERT INTO account_balance(id, balance, date) VALUES  (1, 24000.00, '2023-01-15');
INSERT INTO account_balance(id, balance, date) VALUES (2, 24000.00, '2023-01-16');
INSERT INTO account_balance(id, balance, date) VALUES (3, 19000.00, '2023-01-17');
INSERT INTO account_balance(id, balance, date) VALUES (4, 19000.00, '2023-01-18');
INSERT INTO account_balance(id, balance, date) VALUES (5, 22000.00, '2023-01-19');
INSERT INTO account_balance(id, balance, date) VALUES (6, 24000.00, '2023-01-20');
INSERT INTO account_balance(id, balance, date) VALUES (7, 24000.00, '2023-01-21');
INSERT INTO account_balance(id, balance, date) VALUES (8, 29000.00, '2023-01-22');
INSERT INTO account_balance(id, balance, date) VALUES (9, 29000.00, '2023-01-23');
INSERT INTO account_balance(id, balance, date) VALUES (10, 29000.00, '2023-01-24');

CREATE TABLE IF NOT EXISTS account_balance

(id INT,

balance DECIMAL(10,2),

date DATE

);

INSERT INTO account_balance(id, balance, date) VALUES (1, 24000.00, '2023-01-15');

INSERT INTO account_balance(id, balance, date) VALUES (2, 24000.00, '2023-01-16');

INSERT INTO account_balance(id, balance, date) VALUES (3, 19000.00, '2023-01-17');

INSERT INTO account_balance(id, balance, date) VALUES (4, 19000.00, '2023-01-18');

INSERT INTO account_balance(id, balance, date) VALUES (5, 22000.00, '2023-01-19');

INSERT INTO account_balance(id, balance, date) VALUES (6, 24000.00, '2023-01-20');

INSERT INTO account_balance(id, balance, date) VALUES (7, 24000.00, '2023-01-21');

INSERT INTO account_balance(id, balance, date) VALUES (8, 29000.00, '2023-01-22');

INSERT INTO account_balance(id, balance, date) VALUES (9, 29000.00, '2023-01-23');

INSERT INTO account_balance(id, balance, date) VALUES (10, 29000.00, '2023-01-24');

Solution...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

18: PySpark problem solving Interview Q&As with tutorials – problem 2

Problem: Convert the below table

+--------+------------+---+
|emp_name|     product|qty|
+--------+------------+---+
|    John|    Notebook|  7|
|   Peter|Mobile phone| 12|
|     Sam|      Camera|  3|
|   David|Mobile phone|  9|
|  Elliot|    Notebook|  3|
|  Elliot|      Camera|  1|
|    John|Mobile phone| 13|
|   Peter|Mobile phone|  2|
+--------+------------+---+

+--------+------------+---+

|emp_name| product|qty|

+--------+------------+---+

| John| Notebook| 7|

| Peter|Mobile phone| 12|

| Sam| Camera| 3|

| David|Mobile phone| 9|

| Elliot| Notebook| 3|

| Elliot| Camera| 1|

| John|Mobile phone| 13|

| Peter|Mobile phone| 2|

+--------+------------+---+

+--------+------+------------+--------+
|emp_name|Camera|Mobile phone|Notebook|
+--------+------+------------+--------+
|  Elliot|     1|        null|       3|
|    John|  null|          13|       7|
|     Sam|     3|        null|    null|
|   Peter|  null|          14|    null|
|   David|  null|           9|    null|
+--------+------+------------+--------+

+--------+------+------------+--------+

+--------+------+------------+--------+

| Elliot| 1| null| 3|

| John| null| 13| 7|

| Sam| 3| null| null|

+--------+------+------------+--------+

Step 1: Let’s create the sample data using PySpark. problem.py:


from pyspark.sql import SparkSession
from pyspark.sql.types import *
from pyspark.sql import functions as F
from pyspark.sql import DataFrame
from functools import reduce

def solve(spark):

    sales = (
        ('John', 'Notebook', 7),
        ('Peter', 'Mobile phone', 12),
        ('Sam', 'Camera', 3),
        ('David', 'Mobile phone', 9),
        ('Elliot', 'Notebook', 3),
        ('Elliot', 'Camera', 1),
        ('John', 'Mobile phone', 13),
        ('Peter', 'Mobile phone', 2)
     )
 
    data_schema = [StructField('emp_name', StringType(), False), StructField('product',StringType(),False), StructField('qty', IntegerType(), False)]
    final_struc = StructType(fields=data_schema)
 
    rdd = spark.sparkContext.parallelize(sales);
 
    df_sales = spark.createDataFrame(rdd, final_struc)
    df_sales.show()


if __name__ == '__main__':
    spark = SparkSession.builder.appName("Spark problem solving")\
                                .master("local[*]")\
                                .getOrCreate()
    solve(spark)

from pyspark.sql import SparkSession

from pyspark.sql.types import *

from pyspark.sql import functions as F

from pyspark.sql import DataFrame

from functools import reduce

def solve(spark):

sales = (

('John', 'Notebook', 7),

('Peter', 'Mobile phone', 12),

('Sam', 'Camera', 3),

('David', 'Mobile phone', 9),

('Elliot', 'Notebook', 3),

('Elliot', 'Camera', 1),

('John', 'Mobile phone', 13),

('Peter', 'Mobile phone', 2)

)

data_schema = [StructField('emp_name', StringType(), False), StructField('product',StringType(),False), StructField('qty', IntegerType(), False)]

final_struc = StructType(fields=data_schema)

rdd = spark.sparkContext.parallelize(sales);

df_sales = spark.createDataFrame(rdd, final_struc)

df_sales.show()

if __name__ == '__main__':

spark = SparkSession.builder.appName("Spark problem solving")\

.master("local[*]")\

.getOrCreate()

solve(spark)

Run as:


$ spark-submit problem.py

$ spark-submit problem.py

Outputs:


+--------+------------+---+
|emp_name|     product|qty|
+--------+------------+---+
|    John|    Notebook|  7|
|   Peter|Mobile phone| 12|
|     Sam|      Camera|  3|
|   David|Mobile phone|  9|
|  Elliot|    Notebook|  3|
|  Elliot|      Camera|  1|
|    John|Mobile phone| 13|
|   Peter|Mobile phone|  2|
+--------+------------+---+

+--------+------------+---+

|emp_name| product|qty|

+--------+------------+---+

| John| Notebook| 7|

| Peter|Mobile phone| 12|

| Sam| Camera| 3|

| David|Mobile phone| 9|

| Elliot| Notebook| 3|

| Elliot| Camera| 1|

| John|Mobile phone| 13|

| Peter|Mobile phone| 2|

+--------+------------+---+

pivot( ) to the rescue Step 2: The code to create the results. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

18: Python Abstract Base Classes (i.e. ABC) tutorial

Q. What is the difference between abstract class and interface in Python? A. Python doesn’t have a Java style formal interface, hence distinction between abstraction and interface doesn’t exist. Java uses interfaces because it doesn’t have multiple inheritance, but Python has multiple inheritance. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

18: Q104 – Q108 Scala typeclass pattern for decoupling with polymorphism

Q104. What is the importance of “typeclass” pattern in Scala? A104. Typeclass pattern is to achieve decoupling via adhoc polymorphism. Polymorphism separates interfaces from implementations, to decouple what from how. Polymorphism improves code readability with better code organization. … Read more ›...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

18: Q121 – Q124 Choosing between HDFS & AWS S3 for BigData projects interview Q&As

More and more organisations are adopting the policy of “Cloud first architecture” where Cloud based storages like AWS S3 plays a major role. A data lake is a storage repository that holds a vast amount of structured, semi-structured, and unstructured raw data in its native format (aka pristine condition). …...

This content is for 30-Day-Full-Access, 100-Day-Full-Access, 200-Day-Full-Access, and 365-Day-Full-Access members only. Register 80+ Free Java FAQs 80+ Free Big Data FAQs

Already a member? Log in here

« 1 … 11 12 13 14 15 16 17 18 19 … 23 »

Blog Archives

16B: Scala Monoids, Functors & Monads interview Q&As

17: 50+ SQL scenarios based interview Q&As on teams & matches played

17: Coding Scala Way – What is wrong with this code?

17: PySpark problem solving Interview Q&As with tutorials – problem 1

17: Python OOP and @property decorator tutorial

17: Q99 – Q103 Scala applicative functors interview Q&As

17: Spark interview Q&As with coding examples in pyspark (i.e. python)

18 Amazon S3 interview Q&As

18: 50+ SQL scenarios based interview Q&As on account balance

18: PySpark problem solving Interview Q&As with tutorials – problem 2

18: Python Abstract Base Classes (i.e. ABC) tutorial

18: Q104 – Q108 Scala typeclass pattern for decoupling with polymorphism

18: Q121 – Q124 Choosing between HDFS & AWS S3 for BigData projects interview Q&As

300+ Java Interview Questions - FAQs

10+ Companion Techs Interview Q&As

300+ Big Data Interview Questions - FAQs

16+ Java Tech Key Areas Interview Q&As

Your Career Tips to go places

Tutorials on Java & Big Data

300+ Java Interview Questions Answered

Disclaimer