Q01 Often you will have a requirement to report on a certain dimension, which requires you to convert rows into columns or columns into rows. How will you go about accomplishing this using SQL?
A01 The process of converting rows into columns is know as “
…
Example #16: Regular Expression Regular expression (aka regex) to identify numbers in a string. Invoking “findFirstIn” and “findAll” functions of the “Regex” class. “[0-9]+”.r is a short form instead of instantiating it as new Regex(“[0-9]+”).
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
import scala.util.matching.Regex object RegexTest extends App { val numberPattern: Regex = "[0-9]+".r // .r is same as new Regex("[0-9]+") val address = "Suite 121, 79 Belmont Cresent" val result = numberPattern.findFirstIn(address) println(result) //Some(121) println(result.getOrElse("")) //121 val result2 = numberPattern.findAllIn(address) //result2 is Regex.MatchIterator result2.foreach { x => println(s"found a number = $x") } } |
Output
|
1 2 3 4 5 6 |
Some(121) 121 found a number = 121 found a number = 79 |
Example #17: Pattern matching with regex Split the given email address into...
Q01. How will you pass variables or arguments from one notebook to another in Databricks? A01. There are two ways to accomplish this, Firstly, using %run and secondly, using the dbutils.notebook API. %run When you use %run, … Read more ›...
Q16: What are built-in functions in pyspark? A16: There are 4 types of functions in pyspark 1) Built-in function:: from org.apache.spark.sql.functions like to_date(Column e), to_utc_timestamp(Column e), etc. … Read more ›...
Q1. What is a dictionary in Python? A1. A dictionary is a general-purpose data structure for storing a group of objects as a set of keys and each key has a single associated value. A value can be a list or list within a list, … Read more ›...
Q41. What is the purpose of the “async” macro in Scala? A41. The 2.11 and later versions of Scala include the ability to transform code during compilation by using macros. The “async” macro is one of them, which transforms sequential code that uses “Futures” … Read more ›...
Q62. Can you explain the difference between HDFS blocks and input splits? A62. A block is a physical representation of data, and a Split is a logical division of your data or records. For example, an input split might split a large text file at the end of a record...
This extends the Scala Class Vs. Object and Companion object interview Q&As to implement the factory method.
Q1. What is a factory method pattern?
A1. It is an Object Oriented Programming design pattern to instantiate objects. For example,
…
Below question is popular in SQL coding interviews. SQL syntaxes are very easy to learn, but lots of practice is required to be able to understand the business intent, and convert that into SQL statements with proper what if considerations. Break down the requirements to construct the SQL statements.
…
Example #20: nth Fibonacci number
Q1 What is wrong with the following recursive approach to compute the nth Fibonacci number?
A1 It “hangs” for the larger values of “n” like 50.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
object RecursionVsIteration extends App { //1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144 //find nth fibonacci number def fibNaive(n: Long): Long = { if (n <= 1) n else fibNaive(n - 1) + fibNaive(n - 2) } println(fibNaive(8)) // 8th number is 21 println(fibNaive(12)) // 12th number is 144 println(fibNaive(50)) //....hangs } |
Q2 How about using an iterative approach as shown below?
…
Data skipping is a performance optimization that aims at speeding up queries that contain filters (i.e. WHERE clauses). Data can be skipped using partitioning and z-ordering techniques. Q01. What is data partitioning in Databricks? A01. Partitioning involves putting different rows into different folders. … Read more ›...
Given the below employee.csv file in /tmp folder:
|
1 2 3 4 5 6 7 8 9 |
emp_id,emp_name,emp_city,emp_salary 1, John, Sydney, 35000.00 2, Peter, Melbourne, 45000.00 3, Sam, Sydney,55000.00 4, Danny, Melbourne, 45000.00 7, Roger, Sydney, 35000.00 8, Lisa, Sydney, 25000.00 |
Q22: What is the difference between RANK(), ROW_NUMBER(), and DENSE_RANK() windows functions? A22: RANK() assigns the same rank to rows with equal values, … Read more ›...
Q1. What is Python data modelling? A1. Objects are Python’s abstraction for data. All data in a Python program is represented by objects or by relations between objects. As an intermediate to experienced Python developer always refer to the Python’s Data model documentation. … Read more ›...
I have a good news, I got job offer. I like to share with you from my bottom of my heart feeling thankful to you.