Blog Archives
Page 1 of 2
1 2

01: Databricks getting started – Spark, Shell, SQL


Step 1:
Signup to Databricks community edition – https://databricks.com/try-databricks. Fill in the details and you can leave your mobile number blank. Select “COMMUNITY EDITION” ==“GET STARTED



02: Databricks – Spark schemas, casting & PySpark API

Prerequisite: Extends Databricks getting started – Spark, Shell, SQL.

Q: What is a Dataframe?
A: A DataFrame is a data abstraction or a domain-specific language (DSL) for working …



03: Databricks – Spark SCD Type 1

Prerequisite: Extends Databricks getting started – Spark, Shell, SQL.

What is SCD Type 1

SCD stands for Slowly Changing Dimension, and it was explained in 10 Data



04: Databricks – Spark SCD Type 2

Prerequisite: Extends 03: Databricks – Spark SCD Type 1.

What is SCD Type 2

SCD stands for Slowly Changing Dimension, and it was explained in 10 Data



05: Databricks – Spark UDFs

Prerequisite: Extends Databricks getting started – Spark, Shell, SQL.

What is a UDF?

User-Defined Functions (aka UDF) is a feature of Spark SQL to define new …



06: Databricks – Spark Window functions

Prerequisite: Extends Databricks getting started – Spark, Shell, SQL. What is a window function? Q. What are the different types of functions in Spark SQL? A. There are 4 types of …



07: Databricks – groupBy, collect_list & explode

Prerequisite: Extends Databricks – Spark Window functions. Step 1: Create a new Python notebook, and attach it to a cluster. Step 2: Let’s create some data using pyspark.



08: Databricks – Spark problem 1

Prerequisite: Extends Databricks – Spark Window functions. Problem: Convert the below table



09: Databricks – Spark Problem 2

Prerequisite: Extends Databricks – Spark problem 1. Problem: Convert the below table



10: Databricks – Spark ML – Linear Regression

Prerequisite: Extends Databricks getting started – Spark, Shell, SQL.

You can try these tutorials in Scala using Databricks Notebook. There are Scala tutorials covered in Spark using Scala on Zeppelin



11: Databricks – Spark ML – Multivariate Linear Regression

Prerequisite: Extends Databricks – Spark ML – Linear Regression. Problem statement: Predict the house prices by land area in square feet, no of bedrooms, and how old the house is?, which …



11A: Databricks – Spark ML – Pandas Dataframe & Matplotlib

Prerequisite: Extends 11: Databricks – Spark ML – Multivariate Linear Regression. How do you convert Pyspark dataframe to Pndas Dataframe? df.toPandas() converts Pyspark Dataframe to Pandas Dataframe.



12: Databricks – Spark ML – Categorical Features

Prerequisite: Extends Databricks – Spark ML – Linear Regression. Problem statement: Predict the house prices by land area in square feet, house condition as in “Bad”, “Average”, and “Good”, and house …



Page 1 of 2
1 2

800+ Java Interview Q&As Menu

Prepare to fast-track & go places
with multi-offers to choose from & increased earning potential. Expand your horizons along the way by taking the road less travelled.

Career Paths as a Developer

Learn by categories on the go...
Learn by categories such as FAQs – Core Java, Key Area – Low Latency, Core Java – Java 8, JEE – Microservices, Big Data – NoSQL, etc. Some posts belong to multiple categories.
Top