Python is popular in Big Data & data science projects. This tutorial outlines the basic steps to get started with Python on Mac OS.
1. Install Xcode
Xcode can be installed via Apple appstore. Xcode is Apple’s Integrated Development Environment (IDE). Xcode is a large suite of software development tools and libraries from Apple.
…
Q1. What is data backfilling? A1. Backfilling data is a process of reactively processing any missing data for a past time window. Q2. Why do you need to backfill data? A2. … Read more ›...
This extends Architecture Networking. Here are some must know-concepts as a Cloud Engineer, Data Engineer or a Software Engineer.
The above diagram addresses many of the questions that follow.
Q1. What is a VPC in AWS?
A1. A virtual private cloud (VPC) is a virtual network dedicated to your AWS account.
…
This extends Architecture Networking. The below diagram will help you understand many of the following questions. Q01. What do you understand by the Azure terms Tenant, Subscription, Region, … Read more ›...
Example #1: Read from a list & write to a list
Java Way in Scala
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
object ListToList extends App { val inputList = List("Java", "Scala", "Ruby") val outputList = scala.collection.mutable.ListBuffer[String](); for(item <- inputList) { outputList += item + " Programming" } println(outputList.toList) } |
Output: List(Java Programming, Scala Programming, Ruby Programming)
Scala Way in Scala: Using the “map” functional combinator
Immutable code as shown below using the “map”
…
The best way to prepare for the Databricks interview is via the 28 tutorials on getting started with Databricks & PySpark. These tutorials will not only get you started on Databricks, but also help you prepare for the job interviews.
Here you will look at some high level Databricks interview Questions &
…
Q1. What are some of the design considerations for a read heavy system? A1. Before designing any systems, one should gather the functional & non-functional requirements. The SLAs (i.e. Service Level Agreements) have to be clearly defined. A rough-cut capacity planning has to be done in terms of how many...
This extends High level & low level system design considerations for read heavy systems Q1. What are some of the design considerations for a write heavy system? A1. Before designing any systems, one should gather the functional & … Read more ›...
This post shows how to get pyspark installed using the installer for Mac known as the homebrew. Homebrew is a package manager for MacOS (and Linux). It can be used to install, update, and manage various applications and tools on Macs. If you don’t have Homebrew, install it first as a pre-requisite.
…
Q1. What is the Lambda Architecture? A1. It is a data-processing architecture designed to handle Big Data by using both real-time streaming (e.g. Spark streaming, Apache Storm) and batch processing (E.g. Hive, Pig, Spark batch). This means you have to build 2 separate pipelines. … Read more ›...
Q1. What are the key differences between SQL & NoSQL databases? A1. Now a days you have a choice which database to use based on your requirements. There are pros & cons for each database. SQL NoSQL (Not Only SQL) SQL databases store structured data & … Read more ›...
Assumes that Python3 is installed as described in Getting started with Python.
1. Iterators
Iterators don’t compute the value of each item when instantiated. They only compute it when you ask for it. This is known as lazy evaluation. Lazy evaluation is useful when you have a very large data set to compute.
…
Q01. How is Big Data used in industries? A01. The main goal for most organisations is to enhance customer experience, and consequently increase sales. The other goals include cost reduction, better targeted marketing, fraud detection, identifying data breaches to enhance security, making existing processes more efficient, … Read more ›...
I have a good news, I got job offer. I like to share with you from my bottom of my heart feeling thankful to you.