Data models should be filtered and normalized. This is particularly important if datasets are sourced from high data volume repositories such as enterprise data warehouses.
19 Best Practices of Power BI
How to handle duplicate records while inserting data in Databricks
Have you ever faced a challenge where records keep getting duplicated when you are inserting some new data into an existing table in Databricks? If yes, then this blog is for you. Let’s start with a simple use case: Inserting parquet data from one folder in Datalake to a Delta table using Databricks. Follow the... Continue Reading →
How to Copy Files from SharePoint to Datalake using Azure Data factory
Copying files from SharePoint to Datalake or any other target location is one task you cannot ignore as a Data Engineer. Someday someone for sure is going to ask you to do that. So, how can you achieve that? Suppose we need to copy an excel file from SharePoint to Datalake Gen 2. You need... Continue Reading →
Basic concepts anyone starting in SQL should know about
SQL stands for Structured Query Language. SQL is a database management language for relational databases. SQL lets you access and manipulate databases. SQL queries are not case sensitive. (SELECT = select) Division of SQL: Data Manipulation Language (DML) is used to add, update or delete data . Examples: INSERT, DELETE and UPDATE. Data Definition Language (DDL) is used... Continue Reading →
Simple Machine Learning Program in Python to Use Linear Regression to predict House Price.
Requirement: The main requirement is to be able to assign and predict House Price in a given location based on age of house, distance from Airport and number of conveniences stores within 5 km radius.
How to Create Login and Password in SQL Server
This post is to help you create a new login and password for the users who want to use the data in SQL database or Warehouse for Analysis and Reporting.
How to read .csv and .xlsx file in Databricks
How to read .xlsx file: Step 1: In order to read .xlsx file, you need to have the library com.crealytics:spark-excel_2.11:0.12.2 installed in the Databricks cluster. Steps to install library com.crealytics:spark-excel_2.11:0.12.2 to Databircks cluster: Step 1: Select the Databricks cluster where you want to install the library. Step 2: Click on Libraries. Step 3: Click on... Continue Reading →
Everything you need to know about Cloud Computing
Cloud Computing refers to the process of making computing services available to users over the internet.
Everything you need to know about Big Data
Big Data refers to the data which is nearly impossible to process using a single computer as it is huge in size, generated frequently and is in many different formats.
Machine Learning: What is Reinforcement Learning
Reinforcement Learning is an area of Machine Learning which is applied when you don’t have training dataset and want the machine to learn from its experience.
Machine Learning : What is UnSupervised Learning
Unsupervised Learning The basic idea of Unsupervised Learning is to feed machine with as much data as possible without providing any label on the data and letting machine group similar data together to form clusters and provide insights on them. In Unsupervised Learning, the learning algorithm tries to understand relationships between various inputs without any... Continue Reading →
Machine Learning : What is Supervised Learning
In Supervised Machine learning, set of instances are provided to the learning Algorithm, each instance contains an input and an output. The Algorithm learns from these instances and builds a model which can then find output to any new input.
Ization 