Search for a command to run...
Articles tagged with #big-data
In this post we will see the difference between HDFS and Cloud Storage What is cloud object storage? How it is different from HDFS 🥃 Cloud computing makes extensive use of object storage. This has many advantages, including cost, speed, and scalabil...
In this post we will see the top 10 linux command that needs to be remembered 1. sudo 🪴 This SuperUserDo is the most important command Linux newbies will use. Every single command that needs root’s permission, need this sudo command. You can use sud...
Hadoop is an open-source framework for distributed storage and processing. It can be used to store large amounts of data in a reliable, scalable, and inexpensive manner. HDFS is composed of master-slave architecture, which includes the following elem...
📍 Introduction: In this post, we will see how to create a generic data pipeline. 📺 Problem Statement: Suppose we need to pull data from 5 different tables, we can create 5 copy activities. But what if we need to pull data from 100 different tables,...
📍 Introduction: In this post, we will see how to create a generic data set. 📺 Problem Statement: Whenever a pipeline is made a new dataset needs to be created. So if we create a 100 pipeline 100 new data set needs to be created which is not a feasi...
📍 Introduction: In this post, we will see how to create our first data pipeline. Here we will move SQL data to an ADLS storage in CSV format. Steps to create a data pipeline: 👁️🗨️ Create a linked service that will help to set up a connection to ...
