Distributed Processing In Map ReduceDistributed processing in MapReduce may be summarized in three phases: a map phase, a shuffle phase, and a reduce phase. These phases can be overlapped to some degree to improve efficiency. The map step applies the map function to data local to the p...Jan 16, 2024·2 min read
HDFS Architecture 💥Hadoop is an open-source framework for distributed storage and processing. It can be used to store large amounts of data in a reliable, scalable, and inexpensive manner. HDFS is composed of master-slave architecture, which includes the following elem...Dec 27, 2023·1 min read
Differences Between a Data Warehouse, Data Lake, and a Database 🧑🎓What's a Database, and How Does it Work? 🙋♂️ A database is used to store, search and report on structured data from a single source. They are the simplest to create and SQL can be used to query and report on the data. There are both open source and...Dec 26, 2023·3 min read
Apache Spark 🖐️What Is Apache Spark? 🪴 Apache Spark is an open source analytics engine used for big data workloads. It can handle both batches as well as real-time analytics and data processing workloads. Apache Spark started in 2009 as a research project at the U...Dec 25, 2023·2 min read
Hadoop Overview 🧑💻What is Hadoop ? 🧿 Hadoop is an Apache open source framework written in java that allows distributed processing of large datasets across clusters of computers using simple programming models. The Hadoop framework application works in an environment ...Dec 22, 2023·1 min read
Distributed System 🎟️Distributed systems are a collection of independent components and machines located on different systems, communicating in order to operate as a single unit. Types of distributed systems 🥏 There are many models and architectures of distributed syste...Dec 21, 2023·2 min read
Big Data Fundamentals 🌍What is a big data ? 👁️🗨️ There can be various definations of big data but lets consider a simple one. Any data that is characterized by the 5 V's is considered as a big data. 🗺️ Volume - Huge amount of data which any traditional system cannot h...Dec 20, 2023·1 min read
