Analytics Vidhya

OCTOBER 28, 2022

This article was published as a part of the Data Science Blogathon. Introduction Data is defined as information that has been organized in a meaningful way. Data collection is critical for businesses to make informed decisions, understand customers’ […]. The post Data Lake or Data Warehouse- Which is Better?

Data Warehouse 373

Data Warehouse Data Lakes Data Science Analytics 373

Analytics Vidhya

SEPTEMBER 28, 2022

This article was published as a part of the Data Science Blogathon. Introduction The concept of data warehousing dates to the 1980s. DHW, short for Data Warehouse, was presented first by great IBM researchers Barry Devlin and Paul […]. The post Data Warehouse for the Beginners!

Data Warehouse 319

Data Warehouse Computer Science Computer Science Data Science 319

Analytics Vidhya

JULY 22, 2022

This article was published as a part of the Data Science Blogathon Introduction Google’s BigQuery is an enterprise-grade cloud-native data warehouse. Since its inception, BigQuery has evolved into a more economical and fully managed data warehouse that can run lightning-fast […].

Data Engineering 344

Data Engineering Data Engineering Data Engineer Data Engineering 344

Analytics Vidhya

JULY 6, 2021

ArticleVideo Book This article was published as a part of the Data Science Blogathon Introduction A Data Warehouse is Built by combining data from multiple. The post A Brief Introduction to the Concept of Data Warehouse appeared first on Analytics Vidhya.

Data Warehouse 319

Data Warehouse Data Science Analytics Analytics 319

Analytics Vidhya

SEPTEMBER 28, 2022

This article was published as a part of the Data Science Blogathon. Introduction to Data Warehouse SQL Data Warehouse is also a cloud-based data warehouse that uses Massively Parallel Processing (MPP) to run complex queries across petabytes of data rapidly. Import big […].

Data Warehouse 271

Data Warehouse Azure SQL Big Data 271

Analytics Vidhya

MAY 30, 2021

ArticleVideo Book This article was published as a part of the Data Science Blogathon Different components in the Hadoop Framework Introduction Hadoop is. The post HIVE – A DATA WAREHOUSE IN HADOOP FRAMEWORK appeared first on Analytics Vidhya.

Hadoop 320

Hadoop Data Warehouse Data Science Analytics 320

Analytics Vidhya

SEPTEMBER 14, 2021

This article was published as a part of the Data Science Blogathon Introduction Data Science is a team sport, we have members adding value across the analytics/data science lifecycle so that it can drive the transformation by solving challenging business problems.

Data Engineering 390

Data Engineering Data Engineering Data Engineer Data Engineering 390

Analytics Vidhya

APRIL 29, 2022

This article was published as a part of the Data Science Blogathon. Introduction Have you ever wondered how big IT giants store and process huge amounts of data? storing the data […]. storing the data […].

Data Warehouse 267

Data Warehouse Database Data Science Analytics 267

Analytics Vidhya

MARCH 17, 2022

This article was published as a part of the Data Science Blogathon. Machine learning and artificial intelligence, which are at the top of the list of data science capabilities, aren’t just buzzwords; many companies are keen to implement them.

Data Engineering 327

Data Engineering Data Engineering Data Engineer Data Engineering 327

insideBIGDATA

APRIL 6, 2023

In this contributed article, data engineer Koushik Nandiraju discusses how a predictive data and analytics platform aligned with business objectives is no longer an option but a necessity.

Predictive Analytics 367

Predictive Analytics Analytics Analytics Data Warehouse 367

Analytics Vidhya

JULY 29, 2022

This article was published as a part of the Data Science Blogathon. The post How a Delta Lake is Process with Azure Synapse Analytics appeared first on Analytics Vidhya.

Azure 398

Azure Data Warehouse Data Lakes Analytics 398

Analytics Vidhya

SEPTEMBER 1, 2021

This article was published as a part of the Data Science Blogathon A data scientist’s ability to extract value from data is closely related to how well-developed a company’s data storage and processing infrastructure is.

ETL 292

ETL Data Engineering Data Engineering Data Engineer 292

Analytics Vidhya

AUGUST 3, 2022

This article was published as a part of the Data Science Blogathon. Introduction Regarding data analytics, getting insights from a data mart instead of a data warehouse or external data sources can save companies time and produce more targeted results. The idea of ??data

Data Engineering 208

Data Engineering Data Engineering Data Engineer Data Engineering 208

Analytics Vidhya

JULY 18, 2022

This article was published as a part of the Data Science Blogathon. Introduction Organizations with a separate transactional database and data warehouse typically have many data engineering activities. For example, they extract, transform and load data from various sources into their data warehouse.

ETL 291

ETL Data Warehouse Data Engineering Data Engineering 291

Analytics Vidhya

SEPTEMBER 27, 2022

This article was published as a part of the Data Science Blogathon. Businesses have adopted Snowflake as migration from on-premise enterprise data warehouses (such as Teradata) or a more flexibly scalable and easier-to-manage alternative to […].

Data Warehouse 397

Data Warehouse Data Science Analytics Analytics 397

Analytics Vidhya

JUNE 30, 2022

This article was published as a part of the Data Science Blogathon. Introduction Hive is a popular data warehouse built on top of Hadoop that is used by companies like Walmart, Tiktok, and AT&T. It is an important technology for data engineers to learn and master.

Data Warehouse 271

Data Warehouse Hadoop Data Engineering Data Engineering 271

Analytics Vidhya

OCTOBER 28, 2021

This article was published as a part of the Data Science Blogathon What is the need for Hive? The official description of Hive is- ‘Apache Hive data warehouse software project built on top of Apache Hadoop for providing data query and analysis.

Apache Hadoop 349

Apache Hadoop Data Warehouse Hadoop SQL 349

Analytics Vidhya

NOVEMBER 1, 2021

This article was published as a part of the Data Science Blogathon What is ETL? ETL is a process that extracts data from multiple source systems, changes it (through calculations, concatenations, and so on), and then puts it into the Data Warehouse system. ETL stands for Extract, Transform, and Load.

ETL 291

ETL Data Warehouse Data Science Analytics 291

Analytics Vidhya

SEPTEMBER 18, 2022

This article was published as a part of the Data Science Blogathon. Introduction Apache SQOOP is a tool designed to aid in the large-scale export and import of data into HDFS from structured data repositories. Relational databases, enterprise data warehouses, and NoSQL systems are all examples of data storage.

Data Warehouse 271

Data Warehouse Data Science Database Analytics 271

Analytics Vidhya

DECEMBER 26, 2022

This article was published as a part of the Data Science Blogathon. Overview ETL (Extract, Transform, and Load) is a very common technique in data engineering. Traditionally, ETL processes are […].

ETL 306

ETL AWS Data Engineering Data Engineering 306

Analytics Vidhya

JUNE 10, 2021

ArticleVideo Book This article was published as a part of the Data Science Blogathon Introduction Amazon Redshift is a data warehouse service in the cloud. The post Understand All About Amazon Redshift! appeared first on Analytics Vidhya.

Data Warehouse 223

Data Warehouse Data Science Analytics Analytics 223

Analytics Vidhya

DECEMBER 28, 2022

This article was published as a part of the Data Science Blogathon. Source: [link] Introduction If you are familiar with databases, or data warehouses, you have probably heard the term “ETL.” As the amount of data at organizations grow, making use of that data in analytics to derive business insights grows as well.

ETL 212

ETL AWS Data Warehouse Data Science 212

Analytics Vidhya

MAY 10, 2022

This article was published as a part of the Data Science Blogathon. Data Engineers, I am sure this simple article will help you guys better understand Cosmos DB from Azure with nice features. Recently many customers have been looking forward to implementing the Data Migration into Cosmos DB.

Azure 271

Azure Data Engineering Data Engineering Data Engineer 271

Analytics Vidhya

AUGUST 18, 2022

This article was published as a part of the Data Science Blogathon. Introduction More often than not, developers run into issues of an application running on one machine versus not running on another. Dockers help prevent this by ensuring the application runs on any machine if it works on yours. Simply put, if your job as […].

Data Science 400

Data Science Analytics Analytics Data Warehouse 400

Analytics Vidhya

JULY 29, 2022

This article was published as a part of the Data Science Blogathon. Introduction Processing large amounts of raw data from various sources requires appropriate tools and solutions for effective data integration. Building an ETL pipeline using Apache […].

ETL 383

ETL Data Science Analytics Analytics 383

Dataconomy

FEBRUARY 23, 2023

Data engineers play a crucial role in managing and processing big data. They are responsible for designing, building, and maintaining the infrastructure and tools needed to manage and process large volumes of data effectively. What is data engineering?

Big Data 91

Big Data Big Data Data Engineering Data Engineering 91

Analytics Vidhya

JULY 20, 2022

This article was published as a part of the Data Science Blogathon. Introduction Data acclimates to countless shapes and sizes to complete its journey from a source to a destination. The post Developing an End-to-End Automated Data Pipeline appeared first on Analytics Vidhya.

Data Pipeline 361

Data Pipeline ETL Data Science Analytics 361

Analytics Vidhya

OCTOBER 10, 2022

This article was published as a part of the Data Science Blogathon. Introduction Most of you would know the different approaches for building a data and analytics platform. You would have already worked on systems that used traditional warehouses or Hadoop-based data lakes. Selecting one among […].

Data Lakes 350

Data Lakes Hadoop Data Science Analytics 350

Analytics Vidhya

OCTOBER 10, 2022

This article was published as a part of the Data Science Blogathon. Introduction In the modern data world, Lakehouse has become one of the most discussed topics for building a data platform.

Data Lakes 358

Data Lakes Data Science Analytics Analytics 358

Analytics Vidhya

FEBRUARY 28, 2023

This data is used by an organization to find valuable insights which help in improving an organization’s growth and strategies and give them an upper hand over its competitors. This article explains to you the idea […] The post Understanding Dimensional Modeling appeared first on Analytics Vidhya.

Analytics 326

Analytics Analytics Data Warehouse Data Engineering 326

Analytics Vidhya

NOVEMBER 19, 2022

This article was published as a part of the Data Science Blogathon. Introduction The rate of data expansion in this decade is rapid. The requirement to process and store these data has also become problematic.

Cloud Data 329

Cloud Data Data Science Analytics Analytics 329

Analytics Vidhya

JULY 25, 2022

This article was published as a part of the Data Science Blogathon. Introduction These days companies seem to seek ways to integrate data from multiple sources to earn a competitive advantage over other businesses.

Data Pipeline 327

Data Pipeline Data Science Analytics Analytics 327

Analytics Vidhya

DECEMBER 31, 2022

This article was published as a part of the Data Science Blogathon. Introduction Data sharing has become so easy today, and we can share the details with just a few clicks. The post How to Encrypt and Decrypt the Data in PySpark? These details can get leaked if the […].

Data Science 317

Data Science Analytics Analytics Data Warehouse 317

Analytics Vidhya

FEBRUARY 4, 2023

Introduction This article will explain the difference between ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) when data transformation occurs. In ETL, data is extracted from multiple locations to meet the requirements of the target data file and then placed into the file.

ETL 297

ETL Analytics Analytics Data Warehouse 297

Analytics Vidhya

AUGUST 16, 2022

This article was published as a part of the Data Science Blogathon. Introduction The Data science pipeline is the procedure and equipment used to compile raw data from many sources, evaluate it, and display the findings in a clear and concise manner.

Data Science 291

Data Science Analytics Analytics Data Warehouse 291

Analytics Vidhya

JULY 10, 2022

This article was published as a part of the Data Science Blogathon. Introduction With the development of data-driven applications, the complexity of integrating data from multiple simple decision-making sources is often considered a significant challenge.

Data Pipeline 267

Data Pipeline Data Science Analytics Analytics 267

Pickl AI

NOVEMBER 4, 2024

Summary: The fundamentals of Data Engineering encompass essential practices like data modelling, warehousing, pipelines, and integration. Understanding these concepts enables professionals to build robust systems that facilitate effective data management and insightful analysis. What is Data Engineering?

Data Engineering 52

Data Engineering Data Engineering Data Engineer Data Engineering 52