ODSC - Open Data Science

FEBRUARY 2, 2023

Scale is worth knowing if you’re looking to branch into data engineering and working with big data more as it’s helpful for scaling applications. This includes popular tools like Apache Airflow for scheduling/monitoring workflows, while those working with big data pipelines opt for Apache Spark.

Data Science 98

Data Science Data Scientist Computer Science Computer Science 98

Data Science Dojo

JULY 3, 2024

R : Often used for statistical analysis and data visualization. Data Visualization : Techniques and tools to create visual representations of data to communicate insights effectively. Tools like Tableau, Power BI, and Python libraries such as Matplotlib and Seaborn are commonly taught.

Data Science 150

Data Science Machine Learning Machine Learning Data Visualization 150

Alation

APRIL 4, 2023

However, the race to the cloud has also created challenges for data users everywhere, including: Cloud migration is expensive, migrating sensitive data is risky, and navigating between on-prem sources is often confusing for users. To build effective data pipelines, they need context (or metadata) on every source.

DataOps 52

DataOps Data Engineering Data Engineer Data Engineering 52

IBM Journey to AI blog

SEPTEMBER 19, 2023

By analyzing datasets, data scientists can better understand their potential use in an algorithm or machine learning model. The data science lifecycle Data science is iterative, meaning data scientists form hypotheses and experiment to see if a desired outcome can be achieved using available data.

Data Science 94

Data Science Analytics Analytics Data Scientist 94

ODSC - Open Data Science

APRIL 26, 2023

Because they are the most likely to communicate data insights, they’ll also need to know SQL, and visualization tools such as Power BI and Tableau as well. Some of the tools and techniques unique to business analysts are pivot tables, financial modeling in Excel, Power BI Dashboards for forecasting, and Tableau for similar purposes.

Data Analyst 52

Data Analyst Machine Learning Machine Learning Power BI 52

phData

MAY 25, 2023

Features like Power BI Premium Large Dataset Storage and Incremental Refresh should be considered for importing large data volumes. Although a majority of use cases for tools like Tableau or Power BI rely on cached data, use cases like near real-time reporting need to utilize direct queries.

Power BI 84

Power BI Analytics Analytics Azure 84

phData

DECEMBER 4, 2023

The software you might use OAuth with includes: Tableau Power BI Sigma Computing If so, you will need an OAuth provider like Okta, Microsoft Azure AD, Ping Identity PingFederate, or a Custom OAuth 2.0 Data Pipelines “Data pipeline” means moving data in a consistent, secure, and reliable way at some frequency that meets your requirements.

Clustering 98

Clustering Database SQL Data Pipeline 98

Mlearning.ai

JULY 11, 2023

This includes important stages such as feature engineering, model development, data pipeline construction, and data deployment. For instance, feature engineering and exploratory data analysis (EDA) often require the use of visualization libraries like Matplotlib and Seaborn.

Data Science 40

Data Science Exploratory Data Analysis Azure Power BI 40

Alation

JANUARY 17, 2023

It began when some of the popular cloud data warehouses — such as BigQuery, Redshift , and Snowflake — started to appear in the early 2010s. Later, BI tools such as Chartio, Looker, and Tableau arrived on the data scene. Powered by cloud computing, more data professionals have access to the data, too.

Data Warehouse 52

Data Warehouse ETL Tableau Cloud Data 52

ODSC - Open Data Science

JANUARY 7, 2025

Scala is worth knowing if youre looking to branch into data engineering and working with big data more as its helpful for scaling applications. Knowing all three frameworks covers the most ground for aspiring data science professionals, so you cover plenty of ground knowing thisgroup.

Data Scientist 40

Data Scientist Data Science Machine Learning Machine Learning 40

phData

FEBRUARY 14, 2023

Source data formats can only be Parquer, JSON, or Delimited Text (CSV, TSV, etc.). Streamsets Data Collector StreamSets Data Collector Engine is an easy-to-use data pipeline engine for streaming, CDC, and batch ingestion from any source to any destination. The biggest reason is the ease of use.

Data Warehouse 59

Data Warehouse Azure AWS Database 59

Mlearning.ai

FEBRUARY 16, 2023

Thus, the solution allows for scaling data workloads independently from one another and seamlessly handling data warehousing, data lakes , data sharing, and engineering. Further, Snowflake enables easy integrations with numerous business intelligence tools, including PowerBI, Looker, and Tableau.

Data Warehouse 52

Data Warehouse Business Intelligence Business Intelligence Database 52

Alation

DECEMBER 23, 2021

The elf teams used data engineering to improve gift matching and deployed big data to scale the naughty and nice list long ago , before either approach was even considered within our warmer climes. And Santa was hoping to make 2021 his most data-driven year yet. No Need to Pout or Cry: Just Know Your Data!

Data Governance 59

Data Governance Data Pipeline Tableau Big Data 59

Pickl AI

MARCH 19, 2025

Summary: Data engineering tools streamline data collection, storage, and processing. Tools like Python, SQL, Apache Spark, and Snowflake help engineers automate workflows and improve efficiency. Learning these tools is crucial for building scalable data pipelines. Thats where data engineering tools come in!

Data Engineering 52

Data Engineering Data Engineer Data Engineering Data Engineering 52

ODSC - Open Data Science

MARCH 12, 2025

Data Engineerings SteadyGrowth 20182021: Data engineering was often mentioned but overshadowed by modeling advancements. 20222024: As AI models required larger and cleaner datasets, interest in data pipelines, ETL frameworks, and real-time data processing surged.

Data Science 52

Data Science Machine Learning Machine Learning Data Engineering 52