ODSC - Open Data Science

JANUARY 30, 2024

Not only does it involve the process of collecting, storing, and processing data so that it can be used for analysis and decision-making, but these professionals are responsible for building and maintaining the infrastructure that makes this possible; and so much more. Think of data engineers as the architects of the data ecosystem.

Data Engineer 52

Data Engineer Data Engineering Data Engineering Data Engineering 52

ODSC - Open Data Science

JANUARY 18, 2024

Data engineers will also work with data scientists to design and implement data pipelines; ensuring steady flows and minimal issues for data teams. They’ll also work with software engineers to ensure that the data infrastructure is scalable and reliable. Identify your existing data science strengths.

Data Engineer 52

Data Engineer Data Engineering Data Engineering Data Engineering 52

DagsHub

DECEMBER 11, 2023

The following points illustrates some of the main reasons why data versioning is crucial to the success of any data science and machine learning project: Storage space One of the reasons of versioning data is to be able to keep track of multiple versions of the same data which obviously need to be stored as well.

Machine Learning 98

Machine Learning Machine Learning Data Lakes Database 98

ODSC - Open Data Science

DECEMBER 5, 2023

Networking Always a highlight and crowd-pleasure of ODSC conferences, the networking events Monday-Wednesday were well-deserved after long days of data science training sessions. While we may be done with events for 2023, 2024 is looking to be packed full of conferences, meetups, and virtual events. What’s next?

Data Science 98

Data Science Artificial Intelligence Artificial Intelligence Machine Learning 98

ODSC - Open Data Science

MAY 9, 2024

With 2024 surging along, the world of AI and the landscape being created by large language models continues to evolve in a dynamic manner. Innovative AI Tools for 2024 Cosmopedia Now think about this. Whether you’re managing data pipelines or deploying machine learning models, Thunder makes the process smooth and efficient.

Machine Learning 52

Machine Learning Machine Learning AI AI 52

ODSC - Open Data Science

JANUARY 2, 2024

Join us in the city of Boston on April 24th for a full day of talks on a wide range of topics, including Data Engineering, Machine Learning, Cloud Data Services, Big Data Services, Data Pipelines and Integration, Monitoring and Management, Data Quality and Governance, and Data Exploration.

Data Engineer 52

Data Engineer Data Engineering Data Engineering Data Engineering 52

Iguazio

DECEMBER 18, 2023

This means that in 2024, we’re likely to see businesses continue to seek ways to adopt generative AI as a way to enhance their operations. For data science practitioners, productization is key, just like any other AI or ML technology. Will generative AI continue to be one of the hottest topics in 2024 as well?

ML 64

ML ML AI AI 64

ODSC - Open Data Science

FEBRUARY 15, 2024

These systems represent data as knowledge graphs and implement graph traversal algorithms to help find content in massive datasets. These systems are not only useful for a wide range of industries, they are fun for data engineers to work on. So get your pass today, and keep yourself ahead of the curve.

Data Engineering 52

Data Engineering Data Engineer Data Engineering Data Engineering 52

ODSC - Open Data Science

FEBRUARY 6, 2024

Apache Kafka For data engineers dealing with real-time data, Apache Kafka is a game-changer. This open-source streaming platform enables the handling of high-throughput data feeds, ensuring that data pipelines are efficient, reliable, and capable of handling massive volumes of data in real-time.

Data Engineer 52

Data Engineer Data Engineering Data Engineering Data Engineering 52

ODSC - Open Data Science

APRIL 4, 2024

There are many factors, but here, we’d like to hone in on the activities that a data science team engages in. Find out how to weave data reliability and quality checks into the execution of your data pipelines and more. Register now before tickets sell out! Learn more about them here!

Data Visualization 52

Data Visualization Analytics Analytics Big Data Analytics 52

DagsHub

APRIL 7, 2024

Image generated with Midjourney In today’s fast-paced world of data science, building impactful machine learning models relies on much more than selecting the best algorithm for the job. Data scientists and machine learning engineers need to collaborate to make sure that together with the model, they develop robust data pipelines.

Machine Learning 59

Machine Learning Machine Learning ML ML 59

Iguazio

DECEMBER 18, 2023

This means that in 2024, we’re likely to see businesses continue to seek ways to adopt generative AI as a way to enhance their operations. For data science practitioners, productization is key, just like any other AI or ML technology. Will generative AI continue to be one of the hottest topics in 2024 as well?

ML 52

ML ML AI AI 52

Pickl AI

NOVEMBER 4, 2024

Effective data governance enhances quality and security throughout the data lifecycle. What is Data Engineering? Data Engineering is designing, constructing, and managing systems that enable data collection, storage, and analysis. The global data warehouse as a service market was valued at USD 9.06

Data Engineer 52

Data Engineer Data Engineering Data Engineering Data Engineering 52

IBM Journey to AI blog

AUGUST 12, 2024

Wearable devices (such as fitness trackers, smart watches and smart rings) alone generated roughly 28 petabytes (28 billion megabytes) of data daily in 2020. And in 2024, global daily data generation surpassed 402 million terabytes (or 402 quintillion bytes). Massive, in fact.

Big Data 106

Big Data Big Data ML ML 106

ODSC - Open Data Science

FEBRUARY 8, 2024

Answering these questions allows data scientists to develop useful data products that start out simple and can be improved and made more complex over time until the long-term vision is achieved. At the strategy level, we are not interested in what technologies we will use for data warehousing, data pipelines, serving models, etc.

AI 52

AI AI Data Science Data Scientist 52

DagsHub

MAY 7, 2024

This ensures that the knowledge gained by your data science team is utilized by the ML Engineers in later stages of the model/data lifecycle. ML validation continuity: Deepchecks ensures continuity from research to production. The same checks used during research can be used for CI/CD and production monitoring.

Machine Learning 52

Machine Learning Machine Learning ML ML 52

DagsHub

MAY 7, 2024

This ensures that the knowledge gained by your data science team is utilized by the ML Engineers in later stages of the model/data lifecycle. ML validation continuity: Deepchecks ensures continuity from research to production. The same checks used during research can be used for CI/CD and production monitoring.

Machine Learning 52

Machine Learning Machine Learning ML ML 52

DagsHub

DECEMBER 5, 2023

With collaborative coding tools, DagsHub provides a central location for data science teams to visualize, compare, and review their experiments, eliminating the need to set up any infrastructure. DagsHub detects and supports DVC's metrics and params file formats, and it also sets up a DVC remote where we can version our data.

Machine Learning 52

Machine Learning Machine Learning ML ML 52

ODSC - Open Data Science

SEPTEMBER 27, 2024

We carefully curate and share the most impactful AI news & developments, bringing the insights that matter most to the AI and data science community. Subscribe to get this as a newsletter sent to your inbox every Friday!

Artificial Intelligence 40

Artificial Intelligence Artificial Intelligence AI AI 40

Pickl AI

JUNE 7, 2024

This blog will delve into ETL Tools, exploring the top contenders and their roles in modern data integration. Let’s unlock the power of ETL Tools for seamless data handling. Also Read: Top 10 Data Science tools for 2024. Read Further: Azure Data Engineer Jobs. What is ETL?

ETL 40

ETL Data Quality Data Pipeline Data Warehouse 40

phData

DECEMBER 4, 2023

This blog was originally written by Erik Hyrkas and updated for 2024 by Justin Delisi This isn’t meant to be a technical how-to guide — most of those details are readily available via a quick Google search — but rather an opinionated review of key processes and potential approaches. Use with caution, and test before committing to using them.

Clustering 98

Clustering Database SQL Data Pipeline 98

ODSC - Open Data Science

OCTOBER 13, 2024

Lastly, data version control systems, like lakeFS, allow for a Git-like approach to managing data. By versioning datasets in the same way we version code, data teams can experiment, roll back changes, and merge data pipelines safely, all without duplicating data or slowing down operations.

Data Lakes 40

Data Lakes Database Data Pipeline SQL 40

ODSC - Open Data Science

OCTOBER 16, 2024

The upcoming ODSC West 2024 conference provides valuable insights into the key trends shaping the future of LLMs. 1, From Experimentation to Implementation: Building the LLM-Powered Future The theme of building and deploying LLM applications resonates strongly throughout the ODSC West 2024 lineup.

Data Quality 40

Data Quality AI AI Data Pipeline 40

insideBIGDATA

JUNE 4, 2024

Modern data pipeline platform provider Matillion today announced at Snowflake Data Cloud Summit 2024 that it is bringing no-code Generative AI (GenAI) to Snowflake users with new GenAI capabilities and integrations with Snowflake Cortex AI, Snowflake ML Functions, and support for Snowpark Container Services.

Data Pipeline 273

Data Pipeline ML ML AI 273

Pickl AI

MAY 15, 2024

Dreaming of a Data Science career but started as an Analyst? This guide unlocks the path from Data Analyst to Data Scientist Architect. So if you are looking forward to a Data Science career , this blog will work as a guiding light.

Data Analyst 52

Data Analyst Data Scientist Data Science Machine Learning 52

ODSC - Open Data Science

JANUARY 9, 2024

Now that we’re in 2024, it’s important to remember that data engineering is a critical discipline for any organization that wants to make the most of its data. These data professionals are responsible for building and maintaining the infrastructure that allows organizations to collect, store, process, and analyze data.

Data Engineer 97

Data Engineer Data Engineering Data Engineering Data Engineering 97

phData

NOVEMBER 9, 2023

This blog was originally written by Keith Smith and updated for 2024 by Justin Delisi. Snowflake’s Data Cloud has emerged as a leader in cloud data warehousing. Data Sharing: Snowflake allows organizations to securely share data with external partners or customers, making it useful for collaboration and data monetization.

Data Warehouse 98

Data Warehouse Data Lakes Clustering Cloud Data 98