Data Science Dojo

JULY 3, 2024

Data Engineering : Building and maintaining data pipelines, ETL (Extract, Transform, Load) processes, and data warehousing. Artificial Intelligence : Concepts of AI include neural networks, natural language processing (NLP), and reinforcement learning.

Data Science 150

Data Science Machine Learning Machine Learning Data Visualization 150

Pickl AI

JULY 25, 2023

Data engineers are essential professionals responsible for designing, constructing, and maintaining an organization’s data infrastructure. They create data pipelines, ETL processes, and databases to facilitate smooth data flow and storage. Big Data Processing: Apache Hadoop, Apache Spark, etc.

Data Engineering 52

Data Engineering Data Engineering Data Engineer Data Engineering 52

ODSC - Open Data Science

FEBRUARY 2, 2023

Cloud Services The only two to make multiple lists were Amazon Web Services (AWS) and Microsoft Azure. Most major companies are using one of the two, so excelling in one or the other will help any aspiring data scientist. Saturn Cloud is picking up a lot of momentum lately too thanks to its scalability.

Data Science 98

Data Science Data Scientist Computer Science Computer Science 98

ODSC - Open Data Science

JANUARY 18, 2024

This individual is responsible for building and maintaining the infrastructure that stores and processes data; the kinds of data can be diverse, but most commonly it will be structured and unstructured data. They’ll also work with software engineers to ensure that the data infrastructure is scalable and reliable.

Data Engineering 52

Data Engineering Data Engineering Data Engineer Data Engineering 52

IBM Journey to AI blog

AUGUST 6, 2024

Today, all leading CSPs, including Amazon Web Services (AWS Lambda), Microsoft Azure (Azure Functions) and IBM (IBM Cloud Code Engine) offer serverless platforms. Specifically, serverless helps enable something called event-driven AI, where a constant flow of intelligence informs real-time decision-making capabilities.

Cloud Computing 45

Cloud Computing Internet of Things Big Data Big Data 45

Pickl AI

OCTOBER 15, 2024

It supports both batch and real-time data processing , making it highly versatile. Its ability to integrate with cloud platforms like AWS and Azure makes it an excellent choice for businesses moving to the cloud. Apache Nifi Apache Nifi is an open-source ETL tool that automates data flow between systems.

ETL 52

ETL Azure AWS Data Governance 52

ODSC - Open Data Science

FEBRUARY 6, 2024

Apache Kafka For data engineers dealing with real-time data, Apache Kafka is a game-changer. This open-source streaming platform enables the handling of high-throughput data feeds, ensuring that data pipelines are efficient, reliable, and capable of handling massive volumes of data in real-time.

Data Engineering 52

Data Engineering Data Engineering Data Engineer Data Engineering 52

Precisely

MAY 23, 2024

Whatever your approach may be, enterprise data integration has taken on strategic importance. Artificial intelligence (AI) algorithms are trained to detect anomalies. Today’s enterprises need real-time or near-real-time performance, depending on the specific application. Timing matters.

Data Governance 64

Data Governance ETL Data Pipeline Azure 64

Precisely

MAY 23, 2024

Whatever your approach may be, enterprise data integration has taken on strategic importance. Artificial intelligence (AI) algorithms are trained to detect anomalies. Today’s enterprises need real-time or near-real-time performance, depending on the specific application. Timing matters.

Data Governance 52

Data Governance ETL Data Pipeline Azure 52

Pickl AI

OCTOBER 17, 2024

Automation Automation plays a pivotal role in streamlining ETL processes, reducing the need for manual intervention, and ensuring consistent data availability. By automating key tasks, organisations can enhance efficiency and accuracy, ultimately improving the quality of their data pipelines.

ETL 52

ETL Data Warehouse Data Quality Data Governance 52

phData

JULY 5, 2024

The release of ChatGPT in late 2022 introduced generative artificial intelligence to the general public and triggered a new wave of AI-oriented companies, products, and open-source projects that provide tools and frameworks to enable enterprise AI.

AI 52

AI AI Database AWS 52

AWS Machine Learning Blog

DECEMBER 20, 2023

This unified schema streamlines downstream consumption and analytics because the data follows a standardized schema and new sources can be added with minimal data pipeline changes. After the security log data is stored in Amazon Security Lake, the question becomes how to analyze it.

AWS 123

AWS ML ML Algorithm 123

phData

SEPTEMBER 25, 2024

It can onboard chunks of data from different systems into one. Salesforce offers a wide range of tools and services integrated with artificial intelligence called the Einstein platform. It can be hosted on major cloud platforms like AWS, Azure, and GCP.

ETL 52

ETL Database Data Warehouse Analytics 52

Pickl AI

JULY 25, 2024

Improved Decision-making By providing a consolidated and accessible view of data, organisations can identify trends, patterns, and anomalies more quickly, leading to better-informed and timely decisions. Data Ingestion Tools To facilitate the process, various tools and technologies are available. The post What is Data Ingestion?

Apache Kafka 52

Apache Kafka Data Lakes Data Warehouse Data Quality 52

ODSC - Open Data Science

JANUARY 3, 2025

Participants will dive into building real-world AI applications such as chatbots, AI agents, RAG systems, recommendation engines, and data pipelines. Finally, the session will guide attendees through combining agents into full-scale workflows that utilize agentic techniques for reflection and error correction.

AI 40

AI AI ML ML 40

Heartbeat

SEPTEMBER 18, 2023

Whether you rely on cloud-based services like Amazon SageMaker , Google Cloud AI Platform, or Azure Machine Learning or have developed your custom ML infrastructure, Comet integrates with your chosen solution. It goes beyond compatibility with open-source solutions and extends its support to managed services and in-house ML platforms.

ML 52

ML ML Data Scientist Machine Learning 52

The MLOps Blog

FEBRUARY 27, 2025

Think about it this way: it is easy to integrate GDPR-compliant services like ChatGPTs enterprise version or to use AI models in a law-compliant way through platforms such as Azures OpenAI offering , as providers take the necessary steps to ensure their platforms are compliant with regulations.

Machine Learning 64

Machine Learning Machine Learning AI AI 64

ODSC - Open Data Science

JANUARY 7, 2025

Data Engineering Data engineering remains integral to many data science roles, with workflow pipelines being a key focus. Tools like Apache Airflow are widely used for scheduling and monitoring workflows, while Apache Spark dominates big data pipelines due to its speed and scalability.

Data Scientist 40

Data Scientist Data Science Machine Learning Machine Learning 40

AWS Machine Learning Blog

SEPTEMBER 11, 2024

These activities cover disparate fields such as basic data processing, analytics, and machine learning (ML). And finally, some activities, such as those involved with the latest advances in artificial intelligence (AI), are simply not practically possible, without hardware acceleration.

AWS 91

AWS ML ML Clustering 91

Mlearning.ai

JUNE 16, 2023

There is a VSCode Extension that enables its integration into traditional development pipelines. How to use the Codex models to work with code - Azure OpenAI Service Codex is the model powering Github Copilot. GPT-4 Data Pipelines: Transform JSON to SQL Schema Instantly Blockstream’s public Bitcoin API.

AI 52

AI AI Data Analysis Data Analysis 52

The MLOps Blog

MARCH 21, 2023

You don’t need a bigger boat : The repository curated by Jacopo Tagliabue shows how several (mostly open-source) tools can be effectively combined together to run data pipelines at scale with very small teams. Solution Data lakes and warehouses are the two key components of any data pipeline.

Machine Learning 69

Machine Learning Machine Learning Data Scientist ML 69

phData

JULY 18, 2023

Key Advantages of Governance Simplified Change Managment: The complexity of the underlying systems is abstracted away from the user, allowing them to simply and declaratively build and change data pipelines. phData is a team composed of highly skilled data scientists, consultants, engineers, and architects.

Data Warehouse 52

Data Warehouse Analytics Analytics Cloud Data 52