ODSC - Open Data Science

FEBRUARY 24, 2023

Great Expectations GitHub | Website Great Expectations (GX) helps data teams build a shared understanding of their data through quality testing, documentation, and profiling. With Great Expectations , data teams can express what they “expect” from their data using simple assertions.

Exploratory Data Analysis 126

Exploratory Data Analysis Data Visualization Data Analysis Data Analysis 126

Smart Data Collective

APRIL 29, 2020

The extraction of raw data, transforming to a suitable format for business needs, and loading into a data warehouse. Data transformation. This process helps to transform raw data into clean data that can be analysed and aggregated. Data analytics and visualisation. Microsoft Azure.

Data Warehouse 98

Data Warehouse SQL Azure ETL 98

Mlearning.ai

FEBRUARY 16, 2023

Today, companies are facing a continual need to store tremendous volumes of data. The demand for information repositories enabling business intelligence and analytics is growing exponentially, giving birth to cloud solutions. Snowflake data warehouses deliver greater capacity without the need for any additional equipment.

Data Warehouse 52

Data Warehouse Business Intelligence Business Intelligence Database 52

AWS Machine Learning Blog

OCTOBER 9, 2024

To start using OpenSearch for anomaly detection you first must index your data into OpenSearch , from there you can enable anomaly detection in OpenSearch Dashboards. To learn more, see the documentation. To learn more, see the documentation. To learn more, see the documentation.

AWS 92

AWS ML ML Data Quality 92

Pickl AI

APRIL 9, 2025

By 2025, global data volumes are expected to reach 181 zettabytes, according to IDC. To harness this data effectively, businesses rely on ETL (Extract, Transform, Load) tools to extract, transform, and load data into centralized systems like data warehouses. What are ETL Tools?

ETL 52

ETL Data Warehouse AWS Business Intelligence 52

Alation

JANUARY 17, 2023

It is known to have benefits in handling data due to its robustness, speed, and scalability. A typical modern data stack consists of the following: A data warehouse. Data ingestion/integration services. Data orchestration tools. Business intelligence (BI) platforms. Better Data Culture.

Data Warehouse 52

Data Warehouse ETL Tableau Cloud Data 52

IBM Journey to AI blog

OCTOBER 16, 2023

It includes processes that trace and document the origin of data, models and associated metadata and pipelines for audits. How to scale AL and ML with built-in governance A fit-for-purpose data store built on an open lakehouse architecture allows you to scale AI and ML while providing built-in governance tools.

AI 88

AI AI Data Warehouse ML 88

Dataconomy

APRIL 27, 2023

Conversely, OLAP systems are optimized for conducting complex data analysis and are designed for use by data scientists, business analysts, and knowledge workers. OLAP systems support business intelligence, data mining, and other decision support applications.

Database 159

Database Data Scientist Data Mining Data Mining 159

phData

SEPTEMBER 19, 2023

By incorporating metadata into the data model, users can easily discover, understand, and interpret the data stored in the lake. With the amounts of data involved, this can be crucial to utilizing a data lake effectively. However, this can be time-consuming and prone to human error, leading to misinformation.

Data Lakes 52

Data Lakes Data Models Data Modeling Data Warehouse 52

Pickl AI

JULY 3, 2023

What is Business Intelligence? Business Intelligence (BI) refers to the technology, techniques, and practises that are used to gather, evaluate, and present information about an organisation in order to assist decision-making and generate effective administrative action. billion in 2015 and reached around $26.50

Business Intelligence 52

Business Intelligence Business Intelligence SQL Data Visualization 52

Pickl AI

AUGUST 9, 2024

Real-world examples illustrate their application, while tools and technologies facilitate effective hierarchical data management in various industries. Document Hierarchy Structures Maintain thorough documentation of hierarchy designs, including definitions, relationships, and data sources.

Data Warehouse 52

Data Warehouse Data Quality ETL Business Intelligence 52

Smart Data Collective

FEBRUARY 20, 2022

In addition, well-known products boast a lot of implementations and use cases that are comprehensively reflected in the documentation. Another direction in the progress of database monitoring systems is the interoperability with so-called data warehouses, which are increasingly popular among corporate customers.

Database 130

Database Machine Learning Machine Learning Data Warehouse 130

Pickl AI

AUGUST 11, 2024

Consider factors such as data volume, query patterns, and hardware constraints. Document and Communicate Maintain thorough documentation of fact table designs, including definitions, calculations, and relationships. These tools are essential for populating fact tables with accurate and timely data.

Data Quality 52

Data Quality Data Warehouse Data Governance Analytics 52

phData

JULY 18, 2023

With the birth of cloud data warehouses, data applications, and generative AI , processing large volumes of data faster and cheaper is more approachable and desired than ever. First up, let’s dive into the foundation of every Modern Data Stack, a cloud-based data warehouse.

Data Warehouse 52

Data Warehouse Analytics Analytics Cloud Data 52

Alation

FEBRUARY 7, 2023

It wouldn’t be until 2013 that the topic of data lineage would surface again – this time while working on a data warehouse project. Data warehouses obfuscate data’s origin In 2013, I was a Business Intelligence Engineer at a financial services company.

Data Warehouse 52

Data Warehouse ETL Business Intelligence Business Intelligence 52

IBM Journey to AI blog

JULY 17, 2023

These encoder-only architecture models are fast and effective for many enterprise NLP tasks, such as classifying customer feedback and extracting information from large documents. While they require task-specific labeled data for fine tuning, they also offer clients the best cost performance trade-off for non-generative use cases.

AI 74

AI AI Machine Learning Machine Learning 74

IBM Journey to AI blog

OCTOBER 12, 2023

To optimize data analytics and AI workloads, organizations need a data store built on an open data lakehouse architecture. This type of architecture combines the performance and usability of a data warehouse with the flexibility and scalability of a data lake.

AI 93

AI AI Data Scientist Data Governance 93

Alation

JULY 19, 2022

For instance, Alation can return wiki-like articles, conversations, and business intelligence objects, in addition to traditional tables. Documenting data science projects this way in a data catalog saves time and prevents data scientists from wasting resources on duplicate projects.

Data Scientist 59

Data Scientist Data Quality Data Science Data Analyst 59

IBM Journey to AI blog

JULY 3, 2024

As data types and applications evolve, you might need specialized NoSQL databases to handle diverse data structures and specific application requirements. This can be achieved by building data products, AI assistants, AI applications and business intelligence solutions powered by an AI and data platform that uses your trusted data.

AI 45

AI AI Data Quality Database 45

phData

MAY 25, 2023

How to Optimize Power BI and Snowflake for Advanced Analytics Spencer Baucke May 25, 2023 The world of business intelligence and data modernization has never been more competitive than it is today. Microsoft Power BI has been the leader in the analytics and business intelligence platforms category for several years running.

Power BI 84

Power BI Analytics Analytics Azure 84

Precisely

APRIL 3, 2023

It simply wasn’t practical to adopt an approach in which all of an organization’s data would be made available in one central location, for all-purpose business analytics. To speed analytics, data scientists implemented pre-processing functions to aggregate, sort, and manage the most important elements of the data.

ETL 52

ETL Analytics Analytics Database 52

AWS Machine Learning Blog

JUNE 25, 2024

To create and share customer feedback analysis without the need to manage underlying infrastructure, Amazon QuickSight provides a straightforward way to build visualizations, perform one-time analysis, and quickly gain business insights from customer feedback, anytime and on any device. The LLM generates output based on the user prompt.

AWS 125

AWS Natural Language Processing Machine Learning Machine Learning 125

Alation

MAY 24, 2022

With the Open Data Quality Initiative, Alation introduces an Open Data Quality Framework (ODQF), which includes a starter kit for data quality partners. This kit offers an open DQ API, developer documentation, onboarding, integration best practices, and co-marketing support.

Data Quality 98

Data Quality Data Governance ETL Data Observability 98

phData

AUGUST 10, 2023

The implementation of a data vault architecture requires the integration of multiple technologies to effectively support the design principles and meet the organization’s requirements. Data Acquisition: Extracting data from source systems and making it accessible. as well as calculating business keys.

SQL 52

SQL Data Observability Data Quality Data Pipeline 52

Alation

DECEMBER 21, 2022

Transactional systems and data warehouses can then use the golden records as the entity’s most current, trusted representation. Data Catalog and Master Data Management. Having good data is crucial to creating golden records. MDM Build Objects. The MDM tool should help with this.

Data Quality 52

Data Quality Data Warehouse Data Profiling Data Governance 52

Alation

SEPTEMBER 7, 2021

With Snowflake, data stewards have a choice to leverage Snowflake’s governance policies. First, stewards are dependent on data warehouse admins to provide information and to create and edit enforcement policies in Snowflake. Data quality details signal to users whether data can be trusted or used.

Data Governance 59

Data Governance Data Scientist Data Quality Data Profiling 59

Alation

APRIL 14, 2022

Similar to a data warehouse schema, this prep tool automates the development of the recipe to match. Edge computing can be decentralized from on-premises, cellular, data centers, or the cloud. Problems arise when data sources are semantically incompatible. On-premises business intelligence and databases.

Data Governance 59

Data Governance ML ML Cloud Data 59

phData

OCTOBER 11, 2023

Consider a scenario: a data repository residing within a cloud-based data warehouse. KNIME has become a handy tool to prepare and optimize the data. Microsoft Power BI is a dynamic and interactive data visualization platform primarily focusing on business intelligence. See the Power BI documentation.

Power BI 45

Power BI Data Preparation Analytics Analytics 45

Alation

JUNE 9, 2021

We have an explosion, not only in the raw amount of data, but in the types of database systems for storing it ( db-engines.com ranks over 340) and architectures for managing it (from operational datastores to data lakes to cloud data warehouses). Organizations are drowning in a deluge of data.

Data Governance 59

Data Governance Database Cloud Data Machine Learning 59

phData

JULY 21, 2023

Before understanding this data storage, let us know a bit about Tableau. Tableau is one of the most popular data visualization and business intelligence tools that help people see and understand their data. The post Where Does Tableau Store Data? There are mainly 5 components in Tableau architecture.

Tableau 52

Tableau Database Data Engineering Data Engineering 52

phData

JULY 21, 2023

Before understanding this data storage, let us know a bit about Tableau. Tableau is one of the most popular data visualization and business intelligence tools that help people see and understand their data. The post Where Does Tableau Store Data? There are mainly 5 components in Tableau architecture.

Tableau 52

Tableau Database Data Engineering Data Engineering 52

DagsHub

NOVEMBER 11, 2024

So, we must understand the different unstructured data types and effectively process them to uncover hidden patterns. Textual Data Textual data is one of the most common forms of unstructured data and can be in the format of documents, social media posts, emails, web pages, customer reviews, or conversation logs.

AI 52

AI AI Data Lakes Database 52

DagsHub

AUGUST 23, 2024

TDWI Data Quality Framework This framework , developed by the Data Warehousing Institute, focuses on practical methodologies and tools that address managing data quality across various stages of the data lifecycle, including data integration, cleaning, and validation.

Data Quality 52

Data Quality Data Governance Machine Learning Machine Learning 52

phData

MAY 11, 2023

When a new entrant to ETL development reads this article, they could easily have mastered Matillion Designer’s methods or read through the Matillion Versioning Documentation to develop their own approach to ZDLC. A common data type transformation is to convert date fields that were loaded as strings into actual DATE or DATETIME data types.

ETL 52

ETL Analytics Analytics Data Modeling 52

Pickl AI

MARCH 19, 2025

It is widely used for storing and managing structured data, making it an essential tool for data engineers. MongoDB MongoDB is a NoSQL database that stores data in flexible, JSON-like documents. Apache Spark Apache Spark is a powerful data processing framework that efficiently handles Big Data.

Data Engineer 52

Data Engineer Data Engineering Data Engineering Data Engineering 52

Pickl AI

FEBRUARY 23, 2025

Look for features such as scalability (the ability to handle growing datasets), performance (speed of processing), ease of use (user-friendly interfaces), integration capabilities (compatibility with existing systems), security measures (data protection features), and pricing models (licensing costs).

Big Data 52

Big Data Big Data Apache Hadoop Apache Kafka 52

Flipboard

MAY 9, 2025

This pattern creates a comprehensive solution that transforms raw social media data into actionable business intelligence (BI) through advanced AI capabilities. 3B Instruct Amazon Bedrock, the system provides tailored marketing content that adds business value. By integrating LLMs such as Anthropics Claude 3.5

94

94