Hacker News

JULY 4, 2023

The second section will delve more deeply into the various approaches that can be used to handle recursive schema definitions. The following Go Arrow schema definition provides an example of such a schema, instrumented with a collection of annotations. The depth of this definition cannot be predetermined.

Data Modeling 123

Data Modeling Data Models Data Observability 123

Tableau

AUGUST 29, 2024

Spencer Czapiewski August 29, 2024 - 9:52pm Kirk Munroe Chief Analytics Officer & Founding Partner at Paint with Data Kirk Munroe, Chief Analytics Officer and Founding Partner at Paint with Data and Tableau DataDev Ambassador, explains the value of using relationships in your Tableau data models. over 4 years ago!),

Tableau 52

Tableau SQL Data Modeling Data Models 52

Hacker News

JULY 18, 2023

What if you could automatically shard your PostgreSQL database across any number of servers and get industry-leading performance at scale without any special data modelling steps? Schema-based sharding has almost no data modelling restrictions or special steps compared to unsharded PostgreSQL.

Database 123

Database Data Modeling Data Models SQL 123

Tableau

OCTOBER 7, 2024

Tableau is a leader in the analytics market, known for helping organizations see and understand their data, but we recognize that gaps still exist: while many of our joint customers already benefit from dbt and trust the metrics that result from these workflows, they are often disconnected and obscured from Tableau’s analytics layer.

Tableau 137

Tableau Data Quality Analytics Analytics 137

Pickl AI

AUGUST 9, 2024

Hierarchies align data modelling with business processes, making it easier to analyse data in a context that reflects real-world operations. Designing Hierarchies Designing effective hierarchies requires careful consideration of the business requirements and the data model.

Data Warehouse 52

Data Warehouse Data Quality Business Intelligence Business Intelligence 52

Flipboard

NOVEMBER 20, 2024

By combining the capabilities of LLM function calling and Pydantic data models, you can dynamically extract metadata from user queries. Tool use is a powerful feature in Amazon Bedrock that allows models to access external tools or functions to enhance their response generation capabilities.

AWS 149

AWS Natural Language Processing AI AI 149

The Data Governance Institute

AUGUST 6, 2020

Examples of data-related change management are: Changes to allowable values for reference tables Changes to physical data stores that impact the ability to access or protect in-scope data Changes to data models Changes to data definitions Changes to data structures Changes to data movement Changes to the structure of metadata repositories Changes to (..)

Data Governance 52

Data Governance Data Modeling Data Models 52

ODSC - Open Data Science

NOVEMBER 18, 2024

You’ll start by demystifying what vector databases are, with clear definitions, simple explanations, and real-world examples of popular vector databases. You will also gain a practical understanding of how vector databases work, including the processes involved in storing, retrieving, and managing data in high-dimensional vector spaces.

Deep Learning 40

Deep Learning Deep Learning Database Data Science 40

Alation

MAY 31, 2023

They collaborate with IT professionals, business stakeholders, and data analysts to design effective data infrastructure aligned with the organization’s goals. Their broad range of responsibilities include: Design and implement data architecture. Maintain data models and documentation.

Data Governance 52

Data Governance Data Quality Data Silos Data Modeling 52

phData

MAY 25, 2023

While the loss of certain DAX functions is definitely a shortcoming that we hope Microsoft will address in the near future, the impact of these lost DAX functions is not necessarily as big as you would expect. Creating an efficient data model can be the difference between having good or bad performance, especially when using DirectQuery.

Power BI 84

Power BI Analytics Analytics Azure 84

Alation

NOVEMBER 4, 2021

However, I paused and asked myself: What is the value that customers of Alation actually got for non-compliance or data security use cases? The value of the data catalog depends on the audience. For data modelers, value arose from spending less time finding data and more time modeling data.

Data Governance 94

Data Governance Data Quality ML ML 94

Pickl AI

AUGUST 11, 2024

Consider factors such as data volume, query patterns, and hardware constraints. Document and Communicate Maintain thorough documentation of fact table designs, including definitions, calculations, and relationships. Establish data governance policies and processes to ensure consistency in definitions, calculations, and data sources.

Data Quality 52

Data Quality Data Warehouse Data Governance Business Intelligence 52

phData

APRIL 10, 2023

A truly governed self-service analytics model puts data modeling responsibilities in the hands of IT and report generation and analysis in the hands of business users who will actually be doing the analysis. Business users build reports on an IT-owned and IT-created data model that is focused on reporting solutions.

Analytics 52

Analytics Analytics Data Governance Data Modeling 52

Smart Data Collective

NOVEMBER 24, 2021

Based on this assumption, specialists relied on false predictive data models that could only reflect a simplified picture of the possible future. In this paradigm, any minor deviations in data (which, in fact, could predict something) could simply be ignored or perceived as exceptions. Hypothesis definition.

Analytics 130

Analytics Analytics Business Intelligence Business Intelligence 130

IBM Journey to AI blog

SEPTEMBER 13, 2024

Over the past few years, Salesforce has made heavy investments in Data Cloud. Data Cloud works to unlock trapped data by ingesting and unifying data from across the business. Data Cloud works to unlock trapped data by ingesting and unifying data from across the business.

AI 49

AI AI Natural Language Processing Tableau 49

Dataconomy

OCTOBER 17, 2023

Whether you are starting or revamping an existing data warehouse, designing a step-by-step guide can help cement your architecture design while avoiding common missteps. But It’s always better to call data warehouse experts before making a big decision.

Data Warehouse 103

Data Warehouse Big Data Big Data ETL 103

Mlearning.ai

FEBRUARY 13, 2023

where each word represents a key and each definition represents a value. These databases are designed for fast data retrieval and are ideal for applications that require quick data access and low latency, such as caching, session management, and real-time analytics. Cassandra and HBase are widely used with IoT 6.

Database 52

Database Data Modeling Data Models Big Data 52

Precisely

MARCH 7, 2024

Many people use the term to describe a data quality metric. Technical users, including database administrators, might tell you that data integrity concerns whether or not the data conforms to a pre-defined data model. To be sure, data quality is a critically important part of that picture.

Data Quality 52

Data Quality Database Administration Data Governance Analytics 52

Alation

DECEMBER 21, 2022

A data catalog communicates the organization’s data quality policies so people at all levels understand what is required for any data element to be mastered. Documenting rule definitions and corrective actions guide domain owners and stewards in addressing quality issues. MDM Model Objects.

Data Quality 52

Data Quality Data Warehouse Data Profiling Data Governance 52

Alation

APRIL 26, 2021

In your organization, are you ever confused by different definitions of business terms? If you’re thinking “business term definitions” are straightforward, think again. A business glossary helps an organization agree and align on internal definitions. List of business terms and their definitions.

Database 52

Database Machine Learning Machine Learning Database Administration 52

phData

JUNE 27, 2023

Data Lineage See exactly how your data is connected, edit on the fly, and follow your outputs as they flow downstream to your final dashboard and elements. Is dbt an Ideal Fit for YOUR Organization’s Data Stack? dbt’s addition of data freshness, quality, and cataloging is just another example of Sigma’s vision.

SQL 52

SQL Database Data Quality Data Warehouse 52

Smart Data Collective

NOVEMBER 18, 2020

This new approach has proven to be much more effective, so it is a skill set that people must master to become data scientists. Definition: Data Mining vs Data Science. Data mining is an automated data search based on the analysis of huge amounts of information. Data Mining Techniques and Data Visualization.

Data Mining 143

Data Mining Data Mining Data Mining Data Science 143

Tableau

SEPTEMBER 23, 2021

No single source of truth: There may be multiple versions or variations of similar data sets, but which is the trustworthy data set users should default to? Missing data definitions and formulas: People need to understand exactly what the data represents, in the context of the business, to use it effectively.

Data Governance 101

Data Governance Analytics Analytics Tableau 101

ODSC - Open Data Science

SEPTEMBER 27, 2023

Understanding Data Lakes A data lake is a centralized repository that stores structured, semi-structured, and unstructured data in its raw format. Unlike traditional data warehouses or relational databases, data lakes accept data from a variety of sources, without the need for prior data transformation or schema definition.

Data Lakes 98

Data Lakes Data Warehouse Database Big Data 98

Tableau

APRIL 18, 2022

Monitor data sources according to policies you customize to help users know if fresh, quality data is ready for use. Shine a light on who or what is using specific data to speed up collaboration or reduce disruption when changes happen. Data modeling. Data preparation.

Tableau 101

Tableau Data Quality Analytics Analytics 101

Tableau

SEPTEMBER 23, 2021

No single source of truth: There may be multiple versions or variations of similar data sets, but which is the trustworthy data set users should default to? Missing data definitions and formulas: People need to understand exactly what the data represents, in the context of the business, to use it effectively.

Data Governance 97

Data Governance Analytics Analytics Tableau 97

AWS Machine Learning Blog

FEBRUARY 27, 2024

The SageMaker project template includes seed code corresponding to each step of the build and deploy pipelines (we discuss these steps in more detail later in this post) as well as the pipeline definition—the recipe for how the steps should be run.

Machine Learning 129

Machine Learning Machine Learning ML ML 129

Tableau

APRIL 18, 2022

Monitor data sources according to policies you customize to help users know if fresh, quality data is ready for use. Shine a light on who or what is using specific data to speed up collaboration or reduce disruption when changes happen. Data modeling. Data preparation.

Tableau 98

Tableau Data Quality Analytics Analytics 98

IBM Journey to AI blog

SEPTEMBER 5, 2024

Start small by setting measurable goals and assigning ownership of data domains. Establishing standardized definitions and control measures builds a solid foundation that evolves as the framework matures. Define roles and responsibilities A successful data governance framework requires clearly defined roles and responsibilities.

Data Governance 75

Data Governance Data Quality DataOps Artificial Intelligence 75

Smart Data Collective

APRIL 14, 2021

Main features include the ability to access and operationalize data through the LookML library. It also allows you to create your data and creating consistent dataset definitions using LookML. Formerly known as Periscope, Sisense is a business intelligence tool ideal for cloud data teams.

Business Intelligence 135

Business Intelligence Business Intelligence Tableau SQL 135

Hacker News

FEBRUARY 11, 2024

The quality of the roles was about the same as I normally get (albeit at a much higher rate) – some I really liked, some were OK, and some were definitely not for me. Maybe I could see more profile viewers, I am not sure, but it is definitely not showing who all of them are. Applying to known companies.

Python 181

Python Database Data Modeling Data Models 181

Heartbeat

DECEMBER 7, 2023

Data should be designed to be easily accessed, discovered, and consumed by other teams or users without requiring significant support or intervention from the team that created it. Data should be created using standardized data models, definitions, and quality requirements.

Data Governance 52

Data Governance Deep Learning Deep Learning Artificial Intelligence 52

phData

AUGUST 18, 2023

Classify column is categorizing or organizing existing data into different buckets or definitions. Classify and Fill can be very similar if you are asking AI to categorize or organize your data. Whereas Fill will use populate types based on its own definitions. What is the Classify Column Option?

AI 52

AI AI Artificial Intelligence Artificial Intelligence 52

Iguazio

JANUARY 22, 2024

Definitions: Foundation Models, Gen AI, and LLMs Before diving into the practice of productizing LLMs, let’s review the basic definitions of GenAI elements: Foundation Models (FMs) - Large deep learning models that are pre-trained with attention mechanisms on massive datasets.

Data Pipeline 52

Data Pipeline ML ML Data Warehouse 52

Pickl AI

OCTOBER 16, 2024

Data Model : RDBMS relies on a structured schema with predefined relationships among tables, whereas NoSQL databases use flexible data models (e.g., key-value pairs, document-based) that accommodate unstructured data. Scalability : RDBMS typically scales vertically by adding more resources to a single server.

Database 52

Database SQL Big Data Big Data 52

phData

SEPTEMBER 13, 2023

Some third-party tools like Fivetran provide exceptional data modeling capabilities, which can be extremely helpful down the road. This is the easiest and fastest way to onboard your data into Snowflake. It also automates the creation of Data Definition Language (DDL) on the Snowflake side, saving setup time.

Tableau 52

Tableau Data Pipeline Data Silos Analytics 52

Tableau

OCTOBER 8, 2021

With its LookML modeling language, Looker provides a unique, modern approach to define governed and reusable data models to build a trusted foundation for analytics.

Tableau 137

Tableau Analytics Analytics Machine Learning 137