Hacker News

JULY 4, 2023

The second section will delve more deeply into the various approaches that can be used to handle recursive schema definitions. The following Go Arrow schema definition provides an example of such a schema, instrumented with a collection of annotations. The depth of this definition cannot be predetermined.

Data Models 123

Data Models Data Modeling Data Observability 123

Towards AI

JULY 17, 2024

Back during my time as CTO of Locally, I was introduced to GraphDB as a mechanism for defining and discovering relationships between data, even using it as a simple definition store, it allows for depth and breadth-first searches to help discover relationships that might not have been explicitly defined.

Database 111

Database Data Models Data Modeling SQL 111

Tableau

AUGUST 29, 2024

Spencer Czapiewski August 29, 2024 - 9:52pm Kirk Munroe Chief Analytics Officer & Founding Partner at Paint with Data Kirk Munroe, Chief Analytics Officer and Founding Partner at Paint with Data and Tableau DataDev Ambassador, explains the value of using relationships in your Tableau data models. over 4 years ago!),

Tableau 96

Tableau SQL Data Models Data Modeling 96

Hacker News

JULY 18, 2023

What if you could automatically shard your PostgreSQL database across any number of servers and get industry-leading performance at scale without any special data modelling steps? Schema-based sharding has almost no data modelling restrictions or special steps compared to unsharded PostgreSQL.

Database 123

Database SQL Data Models Data Modeling 123

Smart Data Collective

NOVEMBER 24, 2021

Based on this assumption, specialists relied on false predictive data models that could only reflect a simplified picture of the possible future. In this paradigm, any minor deviations in data (which, in fact, could predict something) could simply be ignored or perceived as exceptions. Hypothesis definition.

Analytics 134

Analytics Analytics Business Intelligence Business Intelligence 134

Tableau

OCTOBER 7, 2024

Tableau is a leader in the analytics market, known for helping organizations see and understand their data, but we recognize that gaps still exist: while many of our joint customers already benefit from dbt and trust the metrics that result from these workflows, they are often disconnected and obscured from Tableau’s analytics layer.

Tableau 138

Tableau Data Quality Analytics Analytics 138

Smart Data Collective

NOVEMBER 18, 2020

This new approach has proven to be much more effective, so it is a skill set that people must master to become data scientists. Definition: Data Mining vs Data Science. Data mining is an automated data search based on the analysis of huge amounts of information. Data Mining Techniques and Data Visualization.

Data Mining 144

Data Mining Data Mining Data Mining Data Science 144

Smart Data Collective

APRIL 14, 2021

Main features include the ability to access and operationalize data through the LookML library. It also allows you to create your data and creating consistent dataset definitions using LookML. Formerly known as Periscope, Sisense is a business intelligence tool ideal for cloud data teams.

Business Intelligence 138

Business Intelligence Business Intelligence Tableau SQL 138

Alation

NOVEMBER 4, 2021

However, I paused and asked myself: What is the value that customers of Alation actually got for non-compliance or data security use cases? The value of the data catalog depends on the audience. For data modelers, value arose from spending less time finding data and more time modeling data.

Data Governance 94

Data Governance Data Quality ML ML 94

IBM Journey to AI blog

OCTOBER 30, 2023

A successful public health response to a future pandemic will rely on collecting and managing critical data, investing in smart, capable and flexible data modernization systems, and preparing people with the proper knowledge and skills. Lesson 1: Use a data model built for public health.

Data Models 78

Data Models Data Modeling Data Silos Machine Learning 78

ODSC - Open Data Science

JUNE 12, 2023

This article is an excerpt from the book Expert Data Modeling with Power BI, Third Edition by Soheil Bakhshi, a completely updated and revised edition of the bestselling guide to Power BI and data modeling. No-code/low-code experience using a diagram view in the data preparation layer similar to Dataflows.

Power BI 52

Power BI Data Warehouse ETL Data Preparation 52

Tableau

OCTOBER 8, 2021

With its LookML modeling language, Looker provides a unique, modern approach to define governed and reusable data models to build a trusted foundation for analytics.

Tableau 138

Tableau Analytics Analytics Machine Learning 138

Tableau

SEPTEMBER 23, 2021

No single source of truth: There may be multiple versions or variations of similar data sets, but which is the trustworthy data set users should default to? Missing data definitions and formulas: People need to understand exactly what the data represents, in the context of the business, to use it effectively.

Data Governance 102

Data Governance Analytics Analytics Tableau 102

phData

MAY 25, 2023

While the loss of certain DAX functions is definitely a shortcoming that we hope Microsoft will address in the near future, the impact of these lost DAX functions is not necessarily as big as you would expect. Creating an efficient data model can be the difference between having good or bad performance, especially when using DirectQuery.

Power BI 84

Power BI Analytics Analytics Azure 84

Hacker News

FEBRUARY 11, 2024

The quality of the roles was about the same as I normally get (albeit at a much higher rate) – some I really liked, some were OK, and some were definitely not for me. Maybe I could see more profile viewers, I am not sure, but it is definitely not showing who all of them are. Applying to known companies.

Python 181

Python Database Data Models Data Modeling 181

The Data Governance Institute

AUGUST 6, 2020

Examples of data-related change management are: Changes to allowable values for reference tables Changes to physical data stores that impact the ability to access or protect in-scope data Changes to data models Changes to data definitions Changes to data structures Changes to data movement Changes to the structure of metadata repositories Changes to (..)

Data Governance 52

Data Governance Data Models Data Modeling 52

Tableau

SEPTEMBER 23, 2021

No single source of truth: There may be multiple versions or variations of similar data sets, but which is the trustworthy data set users should default to? Missing data definitions and formulas: People need to understand exactly what the data represents, in the context of the business, to use it effectively.

Data Governance 97

Data Governance Analytics Analytics Tableau 97

Tableau

APRIL 18, 2022

Monitor data sources according to policies you customize to help users know if fresh, quality data is ready for use. Shine a light on who or what is using specific data to speed up collaboration or reduce disruption when changes happen. Data modeling. Data preparation.

Tableau 102

Tableau Data Quality Analytics Analytics 102

Tableau

APRIL 18, 2022

Monitor data sources according to policies you customize to help users know if fresh, quality data is ready for use. Shine a light on who or what is using specific data to speed up collaboration or reduce disruption when changes happen. Data modeling. Data preparation.

Tableau 98

Tableau Data Quality Analytics Analytics 98

Dataconomy

OCTOBER 17, 2023

Whether you are starting or revamping an existing data warehouse, designing a step-by-step guide can help cement your architecture design while avoiding common missteps. But It’s always better to call data warehouse experts before making a big decision.

Data Warehouse 103

Data Warehouse Big Data Big Data ETL 103

AWS Machine Learning Blog

FEBRUARY 27, 2024

The SageMaker project template includes seed code corresponding to each step of the build and deploy pipelines (we discuss these steps in more detail later in this post) as well as the pipeline definition—the recipe for how the steps should be run.

Machine Learning 131

Machine Learning Machine Learning ML ML 131

ODSC - Open Data Science

SEPTEMBER 27, 2023

Understanding Data Lakes A data lake is a centralized repository that stores structured, semi-structured, and unstructured data in its raw format. Unlike traditional data warehouses or relational databases, data lakes accept data from a variety of sources, without the need for prior data transformation or schema definition.

Data Lakes 98

Data Lakes Data Warehouse Database Big Data 98

Alation

APRIL 26, 2021

In your organization, are you ever confused by different definitions of business terms? If you’re thinking “business term definitions” are straightforward, think again. A business glossary helps an organization agree and align on internal definitions. List of business terms and their definitions.

Database 52

Database Machine Learning Machine Learning Data Wrangling 52

Pickl AI

AUGUST 9, 2024

Hierarchies align data modelling with business processes, making it easier to analyse data in a context that reflects real-world operations. Designing Hierarchies Designing effective hierarchies requires careful consideration of the business requirements and the data model.

Data Warehouse 52

Data Warehouse Data Quality ETL Business Intelligence 52

O'Reilly Media

OCTOBER 19, 2021

While there isn’t an authoritative definition for the term, it shares its ethos with its predecessor, the DevOps movement in software engineering: by adopting well-defined processes, modern tooling, and automated workflows, we can streamline the process of moving from development to robust production deployments. Why did something break?

ML 145

ML ML Data Scientist AWS 145

Alation

MAY 31, 2023

They collaborate with IT professionals, business stakeholders, and data analysts to design effective data infrastructure aligned with the organization’s goals. Their broad range of responsibilities include: Design and implement data architecture. Maintain data models and documentation.

Data Governance 52

Data Governance Data Quality Data Silos Data Models 52

Tableau

OCTOBER 8, 2021

With its LookML modeling language, Looker provides a unique, modern approach to define governed and reusable data models to build a trusted foundation for analytics.

Tableau 98

Tableau Analytics Analytics Machine Learning 98

phData

APRIL 10, 2023

A truly governed self-service analytics model puts data modeling responsibilities in the hands of IT and report generation and analysis in the hands of business users who will actually be doing the analysis. Business users build reports on an IT-owned and IT-created data model that is focused on reporting solutions.

Analytics 52

Analytics Analytics Data Governance Data Models 52

ODSC - Open Data Science

FEBRUARY 14, 2023

While this technology is definitely entertaining, it’s not quite clear yet how it can effectively be applied to the needs of the typical enterprise. The database would need to offer a flexible and expressive data model, allowing developers to easily store and query complex data structures.

Database 124

Database AI AI Natural Language Processing 124

Alation

DECEMBER 21, 2022

A data catalog communicates the organization’s data quality policies so people at all levels understand what is required for any data element to be mastered. Documenting rule definitions and corrective actions guide domain owners and stewards in addressing quality issues. MDM Model Objects.

Data Quality 52

Data Quality Data Warehouse Data Profiling Data Governance 52

Mlearning.ai

FEBRUARY 13, 2023

where each word represents a key and each definition represents a value. These databases are designed for fast data retrieval and are ideal for applications that require quick data access and low latency, such as caching, session management, and real-time analytics. Cassandra and HBase are widely used with IoT 6.

Database 52

Database Data Models Data Modeling Big Data 52

AWS Machine Learning Blog

NOVEMBER 15, 2023

A full example of the models dictionary is available in the GitHub repository. For example, if your preprocessors dictionary contains 10 recipes and you have 5 model definitions in the models dictionary, the newly created pipelines dictionary contains 50 preprocessor-model pipelines that are evaluated during HPO.

Algorithm 106

Algorithm AWS ML ML 106

phData

DECEMBER 10, 2024

As with most modeling challenges, the best solution is to work upstream, beginning with the Warehouse configuration, Data Modeling approaches, and then identifying possible Sigma performance levels. Security in Sigma Computing Security was definitely our most discussed topic, and for good reason.

Clustering 52

Clustering Analytics Analytics Data Visualization 52

Alation

FEBRUARY 13, 2020

This achievement is a testament not only to our legacy of helping to create the data catalog category but also to our continued innovation in improving the effectiveness of self-service analytics. A broader definition of Business Intelligence. Enabling workers to find the right data is crucial to promoting self-service analytics.

Business Intelligence 52

Business Intelligence Business Intelligence Analytics Analytics 52

phData

JUNE 27, 2023

Data Lineage See exactly how your data is connected, edit on the fly, and follow your outputs as they flow downstream to your final dashboard and elements. Is dbt an Ideal Fit for YOUR Organization’s Data Stack? dbt’s addition of data freshness, quality, and cataloging is just another example of Sigma’s vision.

SQL 52

SQL Database Data Quality Data Warehouse 52

Pickl AI

AUGUST 11, 2024

Consider factors such as data volume, query patterns, and hardware constraints. Document and Communicate Maintain thorough documentation of fact table designs, including definitions, calculations, and relationships. Establish data governance policies and processes to ensure consistency in definitions, calculations, and data sources.

Data Quality 52

Data Quality Data Warehouse Data Governance Analytics 52

Alation

FEBRUARY 20, 2020

By changing the cost structure of collecting data, it increased the volume of data stored in every organization. Additionally, Hadoop removed the requirement to model or structure data when writing to a physical store. You did not have to understand or prepare the data to get it into Hadoop, so people rarely did.

Data Lakes 52

Data Lakes Hadoop Tableau Big Data 52