Hacker News

JULY 18, 2023

What if you could automatically shard your PostgreSQL database across any number of servers and get industry-leading performance at scale without any special data modelling steps? Schema-based sharding has almost no data modelling restrictions or special steps compared to unsharded PostgreSQL.

Database 123

Database SQL Data Modeling Data Models 123

Smart Data Collective

NOVEMBER 24, 2021

Based on this assumption, specialists relied on false predictive data models that could only reflect a simplified picture of the possible future. In this paradigm, any minor deviations in data (which, in fact, could predict something) could simply be ignored or perceived as exceptions. Hypothesis definition.

Analytics 133

Analytics Analytics Business Intelligence Business Intelligence 133

Tableau

OCTOBER 7, 2024

Tableau is a leader in the analytics market, known for helping organizations see and understand their data, but we recognize that gaps still exist: while many of our joint customers already benefit from dbt and trust the metrics that result from these workflows, they are often disconnected and obscured from Tableau’s analytics layer.

Tableau 137

Tableau Data Quality Analytics Analytics 137

Smart Data Collective

NOVEMBER 18, 2020

This new approach has proven to be much more effective, so it is a skill set that people must master to become data scientists. Definition: Data Mining vs Data Science. Data mining is an automated data search based on the analysis of huge amounts of information. Data Mining Techniques and Data Visualization.

Data Mining 144

Data Mining Data Mining Data Mining Data Science 144

Flipboard

NOVEMBER 20, 2024

By combining the capabilities of LLM function calling and Pydantic data models, you can dynamically extract metadata from user queries. Tool use is a powerful feature in Amazon Bedrock that allows models to access external tools or functions to enhance their response generation capabilities.

AWS 160

AWS Natural Language Processing Machine Learning Machine Learning 160

Smart Data Collective

APRIL 14, 2021

Main features include the ability to access and operationalize data through the LookML library. It also allows you to create your data and creating consistent dataset definitions using LookML. Formerly known as Periscope, Sisense is a business intelligence tool ideal for cloud data teams.

Business Intelligence 137

Business Intelligence Business Intelligence Tableau SQL 137

Alation

NOVEMBER 4, 2021

However, I paused and asked myself: What is the value that customers of Alation actually got for non-compliance or data security use cases? The value of the data catalog depends on the audience. For data modelers, value arose from spending less time finding data and more time modeling data.

Data Governance 94

Data Governance Data Quality ML ML 94

Hacker News

FEBRUARY 11, 2024

The quality of the roles was about the same as I normally get (albeit at a much higher rate) – some I really liked, some were OK, and some were definitely not for me. Maybe I could see more profile viewers, I am not sure, but it is definitely not showing who all of them are. Applying to known companies.

Python 181

Python Database Data Modeling Data Models 181

ODSC - Open Data Science

JUNE 12, 2023

This article is an excerpt from the book Expert Data Modeling with Power BI, Third Edition by Soheil Bakhshi, a completely updated and revised edition of the bestselling guide to Power BI and data modeling. No-code/low-code experience using a diagram view in the data preparation layer similar to Dataflows.

Power BI 52

Power BI Data Warehouse ETL Data Preparation 52

Tableau

OCTOBER 8, 2021

With its LookML modeling language, Looker provides a unique, modern approach to define governed and reusable data models to build a trusted foundation for analytics.

Tableau 137

Tableau Analytics Analytics Machine Learning 137

AWS Machine Learning Blog

FEBRUARY 27, 2024

The SageMaker project template includes seed code corresponding to each step of the build and deploy pipelines (we discuss these steps in more detail later in this post) as well as the pipeline definition—the recipe for how the steps should be run.

Machine Learning 139

Machine Learning Machine Learning ML ML 139

Tableau

SEPTEMBER 23, 2021

No single source of truth: There may be multiple versions or variations of similar data sets, but which is the trustworthy data set users should default to? Missing data definitions and formulas: People need to understand exactly what the data represents, in the context of the business, to use it effectively.

Data Governance 101

Data Governance Analytics Analytics Tableau 101

phData

MAY 25, 2023

While the loss of certain DAX functions is definitely a shortcoming that we hope Microsoft will address in the near future, the impact of these lost DAX functions is not necessarily as big as you would expect. Creating an efficient data model can be the difference between having good or bad performance, especially when using DirectQuery.

Power BI 84

Power BI Analytics Analytics Azure 84

The Data Governance Institute

AUGUST 6, 2020

Examples of data-related change management are: Changes to allowable values for reference tables Changes to physical data stores that impact the ability to access or protect in-scope data Changes to data models Changes to data definitions Changes to data structures Changes to data movement Changes to the structure of metadata repositories Changes to (..)

Data Governance 52

Data Governance Data Modeling Data Models 52

Tableau

SEPTEMBER 23, 2021

No single source of truth: There may be multiple versions or variations of similar data sets, but which is the trustworthy data set users should default to? Missing data definitions and formulas: People need to understand exactly what the data represents, in the context of the business, to use it effectively.

Data Governance 97

Data Governance Analytics Analytics Tableau 97

Tableau

APRIL 18, 2022

Monitor data sources according to policies you customize to help users know if fresh, quality data is ready for use. Shine a light on who or what is using specific data to speed up collaboration or reduce disruption when changes happen. Data modeling. Data preparation.

Tableau 101

Tableau Data Quality Analytics Analytics 101

Dataconomy

OCTOBER 17, 2023

Whether you are starting or revamping an existing data warehouse, designing a step-by-step guide can help cement your architecture design while avoiding common missteps. But It’s always better to call data warehouse experts before making a big decision.

Data Warehouse 103

Data Warehouse Big Data Big Data ETL 103

Tableau

APRIL 18, 2022

Monitor data sources according to policies you customize to help users know if fresh, quality data is ready for use. Shine a light on who or what is using specific data to speed up collaboration or reduce disruption when changes happen. Data modeling. Data preparation.

Tableau 98

Tableau Data Quality Analytics Analytics 98

Tableau

AUGUST 29, 2024

Spencer Czapiewski August 29, 2024 - 9:52pm Kirk Munroe Chief Analytics Officer & Founding Partner at Paint with Data Kirk Munroe, Chief Analytics Officer and Founding Partner at Paint with Data and Tableau DataDev Ambassador, explains the value of using relationships in your Tableau data models. over 4 years ago!),

Tableau 52

Tableau SQL Data Modeling Data Models 52

ODSC - Open Data Science

SEPTEMBER 27, 2023

Understanding Data Lakes A data lake is a centralized repository that stores structured, semi-structured, and unstructured data in its raw format. Unlike traditional data warehouses or relational databases, data lakes accept data from a variety of sources, without the need for prior data transformation or schema definition.

Data Lakes 98

Data Lakes Data Warehouse Database Big Data 98

AWS Machine Learning Blog

NOVEMBER 15, 2023

A full example of the models dictionary is available in the GitHub repository. For example, if your preprocessors dictionary contains 10 recipes and you have 5 model definitions in the models dictionary, the newly created pipelines dictionary contains 50 preprocessor-model pipelines that are evaluated during HPO.

Algorithm 124

Algorithm AWS ML ML 124

Alation

APRIL 26, 2021

In your organization, are you ever confused by different definitions of business terms? If you’re thinking “business term definitions” are straightforward, think again. A business glossary helps an organization agree and align on internal definitions. List of business terms and their definitions.

Database 52

Database Machine Learning Machine Learning Database Administration 52

Pickl AI

AUGUST 9, 2024

Hierarchies align data modelling with business processes, making it easier to analyse data in a context that reflects real-world operations. Designing Hierarchies Designing effective hierarchies requires careful consideration of the business requirements and the data model.

Data Warehouse 52

Data Warehouse Data Quality ETL Business Intelligence 52

Tableau

OCTOBER 8, 2021

With its LookML modeling language, Looker provides a unique, modern approach to define governed and reusable data models to build a trusted foundation for analytics.

Tableau 98

Tableau Analytics Analytics Machine Learning 98

Alation

MAY 31, 2023

They collaborate with IT professionals, business stakeholders, and data analysts to design effective data infrastructure aligned with the organization’s goals. Their broad range of responsibilities include: Design and implement data architecture. Maintain data models and documentation.

Data Governance 52

Data Governance Data Quality Data Silos Data Modeling 52

phData

DECEMBER 10, 2024

As with most modeling challenges, the best solution is to work upstream, beginning with the Warehouse configuration, Data Modeling approaches, and then identifying possible Sigma performance levels. Security in Sigma Computing Security was definitely our most discussed topic, and for good reason.

Clustering 52

Clustering Analytics Analytics Data Visualization 52

O'Reilly Media

OCTOBER 19, 2021

While there isn’t an authoritative definition for the term, it shares its ethos with its predecessor, the DevOps movement in software engineering: by adopting well-defined processes, modern tooling, and automated workflows, we can streamline the process of moving from development to robust production deployments. Why did something break?

ML 140

ML ML Data Scientist AWS 140

ODSC - Open Data Science

FEBRUARY 14, 2023

While this technology is definitely entertaining, it’s not quite clear yet how it can effectively be applied to the needs of the typical enterprise. The database would need to offer a flexible and expressive data model, allowing developers to easily store and query complex data structures.

Database 124

Database AI AI Natural Language Processing 124

phData

APRIL 10, 2023

A truly governed self-service analytics model puts data modeling responsibilities in the hands of IT and report generation and analysis in the hands of business users who will actually be doing the analysis. Business users build reports on an IT-owned and IT-created data model that is focused on reporting solutions.

Analytics 52

Analytics Analytics Data Governance Data Modeling 52

Alation

DECEMBER 21, 2022

A data catalog communicates the organization’s data quality policies so people at all levels understand what is required for any data element to be mastered. Documenting rule definitions and corrective actions guide domain owners and stewards in addressing quality issues. MDM Model Objects.

Data Quality 52

Data Quality Data Warehouse Data Profiling Data Governance 52

Mlearning.ai

FEBRUARY 13, 2023

where each word represents a key and each definition represents a value. These databases are designed for fast data retrieval and are ideal for applications that require quick data access and low latency, such as caching, session management, and real-time analytics. Cassandra and HBase are widely used with IoT 6.

Database 52

Database Data Modeling Data Models Big Data 52

Precisely

MARCH 7, 2024

Many people use the term to describe a data quality metric. Technical users, including database administrators, might tell you that data integrity concerns whether or not the data conforms to a pre-defined data model. To be sure, data quality is a critically important part of that picture.

Data Quality 52

Data Quality Database Administration Data Governance Analytics 52

IBM Journey to AI blog

MARCH 29, 2024

GraphQL is defined by API schema written in the GraphQL schema definition language. Each schema specifies the types of data the user can query or modify, and the relationships between the types. Resolvers also provide data format specifications and enable the system to stitch together data from various sources.

Data Profiling 94

Data Profiling Database Data Modeling Data Models 94

Alation

FEBRUARY 13, 2020

This achievement is a testament not only to our legacy of helping to create the data catalog category but also to our continued innovation in improving the effectiveness of self-service analytics. A broader definition of Business Intelligence. Enabling workers to find the right data is crucial to promoting self-service analytics.

Business Intelligence 52

Business Intelligence Business Intelligence Analytics Analytics 52

phData

JUNE 27, 2023

Data Lineage See exactly how your data is connected, edit on the fly, and follow your outputs as they flow downstream to your final dashboard and elements. Is dbt an Ideal Fit for YOUR Organization’s Data Stack? dbt’s addition of data freshness, quality, and cataloging is just another example of Sigma’s vision.

SQL 52

SQL Database Data Quality Data Warehouse 52

Pickl AI

AUGUST 11, 2024

Consider factors such as data volume, query patterns, and hardware constraints. Document and Communicate Maintain thorough documentation of fact table designs, including definitions, calculations, and relationships. Establish data governance policies and processes to ensure consistency in definitions, calculations, and data sources.

Data Quality 52

Data Quality Data Warehouse Data Governance Business Intelligence 52

IBM Journey to AI blog

SEPTEMBER 5, 2024

Start small by setting measurable goals and assigning ownership of data domains. Establishing standardized definitions and control measures builds a solid foundation that evolves as the framework matures. Define roles and responsibilities A successful data governance framework requires clearly defined roles and responsibilities.

Data Governance 75

Data Governance Data Quality DataOps Artificial Intelligence 75