ODSC - Open Data Science

FEBRUARY 24, 2023

There are many well-known libraries and platforms for data analysis such as Pandas and Tableau, in addition to analytical databases like ClickHouse, MariaDB, Apache Druid, Apache Pinot, Google BigQuery, Amazon RedShift, etc. VisiData works with CSV files, Excel spreadsheets, SQL databases, and many other data sources.

Exploratory Data Analysis 126

Exploratory Data Analysis Data Visualization Data Analysis Data Analysis 126

Hacker News

SEPTEMBER 15, 2024

We have people running databases, web servers, key-value stores, lots of applications. We were the first company to really push on running [electronic design automation (EDA)] in the cloud. We changed the model from “I’ve got 80 servers and this is what I use for EDA” to “Today, I have 80 servers. It’s done very well.

AWS 181

AWS EDA Machine Learning Machine Learning 181

Becoming Human

MAY 15, 2023

One is a scripting language such as Python, and the other is a Query language like SQL (Structured Query Language) for SQL Databases. There is one Query language known as SQL (Structured Query Language), which works for a type of database. SQL Databases are MySQL , PostgreSQL , MariaDB , etc. Why do we need databases?

Data Science 75

Data Science Machine Learning Machine Learning Database 75

Pickl AI

DECEMBER 3, 2024

This can be done from various sources such as CSV files, Excel files, or databases. Step 3: Exploratory Data Analysis (EDA) Exploratory Data Analysis (EDA) is a critical step that involves examining the dataset to understand its structure, patterns, and anomalies. During EDA, you can: Check for missing values.

Python 52

Python ML ML Exploratory Data Analysis 52

Pickl AI

MAY 30, 2024

Also Read: Explore data effortlessly with Python Libraries for (Partial) EDA: Unleashing the Power of Data Exploration. These include databases, APIs, web scraping, and public datasets. By checking patterns, distributions, and anomalies, EDA unveils insights crucial for informed decision-making.

Data Analysis 52

Data Analysis Data Analysis Data Science Exploratory Data Analysis 52

Pickl AI

APRIL 3, 2025

Key Processes and Techniques in Data Analysis Data Collection: Gathering raw data from various sources (databases, APIs, surveys, sensors, etc.). Exploratory Data Analysis (EDA): Using statistical summaries and initial visualisations (yes, visualisation plays a role within analysis!) EDA: Calculate overall churn rate.

Data Analysis 52

Data Analysis Data Analysis Data Visualization EDA 52

Heartbeat

FEBRUARY 13, 2023

It is designed to make it easy to track and monitor experiments and conduct exploratory data analysis (EDA) using popular Python visualization frameworks. The SQL database used by Kangas Datagrid enables users to keep many datasets. Kangas’s database-based storage system is more scalable and can better handle large amounts of data.

Data Science 94

Data Science Python Deep Learning Deep Learning 94

IBM Journey to AI blog

OCTOBER 15, 2024

The long-standing partnership between IBM and Intel has led to significant advancements in database performance over the past 25 years. They include built-in accelerators like Intel® Advanced Matrix Extensions (Intel® AMX) that offer a much-needed alternative in the market for customers with growing AI workload demand.

AWS 45

AWS Data Lakes EDA Database 45

Pickl AI

JULY 25, 2023

They create data pipelines, ETL processes, and databases to facilitate smooth data flow and storage. Their primary responsibilities include: Data Collection and Preparation Data Scientists start by gathering relevant data from various sources, including databases, APIs, and online platforms. ETL Tools: Apache NiFi, Talend, etc.

Data Engineer 52

Data Engineer Data Engineering Data Engineering Data Engineering 52

Mlearning.ai

MARCH 15, 2023

Data Extraction, Preprocessing & EDA & Machine Learning Model development Data collection : Automatically download the stock historical prices data in CSV format and save it to the AWS S3 bucket. Data Extraction, Preprocessing & EDA : Extract & Pre-process the data using Python and perform basic Exploratory Data Analysis.

Python 52

Python AWS Exploratory Data Analysis Machine Learning 52

Dataconomy

JULY 28, 2023

Exploratory data analysis (EDA) Before preprocessing data, conducting exploratory data analysis is crucial to understand the dataset’s characteristics, identify patterns, detect outliers, and validate missing values. EDA provides insights into the data distribution and informs the selection of appropriate preprocessing techniques.

Power BI 91

Power BI Data Preparation Exploratory Data Analysis Machine Learning 91

Pickl AI

NOVEMBER 4, 2024

Perform exploratory Data Analysis (EDA) using Pandas and visualise your findings with Matplotlib or Seaborn. Additionally, learn about data storage options like Hadoop and NoSQL databases to handle large datasets. This project helps you understand data cleaning and the importance of insights derived from data.

Data Science 52

Data Science Python Machine Learning Machine Learning 52

AWS Machine Learning Blog

AUGUST 12, 2024

These could include exports from customer relationship management (CRM), configuration management database (CMDB), and electronic health record (EHR) systems. On this page, you will see your Data Catalog database and tables listed, named patient_data and splunk_ops_data.

ML 120

ML ML AWS AI 120

Mlearning.ai

AUGUST 10, 2023

Extract Data We will use Google Trends as a database to extract data, it is a public web-based tool that allows users to explore the popularity of search queries on Google. We have to create a database for the project: Figure 8: Creating a Dabase in pgAdmin4 Next, we have to write database’s name and save?. Windows NT 10.0;

Business Intelligence 52

Business Intelligence Business Intelligence ETL Power BI 52

Pickl AI

JULY 12, 2024

Public Datasets: Utilising publicly available datasets from repositories like Kaggle or government databases. Exploratory Data Analysis (EDA) EDA is a crucial preliminary step in understanding the characteristics of the dataset. Web Scraping : Extracting data from websites and online sources.

Artificial Intelligence 52

Artificial Intelligence Artificial Intelligence Python Natural Language Processing 52

Kaggle

JULY 29, 2020

An ETL process was built to take the CSV, find the corresponding text articles and load the data into a SQLite database. The early days of the effort were spent on EDA and exchanging ideas with other members of the community. cord19q has the logic for ETL, building the embeddings index and running the custom BERT QA model.

ETL 71

ETL Data Scientist Machine Learning Machine Learning 71

Pickl AI

DECEMBER 16, 2024

databases, APIs, CSV files). Exploratory Data Analysis (EDA): Conduct EDA to identify trends, seasonal patterns, and correlations within the dataset. Step 2: Data Gathering Collect relevant historical data that will be used for forecasting. Making Data Stationary: Many forecasting models assume stationarity.

AI 52

AI AI Machine Learning Machine Learning 52

Pickl AI

SEPTEMBER 12, 2024

Key Components of Data Science Data Science consists of several key components that work together to extract meaningful insights from data: Data Collection: This involves gathering relevant data from various sources, such as databases, APIs, and web scraping. Data Cleaning: Raw data often contains errors, inconsistencies, and missing values.

Data Analyst 52

Data Analyst Data Science Machine Learning Machine Learning 52

The MLOps Blog

DECEMBER 28, 2022

EDA, as it is popularly called, is the pivotal phase of the project where discoveries are made. Approvals from stakeholders ML projects are inherently iterative by nature. Data scientists frame the business problem and the objective into a statistical solution and start with the very first step of data exploration.

ML 78

ML ML Data Scientist Machine Learning 78

Mlearning.ai

JUNE 29, 2023

In databases, there are specialized tools that people have built to make range queries very very efficiently on data points like this. This is just a sample code implementation without any EDA & feature importance and also data engineering. We can use brute force but for the matter of efficiency we use Kd-Tree like optimizations.

Clustering 52

Clustering Algorithm Data Mining Data Mining 52

Ocean Protocol

JANUARY 10, 2024

To execute, the Ocean Protocol Data Science team and the challenge participants used the provided data made available by the European Commission EDGAR — Emissions Database for Global Atmospheric Research. Two Data Sets were used to weigh carbon emission rates under two different metrics: Co2 (Carbon Dioxide) and GHG (Green House Gases).

Data Science 52

Data Science Exploratory Data Analysis Support Vector Machines Data Analysis 52

Mlearning.ai

JUNE 16, 2023

Top 7 diagrams as code tools for software architecture Copilot [link] Good for: Create a list of all states in US Repetitive task like insert all database fields (no need to type manually of everything) [link] Don’t let copilot generate function (no comment-driven development), but let copilot complete some part Is GitHub Copilot Worth Being Paid?

AI 52

AI AI Data Analysis Data Analysis 52

The MLOps Blog

MAY 31, 2023

So, a better database architecture would be to maintain multiple tables where one of the tables maintains the past 3 months history with session-level details, whereas other tables may contain weekly aggregated click, ATC, and order data.

ML 59

ML ML Algorithm Machine Learning 59

Heartbeat

MARCH 22, 2023

Before building our model, we will also see how we can visualize this data with Kangas as part of exploratory data analysis (EDA). Sometimes, a database will fill with empty strings when no data is found somewhere. In this article, let’s dive deep into the Natural Language Toolkit (NLTK) data processing concepts for NLP data.

Natural Language Processing 52

Natural Language Processing Deep Learning Deep Learning Machine Learning 52

Dataconomy

AUGUST 5, 2024

Guide users on how to clean and preprocess data, handle missing values, normalize datasets, and provide insights on exploratory data analysis (EDA) and inferential statistics. Recommend appropriate visualization tools and chart types, assist in selecting machine learning algorithms, explain database management concepts, and more.”

AI 103

AI AI Exploratory Data Analysis Data Analysis 103

DrivenData Labs

DECEMBER 7, 2023

This particular competition gave me the opportunity to use/evaluate recent trends/SOTA in Large Language models, semantic search, knowledge graphs, and vector databases, which was different from my computer-vision background. : I enjoy participating in machine learning/data-science challenges and have been doing it for a while.

Natural Language Processing 130

Natural Language Processing Clustering Data Science Data Analysis 130

Mlearning.ai

JULY 29, 2023

It is a crucial component of the Exploration Data Analysis (EDA) stage, which is typically the first and most critical step in any data project. This structured format allows for easy analysis, manipulation, and visualization of the data using tools like spreadsheets or database systems. Statistical relationship 1. Scatter plot Fig.

Python 52

Python Data Visualization Data Science Exploratory Data Analysis 52