article thumbnail

Streamlining ETL data processing at Talent.com with Amazon SageMaker

AWS Machine Learning Blog

Established in 2011, Talent.com aggregates paid job listings from their clients and public job listings, and has created a unified, easily searchable platform. Our pipeline belongs to the general ETL (extract, transform, and load) process family that combines data from multiple sources into a large, central repository.

ETL 113
article thumbnail

Big Data – Lambda or Kappa Architecture?

Data Science Blog

Lambda – Architecture Introduced in 2011 during the peak of Big Data’s prominence, the Lambda architecture remains a significant presence in the field. It offers the advantage of having a single ETL platform to develop and maintain.

Big Data 130
article thumbnail

Improving air quality with generative AI

AWS Machine Learning Blog

In this solution, we leverage the reasoning and coding abilities of LLMs for creating reusable Extract, Transform, Load (ETL), which transforms sensor data files that do not conform to a universal standard to be stored together for downstream calibration and analysis. She holds 30+ patents and has co-authored 100+ journal/conference papers.

AWS 130