Open source EL(T) with Meltano and Singer with Douwe Maan (Video)

In this edition of the Data Nerd Herd, Gitlab's Duawe Mann present Meltano. Meltano is an open source, self-hosted, CLI-first ELT platform that brings DevOps best practices such as version control, code review, and continuous integration and deployment (CI/CD) to a space historically dominated by proprietary, hosted (SaaS), UI-first solutions, that end up being poorly customizable black boxes in otherwise transparent data stacks.

At GitLab, where Meltano was founded in 2018, they think that the power of data integration should be available to all, and are on a mission to build a truly competitive alternative to existing proprietary EL(T) solutions, in terms of ease of use, reliability, and quantity and quality of supported data sources.

Meltano leverages the existing ecosystem of open source Singer taps and targets for extraction and loading, and supports dbt transformations and Airflow orchestration through a thin plugin-based integration layer. This makes Meltano the easiest way to get started with open source ELT, without imposing arbitrary limitations on those advanced users who want to take a look behind the curtain and interact with the plugins directly.

More on Meltano

About the presenter: Douwe Maan leads the Meltano project at GitLab, which he joined in 2015 as its fifth software engineer, when GitLab's community of open source contributors still greatly outnumbered the in-house engineering team. He became GitLab's first Engineering Lead, and ran the team responsible for all version control and code review functionality until 2019, when he moved to Meltano to attempt to replicate GitLab's success in commoditizing DevOps tooling in the data integration space.