Efficient Data Processing Spark alternatives for AI agents.

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

CLI task management & automation tool

A list of useful resources to learn Data Engineering from scratch

Smarter data pipelines for audio.

Privacy and Security focused Segment-alternative, in Golang and React

🔥🔥🔥 Open source Reverse ETL - alternative to hightouch and census.

Streaming reactive and dataflow graphs in Python

Example end to end data engineering project.

Practical Data Engineering: A Hands-On Real-Estate Project Guide

Memphis.dev is a highly scalable and effortless data streaming platform

Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more

End-to-end Data Lakehouse project built on Databricks, following the Medallion Architecture (Bronze, Silver, Gold). Covers real-world data engineering and analytics workflows using Spark, PySpark, SQL, Delta Lake, and Unity Catalog. Designed for learning, portfolio building, and job interviews.

scikit-learn: machine learning in Python

Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.

An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.