AI Agent Skills

Code repository for the "PySpark in Action" book

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks…

Open Source LeetCode for PySpark, Spark, Pandas and DBT/Snowflake

Notes on Apache Spark (pyspark)

SQL data analysis & visualization projects using MySQL, PostgreSQL, SQLite, Tableau, Apache Spark and pySpark.

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

Execute arbitrary Python or PySpark code on Fabric Spark compute without creating a notebook artifact; ephemeral Livy sessions with full Delta table access. Automaticall…

Scriptis is for interactive data analysis with script development(SQL, Pyspark, HiveQL), task submission(Spark, Hive), UDF, function, resource management and intelligent…

End-to-end Data Lakehouse project built on Databricks, following the Medallion Architecture (Bronze, Silver, Gold). Covers real-world data engineering and analytics work…

Big Data Modeling, MapReduce, Spark, PySpark @ Santa Clara University

Diagnose slow, expensive, or regressed Apache Spark and PySpark applications by comparing runtime evidence against a healthy run. Use for long stages, stragglers, skew,…