AI Agent Skills

Apache Spark - A unified analytics engine for large-scale data processing

Interactive and Reactive Data Science using Scala and Spark.

Jupyter magics and kernels for working with remote Spark clusters

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

Security library for Sparkjava: OpenID Connect, SAML2, CAS, OAuth, LDAP, JWT...

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

SQL data analysis & visualization projects using MySQL, PostgreSQL, SQLite, Tableau, Apache Spark and pySpark.

A COBOL parser and Mainframe/EBCDIC data source for Apache Spark

End-to-end Data Lakehouse project built on Databricks, following the Medallion Architecture (Bronze, Silver, Gold). Covers real-world data engineering and analytics work…

Big Data Modeling, MapReduce, Spark, PySpark @ Santa Clara University

1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.

A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.

Scriptis is for interactive data analysis with script development(SQL, Pyspark, HiveQL), task submission(Spark, Hive), UDF, function, resource management and intelligent…

Monitor the stability of a Pandas or Spark dataframe ⚙︎

Quick start: pip install jsoniq ⛈️ RumbleDB 2.1.0 "Cedrus Libani" 🌳 for Apache Spark | Run queries on your large-scale, messy datasets (JSON, text, CSV, Parquet, Delta.…

A simplified, lightweight ETL Framework based on Apache Spark