AI Agent Skills

Apache Spark - A unified analytics engine for large-scale data processing

Interactive and Reactive Data Science using Scala and Spark.

Official Dockerfile for Apache Spark

Jupyter magics and kernels for working with remote Spark clusters

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

Security library for Sparkjava: OpenID Connect, SAML2, CAS, OAuth, LDAP, JWT...

Ultra-Lightweight AI-Powered Big Data Center | 至轻云-超轻量级智能化大数据中心/数据中台

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

SQL data analysis & visualization projects using MySQL, PostgreSQL, SQLite, Tableau, Apache Spark and pySpark.

End-to-end Data Lakehouse project built on Databricks, following the Medallion Architecture (Bronze, Silver, Gold). Covers real-world data engineering and analytics work…

Big Data Modeling, MapReduce, Spark, PySpark @ Santa Clara University

1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.

A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.

Scriptis is for interactive data analysis with script development(SQL, Pyspark, HiveQL), task submission(Spark, Hive), UDF, function, resource management and intelligent…

Monitor the stability of a Pandas or Spark dataframe ⚙︎

Quick start: pip install jsoniq ⛈️ RumbleDB 2.1.0 "Cedrus Libani" 🌳 for Apache Spark | Run queries on your large-scale, messy datasets (JSON, text, CSV, Parquet, Delta.…