.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

Apache Spark - A unified analytics engine for large-scale data processing

Drop-in replacement for Apache Spark UI

Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow

1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.

Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

Pre-submission AI review stress-test for research papers. A Claude Code skill: review, verdict, revise, verify.

TransmogrifAI (pronounced trăns-mŏgˈrə-fī) is an AutoML library for building modular, reusable, strongly typed machine learning workflows on Apache Spark with minimal hand-tuning

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.

Apache Spark docker image