Spark SQL is Apache Spark's module for working with structured data based on DataFrames.

The machine learning library for Apache Spark, providing scalable algorithms and tools for ML pipelines.

Spark Project Hive

Kafka 0.10+ Source For Structured Streaming

spark-testing-base

Spark Project REPL

Spark Project Hive Thrift Server

Elasticsearch Spark (for Spark 2.X)

Livy API

Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.

The Almaren Framework provides a simplified consistent minimalistic layer over Apache Spark. While still allowing you to take advantage of native Apache Spark features.