Data & Analytics · Apache Spark

Migrate a legacy AWS Glue development endpoint to a Glue interactive session, following the official AWS migration checklist.

aws-samples/aws-glue-samples

Guides writing and tuning Apache Spark jobs: DataFrame and RDD code, Spark SQL, partitioning, caching, shuffle tuning and structured streaming.

Jeffallan/claude-skills

Create a new PySpark data source implementation. An agent skill from allisonwang-db/pyspark-data-sources.

allisonwang-db/pyspark-data-sources

A skill your agent uses when the user is writing datafusion-python (Apache DataFusion Python bindings) DataFrame or SQL code.

apache/datafusion-python

Speed up slow Apache Spark jobs by tuning partitions, shuffles, data skew, caching and executor memory, with do and don't rules for PySpark code.

wshobson/agents

Audit and improve datafusion-python functions to accept native Python types (int, float, str, bool) instead of requiring explicit lit() or col() wrapping.

apache/datafusion-python

Build scalable data pipelines, modern data warehouses, and real-time streaming architectures.

davila7/claude-code-templates

Analyze DBSQL queries, including SQL embedded in notebooks (spark.sql(...), %sql cells), for anti-patterns, lint issues, and performance problems, using Databricks-specific dialect and platform…

AltimateAI/data-engineering-skills

Data pipeline expert for ETL, Apache Spark, Airflow, dbt, and data quality

RightNow-AI/openfang

Data engineering patterns for ETL pipelines, data warehousing, Apache Spark, and data quality validation

rohitg00/awesome-claude-code-toolkit

Discovers requirements and designs an end-to-end governed agentic analytics solution using Knowledge Catalog and Managed Service for Apache Spark (Lightning Engine).

google/skills

Execute arbitrary Python or PySpark code on Fabric Spark compute without creating a notebook artifact; ephemeral Livy sessions with full Delta table access.

data-goblin/power-bi-agentic-development

A skill your agent uses when reading from or writing to Neo4j with Apache Spark or Databricks using the Neo4j Connector for Apache Spark 6.0 (org.neo4j.connectors:spark) or 5.x…

neo4j-contrib/neo4j-skills

Build custom Python data sources for Apache Spark using the PySpark DataSource API — batch and streaming readers/writers for external systems.

databricks/databricks-agent-skills

Expert guidance for working with the Apache Spark Catalyst query optimisation framework.

aehrc/pathling

Transform pyspark transformer operations. An agent skill from jeremylongshore/tons-of-skills-marketplace.

jeremylongshore/tons-of-skills-marketplace

Optimize spark sql optimizer operations. An agent skill from jeremylongshore/tons-of-skills-marketplace.

jeremylongshore/tons-of-skills-marketplace

Upgrade Apache Spark applications between major versions (2.x→3.x, 3.x→4.x).

OpenHands/extensions

Assists with benchmarking and profiling the performance of an Apache Spark UDF on the GPU.

Kilo-Org/kilo-marketplace

Best practices for building performant, testable PySpark ETL pipelines with Spark SQL and Apache Iceberg.

Mindrally/skills