Data & Analytics · Python · By wshobson

Sets up data quality checks with Great Expectations, dbt tests and data contracts, with checkpoints and pass-fail reports for pipelines.

wshobson/agents

Patterns for writing production-ready Apache Airflow DAGs: task dependencies, custom operators and sensors, local testing, and rules for what to avoid.

wshobson/agents

Speed up slow Apache Spark jobs by tuning partitions, shuffles, data skew, caching and executor memory, with do and don't rules for PySpark code.

wshobson/agents

Python error handling patterns including input validation, exception hierarchies, and partial failure handling.

wshobson/agents