Package Audit

Package Audit

This skill audits a Python package across eight dimensions, identifies issues, and produces a prioritized, phased improvement plan saved to IMPROVEMENT.md. It does not implement any changes — the output is a plan that the user can review and execute in subsequent sessions.

Before you start

Read the best-practices reference file:

cat references/best-practices.md

Use the checklist in that file as your scanning rubric.

Critical rule

This skill only produces a plan. Do not implement any changes. Every finding goes into IMPROVEMENT.md as an unchecked action item. The user will execute those items later, possibly using other specialized skills.

Workflow

Step 1 — Classify the project

Before auditing, determine what kind of project this is. The project type shapes which audit dimensions apply and how the phased plan is organized.

Look at pyproject.toml, setup.py, the directory structure, and the README to classify into one of:

• Library — distributed on PyPI, has a public API, users import it. Example signals: [project] table, name on PyPI, from mypkg import ... in examples.
• CLI tool — primary interface is command-line. Example signals: [project.scripts] entry points, argparse/click/typer in the code, README shows shell commands.
• Web service — long-running server. Example signals: Flask/FastAPI/Django imports, uvicorn/gunicorn in dependencies, Dockerfile, API routes.
• Internal application — not distributed, used internally. Example signals: no PyPI publishing, no classifiers, may have deployment configs instead.
• Data/ML project — notebooks and pipelines. Example signals: .ipynb files, pandas/numpy/scikit-learn in deps, data/ or models/ directories.

Note the classification at the top of the audit. Skip or adjust dimensions that don't apply (e.g., a CLI tool doesn't need a "convenience API" phase; an internal application doesn't need PyPI classifiers).

Step 2 — Discovery

Systematically examine the project across these eight dimensions. Take notes as you go — you'll use them in Step 3.

1. Project structure

   • List top-level directories and their purposes
   • Identify the layout (src/ vs flat)
   • Check for __init__.py completeness
   • Note any files that shouldn't be tracked (.pyc, __pycache__, .DS_Store, *.egg-info)

2. Packaging & metadata

   • Is there a pyproject.toml? If only setup.py, that's a finding.
   • Does [project] include: name, version, description, readme, requires-python, license, authors, classifiers, dependencies, [project.urls]?
   • Build backend: Hatchling, Setuptools, Poetry, PDM, Flit?
   • Are optional dependencies grouped ([project.optional-dependencies])?
   • Are classifiers valid PyPI trove classifiers?

3. Public API

   • What does the package's __init__.py export?
   • Map every public class and function (names not starting with _)
   • Are there type hints on public signatures?
   • Is the API consistent in naming (snake_case functions, PascalCase classes)?

4. Test coverage

   • Is there a tests/ directory?
   • Which modules have no test file?
   • If possible, run pytest --cov=<package> --cov-report=term-missing
   • Note the overall coverage percentage and uncovered modules

5. Documentation

   • Does the README exist and cover: what it does, install, quick-start, link to full docs?
   • Are there docstrings on public functions/classes?
   • Is there a CHANGELOG.md?
   • Are there example files or usage notebooks?
   • Is there an API reference (Sphinx, MkDocs)?

6. CI/CD

   • Is there a .github/workflows/ directory (or GitLab/Bitbucket equivalent)?
   • Are tests run automatically on push/PR?
   • Is there linting in CI?
   • Is there a release workflow (PyPI publishing on tag)?
   • Is Dependabot/Renovate configured?

7. Security (run these checks if possible)

   • Run bandit -r <package>/ -q and note findings
   • Run pip-audit or check for known vulnerable dependencies
   • Grep for obvious issues: hardcoded passwords/tokens, eval(), exec(), shell=True in subprocess, pickle.load on untrusted data, unsafe XML parsing (xml.etree instead of defusedxml)
   • Check for missing input validation on public API methods

8. Code quality

   • Are type hints used consistently?
   • Is from __future__ import annotations used (for 3.9 support)?
   • Is logging used instead of print()?
   • Are custom exceptions defined, or does everything use built-ins?
   • Is there dead code (unused imports, unreachable branches)?
   • Naming consistency (PEP 8)

Step 3 — Write IMPROVEMENT.md

Save the audit and plan to IMPROVEMENT.md in the project root, following this exact template:

# Improvement Plan

**Project type:** <Library / CLI tool / Web service / Internal app / Data/ML>
**Audit date:** <YYYY-MM-DD>
**Audited against:** [Keep a Changelog 1.1.0, Semantic Versioning 2.0.0,
PEP 621, pytest, bandit]

## Audit Summary

### Strengths

- What the project does well. Keep doing these.
- Be specific — "uses modern pyproject.toml with complete metadata" not "good packaging".

### Critical Issues

Security vulnerabilities, broken functionality, missing input validation, hardcoded
credentials. Each issue must cite the file and line number where applicable.

### Code Quality Issues

Inconsistent patterns, missing type hints, poor error handling, dead code.

### Missing Infrastructure

No tests, no CI, no linting, no changelog, outdated packaging.

### Feature Gaps

<Only if Step 1 classified this as a Library. Skip for other project types.>
Features missing compared to similar packages users would expect.

### Developer Experience Gaps

Poor README, no examples, verbose boilerplate, missing convenience features.

---

## Improvement Phases

### Phase 1: Foundation & Safety

Security fixes, input validation, exception handling, packaging modernization,
CI pipeline, basic test coverage.

- [ ] **P0 · S** — Add input validation to `parser.parse_url()` (`src/pkg/parser.py:42`)
      to reject URLs with `file://` scheme
- [ ] **P0 · M** — Replace `xml.etree.ElementTree` with `defusedxml` for untrusted XML
      parsing (`src/pkg/xml_reader.py`)
- [ ] **P1 · M** — Modernize packaging: migrate `setup.py` to `pyproject.toml` with
      full `[project]` metadata
- [ ] **P1 · L** — Add GitHub Actions workflow: lint + test on push/PR

### Phase 2: Code Quality & Reliability

Type hints, consistent error handling, custom exceptions, logging, docstrings.

- [ ] **P1 · M** — Add type hints to all public functions in `src/pkg/api.py`
- [ ] **P1 · S** — Replace `print()` statements with `logging` (`src/pkg/cli.py:15-28`)
- [ ] **P2 · M** — Define custom exception hierarchy (`PkgError`, `PkgValidationError`,
      `PkgConnectionError`)

### Phase 3: Developer Experience

<Skip for internal applications. Adapt heavily for CLI tools and web services —
these care more about user-facing docs than API ergonomics.>

README rewrite, examples, convenience API, sample files.

- [ ] **P1 · L** — Rewrite README with install/quick-start/example sections
- [ ] **P2 · M** — Add `examples/` directory with runnable usage scripts

### Phase 4: Ecosystem & Growth

<Only for libraries with a public user base. Skip for internal apps and new
projects that haven't hit 1.0 yet.>

Optional integrations, serialization helpers, async support, caching.

- [ ] **P2 · L** — Add optional pandas integration via `[project.optional-dependencies]`
- [ ] **P2 · L** — Provide async client under `pkg.aio` module

---

## Progress

| Phase   | Status      | Items Done | Items Total |
| ------- | ----------- | ---------- | ----------- |
| Phase 1 | Not started | 0          | N           |
| Phase 2 | Not started | 0          | N           |
| Phase 3 | Not started | 0          | N           |
| Phase 4 | Not started | 0          | N           |