Audit Skill Md

<!---
  Licensed to the Apache Software Foundation (ASF) under one
  or more contributor license agreements.  See the NOTICE file
  distributed with this work for additional information
  regarding copyright ownership.  The ASF licenses this file
  to you under the Apache License, Version 2.0 (the
  "License"); you may not use this file except in compliance
  with the License.  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing,
  software distributed under the License is distributed on an
  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
  KIND, either express or implied.  See the License for the
  specific language governing permissions and limitations
  under the License.
-->

Audit skills/datafusion_python/SKILL.md

You are auditing the user-facing skill at skills/datafusion_python/SKILL.md against the current state of the Python API. The skill is the source of truth for how AI coding assistants are taught to write datafusion-python code, so it must match what the project actually ships. This skill identifies gaps caused by upstream syncs, refactors, or renames, and (if asked) applies the edits directly to SKILL.md.

The skill is most usefully run after the check-upstream step of an upstream sync (see dev/release/upstream-sync.md) — once any new APIs are exposed, this skill makes sure they get documented.

What the skill covers

The user-facing SKILL.md documents these public surfaces. This list is not exhaustive — if a new top-level area is added (e.g., a new Catalog API exposed at the package root), include it.

Scope argument

The user may specify a scope via $ARGUMENTS to limit the audit. If no scope is given or all is specified, audit every area.

Inputs to read

Before producing the report:

1. skills/datafusion_python/SKILL.md — the document being audited.
2. The relevant Python module(s) for the chosen scope. Public surface is the __all__ list (where defined) plus class and def symbols not prefixed with _.
3. Cargo.toml (root) for the current datafusion-python version — read the version field under [workspace.package] (format NN.0.0). The major version always matches the upstream datafusion crate, so a single datafusion-python version expresses both. python/datafusion/__init__.py's __version__ is the same value exposed at runtime.
4. Recent commits touching the relevant module(s) for context on what changed since the last sync:

   git log --oneline -- python/datafusion/dataframe.py | head -20

What to look for

Walk through each scoped area and flag four kinds of issues.

1. New APIs not mentioned

For each public symbol in the module's __all__ (or each public class method), check whether it appears anywhere in SKILL.md. A symbol is "covered" if it shows up in:

• A code block (the strongest signal — it's demonstrated).
• The "Available Functions (Categorized)" list.
• The SQL-to-DataFrame Reference table.

Decide whether each missing symbol deserves an entry. Not every public symbol belongs in SKILL.md — the skill is curated for the patterns users hit daily, not exhaustive API reference. Use these heuristics:

• Add it if it replaces or supersedes something already in the skill (e.g., a new operation that is the idiomatic alternative to a documented workaround).
• Add it if it fits a category already present (a new aggregate function goes in the aggregate list; a new join type goes in the joining section).
• Add it if it changes how a documented pattern should be written.
• Skip it if it is genuinely niche / advanced / experimental.
• Skip it if it is internal plumbing exposed for FFI but not user-facing.

When you flag a missing symbol, include a one-line proposed insertion point (which section / which table row) so a reviewer can decide quickly.

2. Stale mentions

For each function name, method name, or import shown in SKILL.md, verify it still exists in the current API:

• Function names mentioned in prose or in the categorized list should appear in python/datafusion/functions/__init__.py's __all__.
• Spark function names mentioned in the "Spark-Compatible Functions" subsection should appear in python/datafusion/functions/spark.py's __all__. Also confirm the divergent-semantics table still matches the current spark vs. main signatures.
• Method calls in code blocks should resolve against the current class.
• Imports (from datafusion import ...) should succeed against the current __init__.py.

A quick way to check imports without running them:

python -c "from datafusion import SessionContext, col, lit; from datafusion import functions as F; print('ok')"

For each stale mention, propose either:

• a rename to the current name, or
• removal if the API is gone with no replacement.

3. Examples that drifted from idiomatic style

The skill teaches a Pythonic style: prefer plain strings to col(...) when a column reference is all you need; prefer raw Python values to lit(...) where auto-wrapping applies. Recent refactors (see the make-pythonic skill) keep moving more functions toward accepting native types.

For each code example in SKILL.md, check:

• Does it use lit(value) where a raw value would work? Comparison RHS, arithmetic with a column, etc. all auto-wrap. (Reserve lit() for the cases listed in pitfall #2.)
• Does it use col("name") where a plain string would work? select(...), aggregate([keys], ...), sort(...), sort_by(...) all accept plain name strings.
• Do functions.py calls match the current pythonic signature for that function? If make-pythonic recently changed a signature (e.g., repeat(string, n: Expr | int)), the example should pass 3 rather than lit(3).
• Does any example use a deprecated or removed parameter name?

For drift, propose the updated snippet. If the change is purely stylistic and the older form still works, mark the suggestion as non-blocking.