New: 98,000+ skills indexed from GitHub repos with 100+ stars · every skill safety-checkedBrowse top skillsTop skills
Esc to closeBrowse all skills
Submit a skillBrowse skills
Submit a skillBrowse skills

Agent skill

Chunking

by VectorSpaceLab in VectorSpaceLab/AREX-Skill

Configure and validate Unstructured post-partition chunking for RAG, embedding, and downstream processing.

Apache-2.0Auto-check passedAI & LLM Engineering

Install Chunking

skills CLI
$ npx skills add VectorSpaceLab/AREX-Skill --skill chunking -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install VectorSpaceLab/AREX-Skill chunking --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/VectorSpaceLab/AREX-Skill.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/repositories/repo-skills/unstructured/sub-skills/chunking .claude/skills/chunking && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
chunking
GitHub stars
328
Token cost
~974 tokens
SKILL.md length
327 words
Files
5 (incl. scripts, references)
Skills in repo
157
Repo updated
First seen
Licence
Apache-2.0

At a glance

Configure and validate Unstructured post-partition chunking for RAG, embedding, and downstream processing.

  • Works in 4 steps: Choose the entry point → Pick one sizing mode → Decide table handling before overlap → …
  • An agent needs chunkelements()
  • SKILL.md covers Start Here, Key References, Routing Boundaries and Quick Patterns, plus 1 more section
  • Runs Python scripts from its folder

What it does

Chunking is an agent skill from VectorSpaceLab/AREX-Skill. Configure and validate Unstructured post-partition chunking for RAG, embedding, and downstream processing. Use when an agent needs chunkelements(), chunkbytitle(), partition-integrated chunking kwargs, character/token limits, overlap, table chunking behavior, or origelements metadata decisions.

Its SKILL.md is about 970 tokens, which your agent loads only when the skill is triggered. The skill folder holds 6 other files, including scripts and reference files (for example `references/api-reference.md`, `references/troubleshooting.md` and `references/workflows.md`).

It sits in AI & LLM Engineering, covering Retrieval-augmented generation and Embeddings. The repository describes itself as: A Skill Library for Automated Machine Learning. The licence is Apache-2.0.

When your agent uses it

  • An agent needs chunkelements()
  • Partition-integrated chunking kwargs
  • Character/token limits
  • Table chunking behavior

Example prompts

  • “/chunking”

Requirements

  • Python 3

Workflow steps

4 steps, taken from the first numbered list in SKILL.md.

  1. Choose the entry point
  2. Pick one sizing mode
  3. Decide table handling before overlap
  4. Decide metadata weight

What it can do on your machine

Read from SKILL.md and the folder at commit ac3fe1a. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Ships 1 file in scripts/ (Python), which the agent can run.

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Chunking loads about 974 tokens when it runs, and up to ~5.1k if it reads all its reference files. Until then it costs about 77 tokens; SKILL.md has 327 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~77
When it runs · the whole SKILL.md, loaded when a task matches
~974
With references · SKILL.md plus every file in references/, read only if the agent opens them
~5.1k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.

SKILL.md

The full file from VectorSpaceLab/AREX-Skill at commit ac3fe1a, republished under its Apache-2.0 licence (© VectorSpaceLab). 327 words, ~974 tokens.

Download SKILL.mdSave it as .claude/skills/chunking/SKILL.md (or your agent's skills folder). This skill also uses 4 other files; get the full folder from GitHub.
name
chunking
description
Configure and validate Unstructured post-partition chunking for RAG, embedding, and downstream processing. Use when an agent needs chunk_elements(), chunk_by_title(), partition-integrated chunking kwargs, character/token limits, overlap, table chunking behavior, or orig_elements metadata decisions.
disable-model-invocation
true
metadata.disco-role
operating
license
Apache 2.0

Chunking

Use this sub-skill after a document has been partitioned into Unstructured elements, or when a partition call should return chunks directly via chunking_strategy.

Start Here

  1. Choose the entry point:
    • Use unstructured.chunking.basic.chunk_elements(elements, ...) when you already have elements and want sequential size-based chunks.
    • Use unstructured.chunking.title.chunk_by_title(elements, ...) when Title elements should start sections and optional page boundaries should split sections.
    • Use partition(..., chunking_strategy="basic" | "by_title", ...) when the partitioner supports integrated chunking and you want one call.
  2. Pick one sizing mode:
    • Character mode: max_characters is the hard maximum, new_after_n_chars is the soft preferred boundary.
    • Token mode: max_tokens is the hard maximum, new_after_n_tokens is the soft preferred boundary, and tokenizer is required.
  3. Decide table handling before overlap:
    • Default isolate_table=True keeps Table and TableChunk separate from surrounding text.
    • Default repeat_table_headers=True repeats detected headers on continuation table chunks.
    • Use skip_table_chunking=True only when oversized tables must pass through unchanged.
  4. Decide metadata weight:
    • Default include_orig_elements=True preserves original elements in metadata.orig_elements.
    • Set include_orig_elements=False for lighter JSON payloads when original metadata is not needed.

Key References

  • references/api-reference.md: public functions, parameters, defaults, output element types, and validation rules.
  • references/workflows.md: RAG, table-heavy, integrated partitioning, token-based, and validation workflows.
  • references/troubleshooting.md: common ValueErrors, token extra issues, overlap pollution, table edge cases, and metadata size trade-offs.
  • scripts/chunk_elements_preview.py: preview chunking behavior from element JSON and summarize chunk types, lengths, table metadata, and orig_elements counts.

Routing Boundaries

  • Route element creation, file parsing, strategies like OCR/table extraction, and partition signatures to the partitioning sub-skill.
  • Route JSON schema interpretation, elements_to_json(), elements_from_json(), and serialized metadata payload design to the elements-and-metadata sub-skill.
  • Keep this sub-skill focused on chunking already-created elements or partition-integrated chunking arguments.

Quick Patterns

python
from unstructured.chunking.title import chunk_by_title

chunks = chunk_by_title(
    elements,
    max_characters=1200,
    new_after_n_chars=900,
    overlap=80,
    overlap_all=False,
    include_orig_elements=False,
)
python
from unstructured.partition.auto import partition

chunks = partition(
    filename="report.pdf",
    chunking_strategy="by_title",
    max_characters=1500,
    new_after_n_chars=1000,
    combine_text_under_n_chars=200,
    multipage_sections=False,
)

Review Checklist

  • Confirm the request uses one sizing mode, not both character and token limits.
  • Explain hard maximum versus soft maximum when recommending values.
  • State whether tables remain isolated, split into TableChunk, or pass through unchanged.
  • State whether metadata.orig_elements is retained and how that affects serialized size.
  • Warn before using overlap_all=True, because it can duplicate text across semantic chunk boundaries.

© VectorSpaceLab, Apache-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 4 other files (scripts, references) in skills/repositories/repo-skills/unstructured/sub-skills/chunking of VectorSpaceLab/AREX-Skill.

  • SKILL.md
  • references/api-reference.md
  • references/troubleshooting.md
  • references/workflows.md
  • scripts/chunk_elements_preview.py

Open the folder on GitHubat commit ac3fe1a

Compare with similar skills

Chunking next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Chunking compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Chunking this skillVectorSpaceLab/AREX-Skill328—~974Automated safety check: PassApache-2.0
Chroma Vector DatabaseOrchestra-Research/AI-Research-SKILLs13k8 repos~2.3kAutomated safety check: PassMIT
Ms Agent Framework RAGshuyu-labs/WebCode278—~1.1kAutomated safety check: PassCustom licence
Pgvector Semantic Searchtimescale/pg-aiguide1.9k1 repos~3.8kAutomated safety check: PassApache-2.0
Evaluate RAGai-evals-course/evals-skills1.5k—~1.9kAutomated safety check: PassApache-2.0
RAG ArchitectJeffallan/claude-skills12k1 repos~2kAutomated safety check: PassMIT

Similar skills

  • Chroma Vector Database

    Orchestra-Research/AI-Research-SKILLs

    Shows how to store documents and embeddings in Chroma, query them by similarity with metadata filters, and persist them to disk for RAG and semantic search projects.

    13k GitHub starsUsed in 8 repos~2.3k tokens
    AI & LLM EngineeringAuto-check passed

  • Ms Agent Framework RAG

    shuyu-labs/WebCode

    Comprehensive guide for building Agentic RAG systems using Microsoft Agent Framework in C.

    278 GitHub stars~1.1k tokensUpdated 3 mo ago
    AI & LLM EngineeringAuto-check passed

  • Pgvector Semantic Search

    timescale/pg-aiguide

    A skill your agent uses for setting up vector similarity search with pgvector for AI/ML embeddings, RAG applications, or semantic search.

    1.9k GitHub starsUsed in 1 repo~3.8k tokens
    AI & LLM EngineeringAuto-check passed

  • Evaluate RAG

    ai-evals-course/evals-skills

    Guides evaluation of a RAG system by diagnosing failures in traces, building a retrieval test set and scoring retrieval and generation separately.

    1.5k GitHub stars~1.9k tokensUpdated 14 days ago
    AI & LLM EngineeringAuto-check passed

  • RAG Architect

    Jeffallan/claude-skills

    Designs retrieval-augmented generation systems: document chunking, embeddings, vector store setup, hybrid search, reranking and retrieval evaluation, with checks at each step.

    12k GitHub starsUsed in 1 repo~2k tokens
    AI & LLM EngineeringAuto-check passed

  • Memory Upgrade

    profbernardoj/everclaw-community-branches

    Diagnose and fix broken memory search in OpenClaw. An agent skill from profbernardoj/everclaw-community-branches.

    112 GitHub stars~574 tokensUpdated 1 mo ago
    AI & LLM EngineeringAuto-check passed

More from VectorSpaceLab/AREX-Skill

All 157 skills in this repo

  • Agent Lightning

    VectorSpaceLab/AREX-Skill

    Use this repo skill for Agent Lightning package tasks: authoring trainable agents, tracing rewards and spans, running LightningStore/Trainer loops, using agl CLI services, choosing examples, and…

    328 GitHub stars~1.4k tokensUpdated 1 mo ago
    Auto-check passed

  • Agent Tools

    VectorSpaceLab/AREX-Skill

    A skill your agent uses when configuring LiteLLM for MCP tools, A2A agents, Claude Code/Cursor agent gateway traffic, MCP auth/OAuth, tool permissions, semantic filtering, or agent-specific proxy…

    328 GitHub stars~1.2k tokensUpdated 1 mo ago
    Auto-check passed

  • Agents And Awel

    VectorSpaceLab/AREX-Skill

    Build and debug DB-GPT agents, tools, skills, teams, and AWEL workflows, including deterministic local DAG runs and HTTP-trigger topology without assuming an LLM, credential, or external service.

    328 GitHub stars~2.3k tokensUpdated 1 mo ago
    Auto-check passed

  • Agents And Middleware

    VectorSpaceLab/AREX-Skill

    Work on the actively maintained LangChain v1 agent package: initchatmodel, createagent, structured output, tools, middleware, embeddings initialization, provider routing, and agent runtime…

    328 GitHub stars~1.2k tokensUpdated 1 mo ago
    Auto-check passed

  • Agents Workflows

    VectorSpaceLab/AREX-Skill

    A skill your agent uses for giskard.agents async chat workflows, tools, prompt templates, structured outputs, retries, rate limiting, embeddings, and optional LiteLLM backend.

    328 GitHub stars~500 tokensUpdated 1 mo ago
    Auto-check passed

  • Alphafold3

    VectorSpaceLab/AREX-Skill

    A skill your agent uses for AlphaFold 3 input preparation, prediction command planning, output interpretation, and Python API inspection.

    328 GitHub stars~1.2k tokensUpdated 1 mo ago
    Auto-check passed

Categories

Questions about Chunking

What does Chunking do?

Configure and validate Unstructured post-partition chunking for RAG, embedding, and downstream processing. Chunking is an agent skill from VectorSpaceLab/AREX-Skill. Configure and validate Unstructured post-partition chunking for RAG, embedding, and downstream processing.

When should I use Chunking?

Chunking fits situations like: an agent needs chunkelements(); partition-integrated chunking kwargs; character/token limits; table chunking behavior.

How do I install Chunking in Claude Code?

Run `npx skills add VectorSpaceLab/AREX-Skill --skill chunking -a claude-code`. Or copy the skill folder (skills/repositories/repo-skills/unstructured/sub-skills/chunking in VectorSpaceLab/AREX-Skill) into .claude/skills/chunking in your project. Claude Code loads it when a task matches its description.

How do I install Chunking in Codex?

Run `npx skills add VectorSpaceLab/AREX-Skill --skill chunking -a codex`. Or copy the skill folder (skills/repositories/repo-skills/unstructured/sub-skills/chunking in VectorSpaceLab/AREX-Skill) into .agents/skills/chunking in your project. Codex loads it when a task matches its description.

Can I use Chunking in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add VectorSpaceLab/AREX-Skill --skill chunking -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/chunking, .gemini/skills/chunking, .github/skills/chunking and .opencode/skills/chunking in your project.

What does Chunking need to run?

Going by SKILL.md and its folder, Chunking needs Python for the scripts in its folder. Our summary lists: Python 3.

Does Chunking access the network?

SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.

Is Chunking safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. The check reads SKILL.md only: the scripts in the folder are not scanned, so read them before running anything.

What licence does Chunking use?

Chunking is published under the Apache-2.0 licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Chunking use?

About 974 tokens (SKILL.md is roughly 3.9k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 4.1k tokens, read only when the agent opens those files.

What are the alternatives to Chunking?

Skills that share tags, products or a category with Chunking: Chroma Vector Database (Orchestra-Research/AI-Research-SKILLs, 13k stars), Ms Agent Framework RAG (shuyu-labs/WebCode, 278 stars), Pgvector Semantic Search (timescale/pg-aiguide, 1.9k stars) and Evaluate RAG (ai-evals-course/evals-skills, 1.5k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Chunking?

VectorSpaceLab (a GitHub organization) maintains it in VectorSpaceLab/AREX-Skill, which has 328 GitHub stars. The repository holds 157 skills in this directory. The repository was last updated on September 3, 2026.

Source: VectorSpaceLab/AREX-Skill on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.