Llamaindex
Orchestra-Research/AI-Research-SKILLs
Data framework for building LLM applications with RAG. An agent skill from Orchestra-Research/AI-Research-SKILLs.
Load local files or raw text into LlamaIndex Documents, split them into metadata-aware nodes, and run ingestion pipelines with cache/docstore controls.
Install Ingestion And Loading
$ npx skills add VectorSpaceLab/AREX-Skill --skill ingestion-and-loading -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install VectorSpaceLab/AREX-Skill ingestion-and-loading --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/VectorSpaceLab/AREX-Skill.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/repositories/repo-skills/llama-index/sub-skills/ingestion-and-loading .claude/skills/ingestion-and-loading && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "ingestion-and-loading" agent skill from https://github.com/VectorSpaceLab/AREX-Skill/tree/main/skills/repositories/repo-skills/llama-index/sub-skills/ingestion-and-loading into .claude/skills/ingestion-and-loading/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "ingestion-and-loading", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/VectorSpaceLab/AREX-Skill/tree/main/skills/repositories/repo-skills/llama-index/sub-skills/ingestion-and-loadingType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add VectorSpaceLab/AREX-Skill --skill ingestion-and-loading -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install VectorSpaceLab/AREX-Skill ingestion-and-loading --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/VectorSpaceLab/AREX-Skill.git skills-src && mkdir -p .agents/skills && cp -r skills-src/skills/repositories/repo-skills/llama-index/sub-skills/ingestion-and-loading .agents/skills/ingestion-and-loading && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "ingestion-and-loading" agent skill from https://github.com/VectorSpaceLab/AREX-Skill/tree/main/skills/repositories/repo-skills/llama-index/sub-skills/ingestion-and-loading into .agents/skills/ingestion-and-loading/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "ingestion-and-loading", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add VectorSpaceLab/AREX-Skill --skill ingestion-and-loading -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install VectorSpaceLab/AREX-Skill ingestion-and-loading --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/VectorSpaceLab/AREX-Skill.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/skills/repositories/repo-skills/llama-index/sub-skills/ingestion-and-loading .cursor/skills/ingestion-and-loading && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "ingestion-and-loading" agent skill from https://github.com/VectorSpaceLab/AREX-Skill/tree/main/skills/repositories/repo-skills/llama-index/sub-skills/ingestion-and-loading into .cursor/skills/ingestion-and-loading/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "ingestion-and-loading", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/VectorSpaceLab/AREX-Skill.git --path skills/repositories/repo-skills/llama-index/sub-skills/ingestion-and-loading--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add VectorSpaceLab/AREX-Skill --skill ingestion-and-loading -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install VectorSpaceLab/AREX-Skill ingestion-and-loading --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/VectorSpaceLab/AREX-Skill.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/skills/repositories/repo-skills/llama-index/sub-skills/ingestion-and-loading .gemini/skills/ingestion-and-loading && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "ingestion-and-loading" agent skill from https://github.com/VectorSpaceLab/AREX-Skill/tree/main/skills/repositories/repo-skills/llama-index/sub-skills/ingestion-and-loading into .gemini/skills/ingestion-and-loading/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "ingestion-and-loading", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install VectorSpaceLab/AREX-Skill ingestion-and-loadingInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add VectorSpaceLab/AREX-Skill --skill ingestion-and-loading -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/VectorSpaceLab/AREX-Skill.git skills-src && mkdir -p .github/skills && cp -r skills-src/skills/repositories/repo-skills/llama-index/sub-skills/ingestion-and-loading .github/skills/ingestion-and-loading && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "ingestion-and-loading" agent skill from https://github.com/VectorSpaceLab/AREX-Skill/tree/main/skills/repositories/repo-skills/llama-index/sub-skills/ingestion-and-loading into .github/skills/ingestion-and-loading/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "ingestion-and-loading", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add VectorSpaceLab/AREX-Skill --skill ingestion-and-loading -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install VectorSpaceLab/AREX-Skill ingestion-and-loading --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/VectorSpaceLab/AREX-Skill.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/skills/repositories/repo-skills/llama-index/sub-skills/ingestion-and-loading .opencode/skills/ingestion-and-loading && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "ingestion-and-loading" agent skill from https://github.com/VectorSpaceLab/AREX-Skill/tree/main/skills/repositories/repo-skills/llama-index/sub-skills/ingestion-and-loading into .opencode/skills/ingestion-and-loading/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "ingestion-and-loading", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
Facts
- Skill name
ingestion-and-loading- GitHub stars
- 331
- Token cost
- ~931 tokens
- SKILL.md length
- 234 words
- Files
- 5 (incl. scripts, references)
- Skills in repo
- 157
- Repo updated
- First seen
- Licence
- MIT
At a glance
Load local files or raw text into LlamaIndex Documents, split them into metadata-aware nodes, and run ingestion pipelines with cache/docstore controls.
- SimpleDirectoryReader configuration
- SKILL.md covers Route Here For, Do Not Use For, Fast Start and Required References, plus 1 more section
- Runs Python scripts from its folder; calls python
- Document/TextNode metadata
What it does
Ingestion And Loading is an agent skill from VectorSpaceLab/AREX-Skill. Load local files or raw text into LlamaIndex Documents, split them into metadata-aware nodes, and run ingestion pipelines with cache/docstore controls. Use for SimpleDirectoryReader configuration, Document/TextNode metadata, SentenceSplitter/TokenTextSplitter/MarkdownNodeParser/HierarchicalNodeParser, IngestionPipeline, and ingestion troubleshooting.
Its SKILL.md is about 930 tokens, which your agent loads only when the skill is triggered. The skill folder holds 6 other files, including scripts and reference files (for example `references/api-reference.md`, `references/troubleshooting.md` and `references/workflows.md`).
It sits in AI & LLM Engineering. It works with LlamaIndex. The repository describes itself as: A Skill Library for Automated Machine Learning. The licence is MIT.
When your agent uses it
- SimpleDirectoryReader configuration
- Document/TextNode metadata
- SentenceSplitter/TokenTextSplitter/MarkdownNodeParser/HierarchicalNodeParser
- IngestionPipeline
Example prompts
- “/ingestion-and-loading”
Requirements
- Python 3
What it can do on your machine
Read from SKILL.md and the folder at commit ac3fe1a. It shows what the files ask for, not the result of running them.
Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
Ships 1 file in scripts/ (Python), which the agent can run.
Shell commands in SKILL.md call:
python
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Context cost
Ingestion And Loading loads about 931 tokens when it runs, and up to ~4.8k if it reads all its reference files. Until then it costs about 94 tokens; SKILL.md has 234 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
Safety
Auto-check passed
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.
SKILL.md
The full file from VectorSpaceLab/AREX-Skill at commit ac3fe1a, republished under its MIT licence (© VectorSpaceLab). 234 words, ~931 tokens.
Download SKILL.mdSave it as
.claude/skills/ingestion-and-loading/SKILL.md (or your agent's skills folder). This skill also uses 4 other files; get the full folder from GitHub.- name
- ingestion-and-loading
- description
- Load local files or raw text into LlamaIndex Documents, split them into metadata-aware nodes, and run ingestion pipelines with cache/docstore controls. Use for SimpleDirectoryReader configuration, Document/TextNode metadata, SentenceSplitter/TokenTextSplitter/MarkdownNodeParser/HierarchicalNodeParser, IngestionPipeline, and ingestion troubleshooting.
- disable-model-invocation
- true
- metadata.disco-role
- operating
- license
- MIT
Ingestion and Loading
Use this sub-skill when the user needs to get data into llama_index.core before indexing or querying.
Route Here For
- Loading files with
SimpleDirectoryReader, includinginput_dir,input_files,exclude,exclude_hidden,exclude_empty,recursive,required_exts,filename_as_id,file_extractor,file_metadata, andraise_on_error. - Creating or inspecting
Document,TextNode, node IDs, metadata,excluded_embed_metadata_keys, andexcluded_llm_metadata_keys. - Splitting documents into nodes with
SentenceSplitter,TokenTextSplitter,MarkdownNodeParser, orHierarchicalNodeParser. - Running
IngestionPipelinewith transformations,IngestionCache, optional docstores,DocstoreStrategy,persist(),load(),run(),arun(), andnum_workers. - Diagnosing no files loaded, skipped hidden/empty/excluded files, encoding failures, optional parser dependencies, oversized metadata, bad chunk overlap, stale caches/docstores, duplicate IDs, and async/parallel ingestion caveats.
Do Not Use For
- Choosing index classes, retrievers, query engines, or response synthesizers; use
../indexing-and-querying/SKILL.md. - Selecting external vector stores, embedding providers, file parser integrations, or optional provider packages; use
../integrations-and-storage/SKILL.md. - Building agents, tools, memory, or workflows; use
../agents-and-workflows/SKILL.md.
Fast Start
from llama_index.core import Document, SimpleDirectoryReader
from llama_index.core.ingestion import IngestionPipeline
from llama_index.core.node_parser import SentenceSplitter
reader = SimpleDirectoryReader(
input_dir="data",
recursive=True,
required_exts=[".md", ".pdf"],
exclude_hidden=True,
filename_as_id=True,
)
documents = reader.load_data()
splitter = SentenceSplitter(chunk_size=1024, chunk_overlap=200)
nodes = splitter.get_nodes_from_documents(documents)
pipeline = IngestionPipeline(transformations=[splitter])
nodes = pipeline.run(documents=documents, show_progress=True)For raw text, bypass readers:
from llama_index.core import Document
doc = Document(text="Release notes...", metadata={"source": "manual"}, id_="release-notes")Required References
- Read
references/workflows.mdfor loading recipes, metadata-aware chunking, and cache/docstore refresh patterns. - Read
references/api-reference.mdfor signatures, defaults, imports, and parser selection rules. - Read
references/troubleshooting.mdwhen ingestion loads nothing, parsing fails, chunks look wrong, or cache/docstore results are stale. - Run
scripts/validate_ingestion_inputs.py --helpbefore proposing aSimpleDirectoryReadersetup for unfamiliar local file trees.
Bundled Helper
Use the safe validator to inspect planned local inputs and print likely reader arguments without importing LlamaIndex or reading file contents:
python sub-skills/ingestion-and-loading/scripts/validate_ingestion_inputs.py data --required-ext .md --required-ext .pdf --recursive --filename-as-idIt reports matched, hidden, empty, excluded, and extension-filtered files plus a copyable SimpleDirectoryReader(...) argument sketch.
© VectorSpaceLab, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
Files
SKILL.md and 4 other files (scripts, references) in skills/repositories/repo-skills/llama-index/sub-skills/ingestion-and-loading of VectorSpaceLab/AREX-Skill.
- SKILL.md
- references/api-reference.md
- references/troubleshooting.md
- references/workflows.md
- scripts/validate_ingestion_inputs.py
Open the folder on GitHubat commit ac3fe1a
Compare with similar skills
Ingestion And Loading next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Ingestion And Loading this skillVectorSpaceLab/AREX-Skill | 331 | — | ~931 | Automated safety check: Pass | MIT | |
| LlamaindexOrchestra-Research/AI-Research-SKILLs | 13k | 2 repos | ~3.7k | Automated safety check: Pass | MIT | |
| Phoenix LLM ObservabilityOrchestra-Research/AI-Research-SKILLs | 13k | 2 repos | ~2.9k | Automated safety check: Pass | MIT | |
| Sentence Transformers EmbeddingsOrchestra-Research/AI-Research-SKILLs | 13k | 2 repos | ~1.6k | Automated safety check: Pass | MIT | |
| Agentsop Multiscale Chunkingagentsope/SkillAlchemy | 436 | — | ~4.9k | Automated safety check: Pass | MIT | |
| Agentsop Reranker Stageagentsope/SkillAlchemy | 436 | — | ~4.9k | Automated safety check: Pass | MIT |
Similar skills
Phoenix LLM Observability
Orchestra-Research/AI-Research-SKILLs
Sets up Arize Phoenix to trace, evaluate and monitor LLM applications, with instrumentation for OpenAI, LangChain and LlamaIndex and a self-hosted server.
Sentence Transformers Embeddings
Orchestra-Research/AI-Research-SKILLs
Generates text embeddings locally with the sentence-transformers library for RAG, semantic search, clustering and similarity, with model picks for general, multilingual and legal text.
Agentsop Multiscale Chunking
agentsope/SkillAlchemy
Designs multiscale chunking for RAG by embedding small units for retrieval precision and returning larger context for synthesis.
Agentsop Reranker Stage
agentsope/SkillAlchemy
Adds and tunes a reranker stage for RAG using the retrieve-wide, rerank-narrow pattern.
Agentsop Hybrid Retrieval
agentsope/SkillAlchemy
Enhancement-overlay SOP for adding sparse (BM25 / keyword) retrieval alongside dense (embedding) retrieval.
More from VectorSpaceLab/AREX-Skill
All 157 skills in this repoAgent Lightning
VectorSpaceLab/AREX-Skill
Use this repo skill for Agent Lightning package tasks: authoring trainable agents, tracing rewards and spans, running LightningStore/Trainer loops, using agl CLI services, choosing examples, and…
Agent Tools
VectorSpaceLab/AREX-Skill
A skill your agent uses when configuring LiteLLM for MCP tools, A2A agents, Claude Code/Cursor agent gateway traffic, MCP auth/OAuth, tool permissions, semantic filtering, or agent-specific proxy…
Agents And Awel
VectorSpaceLab/AREX-Skill
Build and debug DB-GPT agents, tools, skills, teams, and AWEL workflows, including deterministic local DAG runs and HTTP-trigger topology without assuming an LLM, credential, or external service.
Agents And Middleware
VectorSpaceLab/AREX-Skill
Work on the actively maintained LangChain v1 agent package: initchatmodel, createagent, structured output, tools, middleware, embeddings initialization, provider routing, and agent runtime…
Agents Workflows
VectorSpaceLab/AREX-Skill
A skill your agent uses for giskard.agents async chat workflows, tools, prompt templates, structured outputs, retries, rate limiting, embeddings, and optional LiteLLM backend.
Alphafold3
VectorSpaceLab/AREX-Skill
A skill your agent uses for AlphaFold 3 input preparation, prediction command planning, output interpretation, and Python API inspection.
Related
Works with
Categories
Questions about Ingestion And Loading
What does Ingestion And Loading do?
Load local files or raw text into LlamaIndex Documents, split them into metadata-aware nodes, and run ingestion pipelines with cache/docstore controls. Ingestion And Loading is an agent skill from VectorSpaceLab/AREX-Skill. Load local files or raw text into LlamaIndex Documents, split them into metadata-aware nodes, and run ingestion pipelines with cache/docstore controls.
When should I use Ingestion And Loading?
Ingestion And Loading fits situations like: simpleDirectoryReader configuration; document/TextNode metadata; sentenceSplitter/TokenTextSplitter/MarkdownNodeParser/HierarchicalNodeParser; ingestionPipeline.
How do I install Ingestion And Loading in Claude Code?
Run `npx skills add VectorSpaceLab/AREX-Skill --skill ingestion-and-loading -a claude-code`. Or copy the skill folder (skills/repositories/repo-skills/llama-index/sub-skills/ingestion-and-loading in VectorSpaceLab/AREX-Skill) into .claude/skills/ingestion-and-loading in your project. Claude Code loads it when a task matches its description.
How do I install Ingestion And Loading in Codex?
Run `npx skills add VectorSpaceLab/AREX-Skill --skill ingestion-and-loading -a codex`. Or copy the skill folder (skills/repositories/repo-skills/llama-index/sub-skills/ingestion-and-loading in VectorSpaceLab/AREX-Skill) into .agents/skills/ingestion-and-loading in your project. Codex loads it when a task matches its description.
Can I use Ingestion And Loading in Cursor, Gemini CLI or GitHub Copilot?
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add VectorSpaceLab/AREX-Skill --skill ingestion-and-loading -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/ingestion-and-loading, .gemini/skills/ingestion-and-loading, .github/skills/ingestion-and-loading and .opencode/skills/ingestion-and-loading in your project.
What does Ingestion And Loading need to run?
Going by SKILL.md and its folder, Ingestion And Loading needs Python for the scripts in its folder and the command-line tools its instructions call (python). Our summary lists: Python 3.
Does Ingestion And Loading access the network?
SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.
Is Ingestion And Loading safe to install?
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. The check reads SKILL.md only: the scripts in the folder are not scanned, so read them before running anything.
What licence does Ingestion And Loading use?
Ingestion And Loading is published under the MIT licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.
How many tokens does Ingestion And Loading use?
About 931 tokens (SKILL.md is roughly 3.7k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 3.8k tokens, read only when the agent opens those files.
What are the alternatives to Ingestion And Loading?
Skills that share tags, products or a category with Ingestion And Loading: Llamaindex (Orchestra-Research/AI-Research-SKILLs, 13k stars), Phoenix LLM Observability (Orchestra-Research/AI-Research-SKILLs, 13k stars), Sentence Transformers Embeddings (Orchestra-Research/AI-Research-SKILLs, 13k stars) and Agentsop Multiscale Chunking (agentsope/SkillAlchemy, 436 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Who maintains Ingestion And Loading?
VectorSpaceLab (a GitHub organization) maintains it in VectorSpaceLab/AREX-Skill, which has 331 GitHub stars. The repository holds 157 skills in this directory. The repository was last updated on September 3, 2026.
Source: VectorSpaceLab/AREX-Skill on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.