Vaex Out-of-Core DataFrames

What it does

When your agent uses it

• Analyzing a CSV or HDF5 file that is larger than available memory
• Computing fast groupby statistics over a very large table
• Plotting a heatmap or histogram of a huge dataset
• Converting a big CSV into Parquet or Arrow

Example prompts

• “Open trips.hdf5 with Vaex and compute the average fare by pickup hour.”
• “Convert this 40 GB CSV to Parquet without loading it all into memory.”
• “Make a 2D heatmap of latitude against longitude for the whole events table.”

Requirements

• Python with `vaex` installed

Workflow steps

6 steps, taken from the step headings in SKILL.md.

1. 1DataFrames and Data Loading
2. 2Data Processing and Manipulation
3. 3Performance and Optimization
4. 4Data Visualization
5. 5Machine Learning Integration
6. 6I/O Operations

What it can do on your machine

Read from SKILL.md and the folder at commit 14680ec. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  No scripts in the folder and no shell commands in SKILL.md (its code samples are python).

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  No URLs in SKILL.md.

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names no API keys, tokens, secrets or passwords.

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Vaex Out-of-Core DataFrames loads about 1.6k tokens when it runs, and up to ~21k if it reads all its reference files. Until then it costs about 120 tokens; SKILL.md has 540 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~120

When it runs · the whole SKILL.md, loaded when a task matches

~1.6k

With references · SKILL.md plus every file in references/, read only if the agent opens them

~21k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from davila7/claude-code-templates at commit 14680ec, republished under its MIT licence (© davila7). 540 words, ~1,633 tokens.

name

vaex

description

Use this skill for processing and analyzing large tabular datasets (billions of rows) that exceed available RAM. Vaex excels at out-of-core DataFrame operations, lazy evaluation, fast aggregations, efficient visualization of big data, and machine learning on large datasets. Apply when users need to work with large CSV/HDF5/Arrow/Parquet files, perform fast statistics on massive datasets, create visualizations of big data, or build ML pipelines that don't fit in memory.

import vaex

# 1. Open or create DataFrame
df = vaex.open('large_file.hdf5')  # or .csv, .arrow, .parquet
# OR
df = vaex.from_pandas(pandas_df)

# 2. Explore the data
print(df)  # Shows first/last rows and column info
df.describe()  # Statistical summary

# 3. Create virtual columns (no memory overhead)
df['new_column'] = df.x ** 2 + df.y

# 4. Filter with selections
df_filtered = df[df.age > 25]

# 5. Compute statistics (fast, lazy evaluation)
mean_val = df.x.mean()
stats = df.groupby('category').agg({'value': 'sum'})

# 6. Visualize
df.plot1d(df.x, limits=[0, 100])
df.plot(df.x, df.y, limits='99.7%')

# 7. Export if needed
df.export_hdf5('output.hdf5')

import vaex

# Open large CSV (processes in chunks automatically)
df = vaex.from_csv('large_file.csv')

# Export to HDF5 for faster future access
df.export_hdf5('large_file.hdf5')

# Future loads are instant
df = vaex.open('large_file.hdf5')

# Use delay=True to batch multiple operations
mean_x = df.x.mean(delay=True)
std_y = df.y.std(delay=True)
sum_z = df.z.sum(delay=True)

# Execute all at once
results = vaex.execute([mean_x, std_y, sum_z])

# No memory overhead - computed on the fly
df['age_squared'] = df.age ** 2
df['full_name'] = df.first_name + ' ' + df.last_name
df['is_adult'] = df.age >= 18

© davila7, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 6 other files (references) in cli-tool/components/skills/scientific/vaex of davila7/claude-code-templates.

• SKILL.md
• references/core_dataframes.md
• references/data_processing.md
• references/io_operations.md
• references/machine_learning.md
• references/performance.md
• references/visualization.md

Open the folder on GitHubat commit 14680ec

Used in 12 other repositories

We found 14 copies of this SKILL.md (exact, near-identical or edited) in other folders, from 12 other GitHub owners. This page covers the copy in davila7/claude-code-templates, which our catalogue first saw on October 7, 2026.

• jimmc414/Kosmoskosmos-claude-scientific-skills/scientific-skills/vaex
• Microck/ordinary-claude-skillsskills_all/claude-scientific-skills/scientific-skills/vaexskills_categorized/scientific-computing/claude-scientific-skills/scientific-skills/vaex
• aiskillstore/marketplaceskills/davila7/vaexskills/k-dense-ai/vaex
• lamm-mit/scienceclawskills/vaex
• agent-skills-hub/agent-skills-hubskills/vaex
• foryourhealth111-pixel/Vibe-Skillsbundled/skills/vaex
• LeonChaoX/qinyan-academic-skillsskills/11-数据分析与统计建模/vaex
• wu-yc/LabClawskills/general/vaex
• FreedomIntelligence/OpenClaw-Medical-Skillsskills/vaex
• franklee16/academic-research-skillsdata-analysis/vaex
• BioTender-max/awesome-bio-agent-skillsskills/openclaw/vaex
• infometa/workbuddyskillsplugins/codebuddy-plugins-official/external_plugins/scientific-skills/vaex