Multimodal Media Feature Extraction

What it does

When your agent uses it

• Extracting image or video-frame features for a media dataset
• Computing speech or music features from audio recordings
• Running large Gemini batch jobs over media files and retrying failed requests
• Coding visual content semantically with Gemini across a whole dataset

Example prompts

• “Extract pixel features from the images in ./media/posts and keep post_id from posts.xlsx.”
• “Run openSMILE speech features on the interview recordings in ./audio.”
• “Submit the Gemini batch job for the ad videos and show which requests failed.”
• “Pull librosa music features from the tracks in ./playlists.”

Requirements

• Python with Pillow, openSMILE and librosa for the local pipelines
• FFmpeg for video frames
• Google API keys for Gemini in a .env file

What it can do on your machine

Read from SKILL.md and the folder at commit d8aaee7. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  Ships 8 files in scripts/ (Python, from the files we listed), which the agent can run.

  Shell commands in SKILL.md call:

  • python

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  No URLs in SKILL.md.

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names these keys or tokens, usually read from environment variables:

  • GOOGLE_API_KEY

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Multimodal Media Feature Extraction loads about 2k tokens when it runs, and up to ~12k if it reads all its reference files. Until then it costs about 63 tokens; SKILL.md has 775 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~63

When it runs · the whole SKILL.md, loaded when a task matches

~2k

With references · SKILL.md plus every file in references/, read only if the agent opens them

~12k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

• NoteMentions a .env fileSKILL.md:12

  put/` — user assets (input data, media, `.env`, system prompt) can stay wherever they already live; point the scripts at

• NoteMentions a .env fileSKILL.md:18

  - `.env` with `GOOGLE_API_KEY1`-`4` (Gemini only; check project root, home directory, common locations)

• NoteMentions a .env fileSKILL.md:56

  gle-genai`, `python-dotenv` (+ above) | `.env` with `GOOGLE_API_KEY1`-`4` |

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.

SKILL.md

The full file from TyrealQ/q-skills at commit d8aaee7, republished under its MIT licence (© TyrealQ). 775 words, ~2,033 tokens.

name

q-multimodal

description

Extract visual, video, and audio features from media. Use for pixel features (Pillow), video frames (FFmpeg+Pillow), speech/audio features (openSMILE), music features (librosa), and visual semantic analysis (Gemini API batch or standard).

python 0uploadMedia.py --config /path/to/config.py --submit --max-batch-gb 2 --key 1
python 3checkStatus.py --config /path/to/config.py --poll
python 4retryErrors.py --config /path/to/config.py --preview
# >500 failures: batch retry
python 4retryErrors.py --config /path/to/config.py --submit
python 3checkStatus.py --config /path/to/config.py --poll
python 4retryErrors.py --config /path/to/config.py --collect
# <=500 failures or after batch retries: live fallback
python 4retryErrors.py --config /path/to/config.py --standard
python 5review.py --config /path/to/config.py --merge

© TyrealQ, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 26 other files (scripts, references) in skills/q-scholar/q-multimodal of TyrealQ/q-skills.

• SKILL.md
• references/audio-features.md
• references/checkpoint-format.md
• references/gemini-batch-workflow.md
• references/gemini-standard.md
• references/image-visual-features.md
• references/multi-key-management.md
• references/music-features.md
• references/video-visual-features.md
• scripts/common.py
• scripts/gemini/batch/0uploadMedia.py
• scripts/gemini/batch/1buildJsonl.py
• scripts/gemini/batch/2submitJobs.py
• scripts/gemini/batch/3checkStatus.py
• scripts/gemini/batch/4retryErrors.py
• scripts/gemini/batch/5review.py
• scripts/gemini/batch/utils.py
• … and 10 more

Open the folder on GitHubat commit d8aaee7