Hugging Face Tokenizers
Orchestra-Research/AI-Research-SKILLs
Shows how to load, train and use fast Hugging Face tokenizers, with BPE, WordPiece and Unigram models, padding, truncation and alignment tracking.
Add or modify a Lizard language reader. An agent skill from terryyin/lizard.
Install Lizard Language Support
$ npx skills add terryyin/lizard --skill lizard-language-support -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install terryyin/lizard lizard-language-support --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/terryyin/lizard.git skills-src && mkdir -p .claude/skills && cp -r skills-src/.agents/skills/lizard-language-support .claude/skills/lizard-language-support && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "lizard-language-support" agent skill from https://github.com/terryyin/lizard/tree/master/.agents/skills/lizard-language-support into .claude/skills/lizard-language-support/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "lizard-language-support", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/terryyin/lizard/tree/master/.agents/skills/lizard-language-supportType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add terryyin/lizard --skill lizard-language-support -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install terryyin/lizard lizard-language-support --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/terryyin/lizard.git skills-src && mkdir -p .agents/skills && cp -r skills-src/.agents/skills/lizard-language-support .agents/skills/lizard-language-support && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "lizard-language-support" agent skill from https://github.com/terryyin/lizard/tree/master/.agents/skills/lizard-language-support into .agents/skills/lizard-language-support/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "lizard-language-support", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add terryyin/lizard --skill lizard-language-support -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install terryyin/lizard lizard-language-support --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/terryyin/lizard.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/.agents/skills/lizard-language-support .cursor/skills/lizard-language-support && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "lizard-language-support" agent skill from https://github.com/terryyin/lizard/tree/master/.agents/skills/lizard-language-support into .cursor/skills/lizard-language-support/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "lizard-language-support", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/terryyin/lizard.git --path .agents/skills/lizard-language-support--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add terryyin/lizard --skill lizard-language-support -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install terryyin/lizard lizard-language-support --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/terryyin/lizard.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/.agents/skills/lizard-language-support .gemini/skills/lizard-language-support && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "lizard-language-support" agent skill from https://github.com/terryyin/lizard/tree/master/.agents/skills/lizard-language-support into .gemini/skills/lizard-language-support/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "lizard-language-support", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install terryyin/lizard lizard-language-supportInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add terryyin/lizard --skill lizard-language-support -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/terryyin/lizard.git skills-src && mkdir -p .github/skills && cp -r skills-src/.agents/skills/lizard-language-support .github/skills/lizard-language-support && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "lizard-language-support" agent skill from https://github.com/terryyin/lizard/tree/master/.agents/skills/lizard-language-support into .github/skills/lizard-language-support/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "lizard-language-support", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add terryyin/lizard --skill lizard-language-support -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install terryyin/lizard lizard-language-support --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/terryyin/lizard.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/.agents/skills/lizard-language-support .opencode/skills/lizard-language-support && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "lizard-language-support" agent skill from https://github.com/terryyin/lizard/tree/master/.agents/skills/lizard-language-support into .opencode/skills/lizard-language-support/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "lizard-language-support", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
Facts
- Skill name
lizard-language-support- GitHub stars
- 2.5k
- Token cost
- ~1.1k tokens
- SKILL.md length
- 304 words
- Files
- 1
- Skills in repo
- 19
- Repo updated
- First seen
- Licence
- Custom licence
At a glance
Add or modify a Lizard language reader. An agent skill from terryyin/lizard.
- Works in 4 steps: Create Language Reader Class → Implement Token Generation → State Machine Implementation → …
- Adding language support
- SKILL.md covers Overview, Step-by-Step Guide, Key Concepts and Best Practices, plus 3 more sections
- Calls python
What it does
Lizard Language Support is an agent skill from terryyin/lizard. Add or modify a Lizard language reader. Use when adding language support, changing tokenization or state-machine parsing, or working in lizardlanguages/ or test/testlanguages/.
Its SKILL.md is about 1.1k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.
It sits in AI & LLM Engineering, covering Natural language processing. The repository describes itself as: A simple code complexity analyser without caring about the C/C++ header files or Java imports, supports most of the popular languages.
When your agent uses it
- Adding language support
- Changing tokenization
- State-machine parsing
- Working in lizardlanguages/
Example prompts
- “/lizard-language-support”
Requirements
- Python 3
Workflow steps
4 steps, taken from the step headings in SKILL.md.
- Create Language Reader Class
- Implement Token Generation
- State Machine Implementation
- Add Test Cases
What it can do on your machine
Read from SKILL.md and the folder at commit d5228c1. It shows what the files ask for, not the result of running them.
Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
Shell commands in SKILL.md call:
python
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Context cost
Lizard Language Support loads about 1.1k tokens when it runs. Until then it costs about 51 tokens; SKILL.md has 304 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
Safety
Auto-check passed
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.
SKILL.md
Its licence (Custom licence) doesn't allow us to republish the file, so here is its outline and opening line. It has 304 words (~1,057 tokens).
“This guide explains how to add or modify support for programming languages in the Lizard code complexity analyzer.”
- name
- lizard-language-support
- paths
- lizard_languages/**, test/test_languages/**
Files
Just SKILL.md in .agents/skills/lizard-language-support of terryyin/lizard.
Open the folder on GitHubat commit d5228c1
Compare with similar skills
Lizard Language Support next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Lizard Language Support this skillterryyin/lizard | 2.5k | — | ~1.1k | Automated safety check: Pass | Custom licence | |
| Hugging Face TokenizersOrchestra-Research/AI-Research-SKILLs | 13k | 7 repos | ~3.4k | Automated safety check: Pass | MIT | |
| OpenMed Model Card Writermaziyarpanahi/openmed | 5.5k | — | ~1.8k | Automated safety check: Pass | Apache-2.0 | |
| Gptqmodel Tokenizer NormalizationModelCloud/GPTQModel | 1.3k | — | ~1.1k | Automated safety check: Pass | Custom licence | |
| Comparetaishi-i/awesome-japanese-nlp-resources | 1k | 1 repos | ~4.1k | Automated safety check: Notes | CC0-1.0 | |
| Researchtaishi-i/awesome-japanese-nlp-resources | 1k | 1 repos | ~3.5k | Automated safety check: Notes | CC0-1.0 |
Similar skills
OpenMed Model Card Writer
maziyarpanahi/openmed
Fills in a model card for an OpenMed clinical NER or de-identification model from its evaluation reports: intended use, metrics, subgroups and limitations.
Gptqmodel Tokenizer Normalization
ModelCloud/GPTQModel
Diagnose and correct GPT-QModel tokenizer initialization, tokenization normalization, special-token handling, prompt rendering, and chat-template problems.
Compare
taishi-i/awesome-japanese-nlp-resources
Compare several Japanese NLP libraries, models, or datasets for a keyword (a specific tool name, or a function/task like '形態素解析') across a handful of criteria chosen for that comparison, rendered as…
Research
taishi-i/awesome-japanese-nlp-resources
Analyze current trends and challenges in Japanese NLP for a topic.
Analyzing Text With NLP
jeremylongshore/tons-of-skills-marketplace
Execute this skill enables AI assistant to perform natural language processing and text analysis using the nlp-text-analyzer plugin.
More from terryyin/lizard
All 19 skills in this repoDough Execute Plan
terryyin/lizard
Executes one selected story or bounded retrospective correction through an executable plan, or one authorized planless slice from a selected simple story or a contextual instruction, with…
Dough Product Backlog
terryyin/lizard
Maintains and reprioritizes a product backlog list of canonical story or bounded-correction references, and records necessary blocking story dependencies in canonical homes.
Dough Execution Retrospective
terryyin/lizard
Reviews planned, completed planless quick, or quick-to-planned execution against original intent, aggregate commits, current whole-product architecture, and tests, including after cleanup.
Dough Land
terryyin/lizard
Dough Land lands everything in one reviewed checkout on the authorized remote trunk: commits all of its changes, reconciles and publishes them without force, and refreshes a supplied default…
Dough Story Refinement
terryyin/lizard
Clarifies selected stories before execution planning by establishing goal, scope, and key examples in each story's seed, distinguishing promises from rejection constraints.
Dough Story Wrap Up
terryyin/lizard
Closes one completed feature story, bounded retrospective correction, or context-only planless execution using available execution context.
Related
Categories
Questions about Lizard Language Support
What does Lizard Language Support do?
Add or modify a Lizard language reader. An agent skill from terryyin/lizard. Lizard Language Support is an agent skill from terryyin/lizard. Add or modify a Lizard language reader.
When should I use Lizard Language Support?
Lizard Language Support fits situations like: adding language support; changing tokenization; state-machine parsing; working in lizardlanguages/.
How do I install Lizard Language Support in Claude Code?
Run `npx skills add terryyin/lizard --skill lizard-language-support -a claude-code`. Or copy the skill folder (.agents/skills/lizard-language-support in terryyin/lizard) into .claude/skills/lizard-language-support in your project. Claude Code loads it when a task matches its description.
How do I install Lizard Language Support in Codex?
Run `npx skills add terryyin/lizard --skill lizard-language-support -a codex`. Or copy the skill folder (.agents/skills/lizard-language-support in terryyin/lizard) into .agents/skills/lizard-language-support in your project. Codex loads it when a task matches its description.
Can I use Lizard Language Support in Cursor, Gemini CLI or GitHub Copilot?
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add terryyin/lizard --skill lizard-language-support -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/lizard-language-support, .gemini/skills/lizard-language-support, .github/skills/lizard-language-support and .opencode/skills/lizard-language-support in your project.
What does Lizard Language Support need to run?
Going by SKILL.md and its folder, Lizard Language Support needs the command-line tools its instructions call (python). Our summary lists: Python 3.
Does Lizard Language Support access the network?
SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.
Is Lizard Language Support safe to install?
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.
What licence does Lizard Language Support use?
Lizard Language Support has a licence file (the repository's licence) that doesn't match a standard licence. Read it on GitHub before reusing the skill.
How many tokens does Lizard Language Support use?
About 1.1k tokens (SKILL.md is roughly 4.2k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.
What are the alternatives to Lizard Language Support?
Skills that share tags, products or a category with Lizard Language Support: Hugging Face Tokenizers (Orchestra-Research/AI-Research-SKILLs, 13k stars), OpenMed Model Card Writer (maziyarpanahi/openmed, 5.5k stars), Gptqmodel Tokenizer Normalization (ModelCloud/GPTQModel, 1.3k stars) and Compare (taishi-i/awesome-japanese-nlp-resources, 1k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Who maintains Lizard Language Support?
terryyin (a GitHub user) maintains it in terryyin/lizard, which has 2,549 GitHub stars. The repository holds 19 skills in this directory. The repository was last updated on October 6, 2026.
Source: terryyin/lizard on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.