Search

Search the awesome-japanese-nlp-resources database for the user's query.

Claude Code and Codex

This skill is shared by the Claude Code and Codex versions of the plugin. The steps are the same in both tools; only these details differ:

• Query — Claude Code: the arguments of /awesome-japanese-nlp-resources:search, appended at the end of this skill as ARGUMENTS: …. Codex: the user's message that invoked $awesome-japanese-nlp-resources:search, minus that $… mention. If the skill was picked automatically rather than invoked by name, use the user's request as the query.
• Plugin root — Claude Code: ${CLAUDE_PLUGIN_ROOT}. Codex: the directory two levels above this SKILL.md (use its absolute path).
• Shell — run the commands below with Claude Code's Bash tool or Codex's shell tool. Copy each Python script in full and run it as written, changing only its placeholders (RESOURCES_PATH, the keyword lists) — don't shorten it, drop passes, or alter its scores and thresholds.
• Commands — write any command you show the user in the current tool's form: /awesome-japanese-nlp-resources:<skill> in Claude Code, $awesome-japanese-nlp-resources:<skill> in Codex.

Results must come from the bundled data. If the data file can't be read (for example, shell commands are blocked or fail to start), say so and link https://github.com/taishi-i/awesome-japanese-nlp-resources instead of answering from memory or web search.

Instructions

Step 0 — Validate input

If the query is empty or blank, stop immediately and output (in Codex, write the commands with $ instead of /):

Usage: /awesome-japanese-nlp-resources:search <query>

Examples:
  /awesome-japanese-nlp-resources:search morphological analysis
  /awesome-japanese-nlp-resources:search BERT
  /awesome-japanese-nlp-resources:search named entity recognition
  /awesome-japanese-nlp-resources:search text classification dataset
  /awesome-japanese-nlp-resources:search sentence embedding

Please pass the keyword(s) you want to search for as the argument.

---

使い方: /awesome-japanese-nlp-resources:search <query>

クエリ例:
  /awesome-japanese-nlp-resources:search 形態素解析
  /awesome-japanese-nlp-resources:search BERT
  /awesome-japanese-nlp-resources:search 固有表現認識
  /awesome-japanese-nlp-resources:search テキスト分類 データセット
  /awesome-japanese-nlp-resources:search 文埋め込み

検索したいキーワードを引数に指定してください。

Do not proceed to Step 1 if the query is empty.

Step 1 — Interpret the query

The data descriptions are in English, so always convert the query intent to English keywords before searching.

Keyword rules — read before choosing keywords:

1. Use stems, not full words. Substring match is used, so morpholog catches "morphology", "morphological", "morphological analyzer". Other examples: embed → embedding/embeddings, classif → classification/classifier, translat → translation/translate, generat → generation/generative, segment → segmentation/segmenter, recogni → recognition/recognizer, extract → extraction/extractor, retriev → retrieval/retrieve.
2. Add domain-specific tool names. When the query maps to a known NLP domain, include the well-known tool names present in the database:

3. When the query contains Japanese text, also keep 2–4 raw Japanese terms/phrases lifted directly from the query (not translated) as a separate ja_keywords list. Aliases and some descriptions (al, d_ja — see Step 3) are Japanese-only, so a literal Japanese substring catches entries an English-only translation would miss entirely — nicknames like ボイボ (VOICEVOX), めかぶ (mecab), or a Japanese technical term that never got glossed into the English description. Leave ja_keywords empty for English queries.
4. Aim for 4–6 keywords. Fewer miss items; more than 6 inflates low-quality partial matches.
5. If none of the above domains fit, translate the query intent literally to English stems.

Step 2 — Locate the data file

The data file ships with the plugin at data/resources.json under the plugin root (see "Claude Code and Codex" above). Resolve its absolute path, falling back to a scoped search only if the install is unusual:

PLUGIN_ROOT="${CLAUDE_PLUGIN_ROOT}"  # Codex: replace with the plugin root, two levels above this SKILL.md
RESOURCES_PATH="$PLUGIN_ROOT/data/resources.json"
[ -f "$RESOURCES_PATH" ] || RESOURCES_PATH="$(find "${CODEX_HOME:-$HOME/.codex}/plugins" "${HOME}/.claude/plugins" -type f -name resources.json 2>/dev/null | grep "awesome-japanese-nlp-resources/" | head -1)"
echo "RESOURCES_PATH=$RESOURCES_PATH"

Use the resulting absolute RESOURCES_PATH wherever Step 3 opens the data file — write the path itself into the script, since shell variables may not persist between commands.

The plugin also ships data/multilingual_resources.json (same item format) listing multilingual libraries, models, and datasets (GitHub repositories) that also support Japanese, from docs/multilingual.md. The scripts below load it automatically when it exists; its items have categories like Multilingual (Speech recognition).

Step 3 — Search and score with Python

Do not read the data file directly (no Read tool, cat, head, or similar) — it is about 660 KB and would flood the context. Instead, run the scoring in a single shell command using Python.

Each item in the JSON array has:

• u: GitHub or Hugging Face URL
• n: repository/model name
• d: English description
• d_ja: Japanese description (GitHub-origin items only; match your ja_keywords against this)
• al: curated alternate names / kana nicknames, e.g. ["VOICEVOX", "ボイスボックス", "ボイボ"] (array of strings, only ~40 items have this — treat a hit here as strong as a name match)
• c: category (e.g. Python library, HuggingFace Model (Text Generation), Corpus, Tutorial, Multilingual (Speech recognition), ...)
• s: subcategory / semantic labels (array of strings)
• st: GitHub star count (GitHub items only; absent or 0 otherwise)
• ns: normalized star score 0–10 (log-scaled, GitHub items only)
• dl: Hugging Face download count (HF items only; absent or 0 otherwise)
• nd: normalized download score 0–10 (log-scaled, HF items only)
• sc: pre-computed quality score (higher = more popular/active)
• status: "ok" or "not_found" — items whose repo 404s (~8 of ~1200) are filtered out below; never recommend one

Run the following, substituting RESOURCES_PATH with the absolute path from Step 2, keywords with your English keywords and ja_keywords with your raw Japanese terms, both from Step 1 (ja_keywords may be []):

python3 << 'EOF'
import json, os

with open("RESOURCES_PATH") as f:    # absolute path from Step 2
    data = json.load(f)
multilingual_path = os.path.join(os.path.dirname("RESOURCES_PATH"), "multilingual_resources.json")
if os.path.exists(multilingual_path):
    with open(multilingual_path) as f:
        data += json.load(f)

keywords = ["keyword1", "keyword2", "keyword3"]  # English stems, from Step 1
ja_keywords = []  # raw Japanese terms from Step 1 -- [] for English queries

results = []
for item in data:
    if item.get("status") == "not_found":
        continue  # dead repo -- never recommend it

    n = item.get("n", "").lower()
    d = item.get("d", "").lower()
    d_ja = item.get("d_ja") or ""
    s = " ".join(item.get("s") or []).lower()
    c = item.get("c", "").lower()
    al = " ".join(item.get("al") or []).lower()

    text_score = 0
    for kw in keywords:
        kw = kw.lower()
        if n == kw:       text_score += 20
        elif kw in n:     text_score += 10
        if kw in d:       text_score += 5
        if kw in s:       text_score += 3
        if kw in c:       text_score += 2
        if kw in al:      text_score += 10  # alias hit is name-equivalent

    for kw in ja_keywords:
        if kw in n:       text_score += 10
        if kw in d_ja:    text_score += 5
        if kw in al:      text_score += 10

    if text_score < 8:
        continue

    ns = item.get("ns") or 0
    nd = item.get("nd") or 0
    sc = item.get("sc") or 0
    pop = (ns if ns else nd) * 2.5
    qual = min(5, sc * 5 / 21)
    combined = text_score + pop + qual

    results.append((combined, text_score, item))

results.sort(key=lambda x: -x[0])
seen = {item['n'] for _, _, item in results}

# Supplemental pass: surface high-popularity items from matching categories
# that may have been missed because their descriptions are in Japanese.
# Keys are stems to match against user keywords; values are category prefixes
# (prefix match covers "HuggingFace Model (Text Generation)" etc.).
CATEGORY_KEYWORDS = {
    "tutorial": "Tutorial", "introduc": "Tutorial", "learn": "Tutorial",
    "morpholog": "Python library", "segment": "Python library",
    "mecab": "Python library", "janome": "Python library", "sudachi": "Python library",
    "spacy": "Python library", "ginza": "Python library",
    "corpus": "Corpus", "dataset": "Corpus",
    "bert": "HuggingFace Model", "gpt": "HuggingFace Model",
    "llm": "HuggingFace Model", "llama": "HuggingFace Model",
    "pretrain": "HuggingFace Model", "embed": "HuggingFace Model",
    "model": "Pretrained model",
}
supplement_cats = set()
for kw in keywords:
    for ck, cat in CATEGORY_KEYWORDS.items():
        if ck in kw.lower():
            supplement_cats.add(cat)

if supplement_cats:
    def cat_match(c):
        return any(c == cat or c.startswith(cat + " ") for cat in supplement_cats)
    extras = [
        item for item in data
        if cat_match(item.get("c", ""))
        and (item.get("st", 0) or item.get("dl", 0))
        and item["n"] not in seen
        and item.get("status") != "not_found"
    ]
    extras.sort(key=lambda x: -max(x.get("ns") or 0, x.get("nd") or 0))
    for item in extras[:5]:
        ns = item.get("ns") or 0
        nd = item.get("nd") or 0
        sc = item.get("sc") or 0
        # base 8 = category-match credit (same as the text_score threshold)
        combined = 8 + max(ns, nd) * 2.5 + min(5, sc * 5 / 21)
        results.append((combined, 0, item))
        seen.add(item["n"])

results.sort(key=lambda x: -x[0])
for combined, text_score, item in results[:20]:
    st = item.get("st", 0) or 0
    dl = item.get("dl", 0) or 0
    flag = " [supplemental]" if text_score == 0 else ""
    print(f"score={combined:.1f} text={text_score} st={st} dl={dl}{flag}")
    print(f"  n={item['n']}")
    print(f"  u={item['u']}")
    print(f"  c={item['c']}")
    print(f"  s={item.get('s','')}")
    if item.get('al'):
        print(f"  al={item['al']}")
    print(f"  d={item.get('d','')[:120]}")
    if item.get('d_ja'):
        print(f"  d_ja={item['d_ja'][:120]}")
    print()
EOF

This returns up to 20 candidates. Items marked [supplemental] were added by the category-based pass to recover high-star resources whose descriptions are in Japanese. In Step 4, evaluate supplemental items on semantic fit before including them in the final list.