Agent skill

Scholar Evaluation

by jimmc414 in jimmc414/Kosmos

Systematic framework for evaluating scholarly and research work based on the ScholarEval methodology.

No licenceAuto-check passedResearch & Science

Install Scholar Evaluation

skills CLI
$ npx skills add jimmc414/Kosmos --skill scholar-evaluation -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install jimmc414/Kosmos scholar-evaluation --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/jimmc414/Kosmos.git skills-src && mkdir -p .claude/skills && cp -r skills-src/kosmos-claude-scientific-skills/scientific-skills/scholar-evaluation .claude/skills/scholar-evaluation && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
scholar-evaluation
GitHub stars
595
Used in
1 other repo
Token cost
~2.5k tokens
SKILL.md length
1,082 words
Files
3 (incl. scripts, references)
Skills in repo
6
Repo updated
First seen
Licence
None found

At a glance

Systematic framework for evaluating scholarly and research work based on the ScholarEval methodology.

  • Works in 6 steps: Initial Assessment and Scope Definition → Dimension-Based Evaluation → Scoring and Rating → …
  • Tasks that involve Literature review
  • SKILL.md covers Overview, When to Use This Skill, Evaluation Workflow and Resources, plus 4 more sections
  • Runs Python scripts from its folder

What it does

Scholar Evaluation is an agent skill from jimmc414/Kosmos. Systematic framework for evaluating scholarly and research work based on the ScholarEval methodology. This skill should be used when assessing research papers, evaluating literature reviews, scoring research methodologies, analyzing scientific writing quality, or applying structured evaluation criteria to academic work. Provides comprehensive assessment across multiple dimensions including problem formulation, literature review, methodology, data collection, analysis, results interpretation, and scholarly writing…

Its SKILL.md is about 2.5k tokens, which your agent loads only when the skill is triggered. The skill folder holds 4 other files, including scripts and reference files (for example `references/evaluation_framework.md` and `scripts/calculate_scores.py`).

It sits in Research & Science, covering Literature review, Peer review and Experimental design. The repository describes itself as: Kosmos: An AI Scientist for Autonomous Discovery - An implementation and adaptation to be driven by Claude Code or API - Based on the Kosmos AI Paper -….

When your agent uses it

  • Tasks that involve Literature review
  • Tasks that involve Peer review
  • Tasks that involve Experimental design

Example prompts

  • “/scholar-evaluation”

Requirements

  • Python 3

Workflow steps

6 steps, taken from the step headings in SKILL.md.

  1. Initial Assessment and Scope Definition
  2. Dimension-Based Evaluation
  3. Scoring and Rating
  4. Synthesize Overall Assessment
  5. Provide Actionable Feedback
  6. Contextual Considerations

What it can do on your machine

Read from SKILL.md and the folder at commit 73f4d2a. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Ships 1 file in scripts/ (Python), which the agent can run.

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    Links to these hosts (documentation or services it may open):

    • arxiv.org

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Scholar Evaluation loads about 2.5k tokens when it runs, and up to ~7.5k if it reads all its reference files. Until then it costs about 137 tokens; SKILL.md has 1,082 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~137
When it runs · the whole SKILL.md, loaded when a task matches
~2.5k
With references · SKILL.md plus every file in references/, read only if the agent opens them
~7.5k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.

SKILL.md

Without a licence we can't republish the file, so here is its outline and opening line. It has 1,082 words (~2,539 tokens).

“Apply the ScholarEval framework to systematically evaluate scholarly and research work. This skill provides structured evaluation methodology based on peer-reviewed research assessment criteria, enabling comprehensive analysis of academic papers, research proposals, literature reviews, and scholarly writing across multiple quality dimensions.”

— opening of SKILL.md by jimmc414
name
scholar-evaluation

Read the full SKILL.md on GitHub

Files

SKILL.md and 2 other files (scripts, references) in kosmos-claude-scientific-skills/scientific-skills/scholar-evaluation of jimmc414/Kosmos.

  • SKILL.md
  • references/evaluation_framework.md
  • scripts/calculate_scores.py

Open the folder on GitHubat commit 73f4d2a

Used in 1 other repository

We found 2 copies of this SKILL.md (exact, near-identical or edited) in other folders, from 1 other GitHub owner. This page covers the copy in jimmc414/Kosmos, which our catalogue first saw on October 7, 2026.

Compare with similar skills

Scholar Evaluation next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Scholar Evaluation compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Scholar Evaluation this skilljimmc414/Kosmos5951 repos~2.5kAutomated safety check: PassNone
Academic Paper Writing PipelineImbad0202/academic-research-skills51k—~16kAutomated safety check: PassCustom licence
Academic Research PipelineImbad0202/academic-research-skills51k—~15kAutomated safety check: PassCustom licence
Benchmark Paper TemplateHKUSTDial/Supervisor-Skills8.7k—~2.8kAutomated safety check: PassCC-BY-4.0
Academic Research Suite for CodexImbad0202/academic-research-skills-codex12k—~12kAutomated safety check: PassCustom licence
Social Science Paper Writingfakerqwq/social-science-paper-writing-skill378—~7kAutomated safety check: PassNone

Similar skills

  • Academic Paper Writing Pipeline

    Imbad0202/academic-research-skills

    Runs a 12-agent pipeline that plans, drafts, cites, reviews and formats academic papers, with modes for revision, rebuttals, abstracts and citation checks.

    51k GitHub stars~16k tokensUpdated today
    Research & ScienceAuto-check passed
  • Academic Research Pipeline

    Imbad0202/academic-research-skills

    Orchestrates a ten-stage academic workflow from research to finished manuscript, including integrity checks, two rounds of peer review and revision.

    51k GitHub stars~15k tokensUpdated today
    Research & ScienceAuto-check passed
  • Benchmark Paper Template

    HKUSTDial/Supervisor-Skills

    Structures benchmark and evaluation papers around five pillars, with a completeness audit, an Introduction logic chain, a section skeleton and a pre-submission checklist.

    8.7k GitHub stars~2.8k tokensUpdated 1 mo ago
    Research & ScienceAuto-check passed
  • Academic Research Suite for Codex

    Imbad0202/academic-research-skills-codex

    A router skill that sends academic work such as literature reviews, drafting, citation checks, peer review and revision to the right workflow in the ARS suite.

    12k GitHub stars~12k tokensUpdated 5 days ago
    Research & ScienceAuto-check passed
  • Social Science Paper Writing

    fakerqwq/social-science-paper-writing-skill

    Helps draft, diagnose, review and revise social science papers, from topic and research question to literature review, citation risks and pre-submission checks.

    378 GitHub stars~7k tokensUpdated 4 mo ago
    Research & ScienceAuto-check passed
  • Academic Research

    voidful/academic-skills

    Complete academic research skill suite covering the full pipeline: paper reading (read/explain papers with storytelling), idea generation (brainstorm research directions), experiment design (plan…

    134 GitHub stars~887 tokensUpdated 6 mo ago
    Research & ScienceAuto-check passed

More from jimmc414/Kosmos

  • Markitdown

    jimmc414/Kosmos

    Convert various file formats (PDF, Office documents, images, audio, web content, structured data) to Markdown optimized for LLM processing.

    595 GitHub starsUsed in 2 repos~1.7k tokens
    Auto-check passed
  • Reportlab

    jimmc414/Kosmos

    PDF generation toolkit. An agent skill from jimmc414/Kosmos.

    595 GitHub starsUsed in 1 repo~4.2k tokens
    Auto-check passed
  • Scientific Schematics

    jimmc414/Kosmos

    Create publication-quality scientific diagrams, flowcharts, and schematics using Python (graphviz, matplotlib, schemdraw, networkx).

    595 GitHub stars~16k tokensUpdated today
    Auto-check: notes
  • Hypothesis Generation

    jimmc414/Kosmos

    Generate testable hypotheses from observations. An agent skill from jimmc414/Kosmos.

    595 GitHub stars~1.7k tokensUpdated today
    Auto-check passed
  • Scientific Writing

    jimmc414/Kosmos

    Write scientific manuscripts. An agent skill from jimmc414/Kosmos.

    595 GitHub starsUsed in 1 repo~4.1k tokens
    Auto-check passed

Questions about Scholar Evaluation

What does Scholar Evaluation do?

Systematic framework for evaluating scholarly and research work based on the ScholarEval methodology. Scholar Evaluation is an agent skill from jimmc414/Kosmos. Systematic framework for evaluating scholarly and research work based on the ScholarEval methodology.

When should I use Scholar Evaluation?

Scholar Evaluation fits situations like: tasks that involve Literature review; tasks that involve Peer review; tasks that involve Experimental design.

How do I install Scholar Evaluation in Claude Code?

Run `npx skills add jimmc414/Kosmos --skill scholar-evaluation -a claude-code`. Or copy the skill folder (kosmos-claude-scientific-skills/scientific-skills/scholar-evaluation in jimmc414/Kosmos) into .claude/skills/scholar-evaluation in your project. Claude Code loads it when a task matches its description.

How do I install Scholar Evaluation in Codex?

Run `npx skills add jimmc414/Kosmos --skill scholar-evaluation -a codex`. Or copy the skill folder (kosmos-claude-scientific-skills/scientific-skills/scholar-evaluation in jimmc414/Kosmos) into .agents/skills/scholar-evaluation in your project. Codex loads it when a task matches its description.

Can I use Scholar Evaluation in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add jimmc414/Kosmos --skill scholar-evaluation -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/scholar-evaluation, .gemini/skills/scholar-evaluation, .github/skills/scholar-evaluation and .opencode/skills/scholar-evaluation in your project.

What does Scholar Evaluation need to run?

Going by SKILL.md and its folder, Scholar Evaluation needs Python for the scripts in its folder. Our summary lists: Python 3.

Does Scholar Evaluation access the network?

SKILL.md names 1 domain. As links in the text: arxiv.org. This is read from the text; nothing was executed.

Is Scholar Evaluation safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. The check reads SKILL.md only: the scripts in the folder are not scanned, so read them before running anything.

What licence does Scholar Evaluation use?

No licence was found for Scholar Evaluation or its repository. Without one, default copyright applies: ask the author before reusing or redistributing it.

How many tokens does Scholar Evaluation use?

About 2.5k tokens (SKILL.md is roughly 10k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 5k tokens, read only when the agent opens those files.

What are the alternatives to Scholar Evaluation?

Skills that share tags, products or a category with Scholar Evaluation: Academic Paper Writing Pipeline (Imbad0202/academic-research-skills, 51k stars), Academic Research Pipeline (Imbad0202/academic-research-skills, 51k stars), Benchmark Paper Template (HKUSTDial/Supervisor-Skills, 8.7k stars) and Academic Research Suite for Codex (Imbad0202/academic-research-skills-codex, 12k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Scholar Evaluation?

jimmc414 (a GitHub user) maintains it in jimmc414/Kosmos, which has 595 GitHub stars. The repository holds 6 skills in this directory. The repository was last updated on October 9, 2026.

Source: jimmc414/Kosmos on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.