Official agent skill

Model Selection

by awslabs in awslabs/agent-plugins

Selects a base model for the user's use case by querying SageMaker Hub.

OfficialApache-2.0Auto-check passedAI & LLM Engineering

Install Model Selection

skills CLI
$ npx skills add awslabs/agent-plugins --skill model-selection -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install awslabs/agent-plugins model-selection --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/awslabs/agent-plugins.git skills-src && mkdir -p .claude/skills && cp -r skills-src/plugins/sagemaker-ai/skills/model-selection .claude/skills/model-selection && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
model-selection
GitHub stars
912
Token cost
~844 tokens
SKILL.md length
357 words
Files
12 (incl. scripts, references)
Skills in repo
33
Repo updated
First seen
Licence
Apache-2.0

At a glance

Selects a base model for the user's use case by querying SageMaker Hub.

  • Works in 4 steps: Check Region → Discover Hub → Select Base Model → …
  • The user asks which model to use
  • SKILL.md covers When to Use, Prerequisites, Workflow and References
  • Runs Python scripts from its folder; calls python and aws

What it does

Model Selection is an agent skill from awslabs/agent-plugins, published by the product's own GitHub organization. Selects a base model for the user's use case by querying SageMaker Hub. Use when the user asks which model to use, wants to select or change their base model, mentions a model name or family (e.g., "Llama", "Mistral", "Nova"), or wants to evaluate a base model — always activate even for known model names because the exact Hub model ID must be resolved. Queries available models, presents benchmarks and licenses, and confirms selection.

Its SKILL.md is about 840 tokens, which your agent loads only when the skill is triggered. The skill folder holds 14 other files, including scripts and reference files (for example `references/benchmarks/agenticIndex.md`, `references/benchmarks/codingIndex.md` and `references/benchmarks/gpqa.md`).

It sits in AI & LLM Engineering. It works with Amazon SageMaker, Amazon Web Services and Mistral AI. The repository describes itself as: Agent Plugins for AWS equip AI coding agents with the skills to help you architect, deploy, and operate on AWS. The licence is Apache-2.0.

When your agent uses it

  • The user asks which model to use
  • Wants to select
  • Change their base model
  • Mentions a model name

Example prompts

  • “Mistral”
  • “Use the model-selection skill to select a base model for the user's use case by querying SageMaker Hub”
  • “/model-selection”

Requirements

  • Python 3

Workflow steps

4 steps, taken from the step headings in SKILL.md.

  1. Check Region
  2. Discover Hub
  3. Select Base Model
  4. Confirm Selection

What it can do on your machine

Read from SKILL.md and the folder at commit da51970. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Ships 1 file in scripts/ (Python), which the agent can run.

    Shell commands in SKILL.md call:

    • python
    • aws

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    Links to these hosts (documentation or services it may open):

    • huggingface.co

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Model Selection loads about 844 tokens when it runs, and up to ~11k if it reads all its reference files. Until then it costs about 114 tokens; SKILL.md has 357 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~114
When it runs · the whole SKILL.md, loaded when a task matches
~844
With references · SKILL.md plus every file in references/, read only if the agent opens them
~11k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.

SKILL.md

The full file from awslabs/agent-plugins at commit da51970, republished under its Apache-2.0 licence (© awslabs). 357 words, ~844 tokens.

Download SKILL.mdSave it as .claude/skills/model-selection/SKILL.md (or your agent's skills folder). This skill also uses 11 other files; get the full folder from GitHub.
name
model-selection
description
Selects a base model for the user's use case by querying SageMaker Hub. Use when the user asks which model to use, wants to select or change their base model, mentions a model name or family (e.g., "Llama", "Mistral", "Nova"), or wants to evaluate a base model — always activate even for known model names because the exact Hub model ID must be resolved. Queries available models, presents benchmarks and licenses, and confirms selection.
metadata.version
1.0.0

Model Selection

Guides the user through selecting a base model based on their use case.

When to Use

  • User asks which model to use
  • User wants to select or change their base model
  • User mentions a model name or family (e.g., "Llama", "Mistral", "Nova") — the exact Hub model ID still needs to be resolved
  • User wants to evaluate a base model before deciding whether to finetune

Prerequisites

  • A use_case_spec.md file exists. If not, activate the use-case-specification skill to generate it first.

Workflow

Step 1: Check Region

Run:

python -c "import boto3; print(boto3.session.Session().region_name)"
  • None → STOP. Tell user: "Set your region via export AWS_DEFAULT_REGION=us-west-2 or aws configure."
  • Set → store REGION in context, continue.
Step 2: Discover Hub
  1. List all available SageMaker Hubs in the user's region by calling the SageMaker ListHubs API using the aws___call_aws tool.

  2. From the results, filter out any hub whose HubDescription contains "AI Registry" — these do not contain JumpStart models.

  3. The remaining hubs are eligible (e.g., SageMakerPublicHub and any private hubs).

  4. If exactly one eligible hub exists, use it automatically — do not ask the user.

  5. If multiple eligible hubs exist, present them to the user and ask which one to use. Example:

    I found the following model hubs:
    - SageMakerPublicHub — SageMaker Public Hub
    - Private-Hub-XYZ — Private Hub models
    Which hub would you like to use?
  6. Store the selected hub name for use in subsequent steps.

Show full SKILL.md (152 more words)Show less
Step 3: Select Base Model

First, retrieve all available SageMaker Hub model names by running: python model-selection/scripts/get_model_names.py <hub-name>.

Present all available models to the user with their licenses before making any recommendations. Cross-reference the model list with references/model-licenses.md and display each as <model name> - [<license>](<url>). For example: "Qwen3-4B - Apache 2.0"

If you already know the model the user wants to use (from conversation context or planning files), confirm that it's in the list, display its license, and move on. Otherwise, help the user pick a model following the instructions in references/model-selection.md. Important: Make sure to remember this list of available models when helping with model selection. Don't recommend a model that's not available to the user.

Step 4: Confirm Selection

Present a summary to the user:

Here's what we've selected:
- Base model: [model name]

Ask if they'd like to proceed with this model.

References

  • references/model-selection.md — Model selection instructions and benchmark descriptions
  • references/model-licenses.md — Model license information for display during model selection

© awslabs, Apache-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 11 other files (scripts, references) in plugins/sagemaker-ai/skills/model-selection of awslabs/agent-plugins.

  • SKILL.md
  • references/benchmarks/agenticIndex.md
  • references/benchmarks/codingIndex.md
  • references/benchmarks/gpqa.md
  • references/benchmarks/hle.md
  • references/benchmarks/ifbench.md
  • references/benchmarks/intelligenceIndex.md
  • references/benchmarks/mmmuPro.md
  • references/benchmarks/tau2.md
  • references/model-licenses.md
  • references/model-selection.md
  • scripts/get_model_names.py

Open the folder on GitHubat commit da51970

Compare with similar skills

Model Selection next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Model Selection compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Model Selection this skillawslabs/agent-plugins912—~844Automated safety check: PassApache-2.0
SageMaker Serving Image Selectionhuggingface/skills11k1 repos~4.6kAutomated safety check: PassApache-2.0
Python Environment Setup for SageMakerhuggingface/skills11k2 repos~1.7kAutomated safety check: PassApache-2.0
SageMaker Deployment Plannerhuggingface/skills11k1 repos~2.1kAutomated safety check: PassApache-2.0
Hf Cloud Serving Image Selectionwaybarrios/opencode-power-pack533—~4.3kAutomated safety check: PassApache-2.0
AWS AI MLaws/agent-toolkit-for-aws2.8k—~1.7kAutomated safety check: PassApache-2.0

Similar skills

  • Official

    Chooses the right serving container and current image URI for deploying a Hugging Face model to a SageMaker endpoint, preferring Hugging Face images over generic ones.

    11k GitHub starsUsed in 1 repo~4.6k tokens
    AI & LLM EngineeringAuto-check passed
  • Official

    Sets up an isolated Python environment with a supported interpreter and current boto3 before any SageMaker deployment, training or AWS automation code runs.

    11k GitHub starsUsed in 2 repos~1.7k tokens
    AI & LLM EngineeringAuto-check passed
  • Official

    Entry point for hosting a model on Amazon SageMaker: asks a few questions, picks a deployment pathway and hands off to the specialist skills.

    11k GitHub starsUsed in 1 repo~2.1k tokens
    AI & LLM EngineeringAuto-check passed
  • Hf Cloud Serving Image Selection

    waybarrios/opencode-power-pack

    Select and verify the current region-specific serving container URI for a SageMaker model deployment.

    533 GitHub stars~4.3k tokensUpdated yesterday
    AI & LLM EngineeringAuto-check passed
  • AWS AI ML

    aws/agent-toolkit-for-aws

    Official

    Selects, deploys, and customizes AI models on Amazon SageMaker.

    2.8k GitHub stars~1.7k tokensUpdated today
    AI & LLM EngineeringAuto-check passed
  • Official

    Finds or validates a usable SageMaker execution role before deploying or training, so scripts do not try to create IAM roles they lack permission to create.

    11k GitHub starsUsed in 1 repo~1.8k tokens
    DevOps & CloudAuto-check passed

More from awslabs/agent-plugins

All 33 skills in this repo
  • Dataset Evaluation

    awslabs/agent-plugins

    Official

    Validates dataset formatting and quality for SageMaker model fine-tuning (SFT, DPO, or RLVR).

    912 GitHub starsUsed in 2 repos~1.3k tokens
    Auto-check passed
  • Dataset Transformation

    awslabs/agent-plugins

    Official

    Generates code that transforms datasets between ML schemas for model training or evaluation.

    912 GitHub starsUsed in 2 repos~3.5k tokens
    Auto-check passed
  • Finetuning Technique

    awslabs/agent-plugins

    Official

    Selects a fine-tuning technique (SFT, DPO, RLVR, or RLAIF) for the user's use case and validates it against the selected model's available recipes.

    912 GitHub starsUsed in 1 repo~604 tokens
    Auto-check passed
  • AWS Lambda Managed Instances

    awslabs/agent-plugins

    Official

    Evaluate, configure, and migrate workloads to AWS Lambda Managed Instances (LMI).

    912 GitHub stars~4k tokensUpdated yesterday
    Auto-check passed
  • Hyperpod Issue Report

    awslabs/agent-plugins

    Official

    Generate comprehensive issue reports from HyperPod clusters (EKS and Slurm) by collecting diagnostic logs and configurations for troubleshooting and AWS Support cases.

    912 GitHub stars~890 tokensUpdated yesterday
    Auto-check passed
  • Hyperpod Performance Debugger

    awslabs/agent-plugins

    Official

    Diagnose performance issues on Amazon SageMaker HyperPod clusters — uneven NCCL bandwidth across nodes and poor filesystem throughput.

    912 GitHub stars~4.1k tokensUpdated yesterday
    Auto-check passed

Questions about Model Selection

What does Model Selection do?

Selects a base model for the user's use case by querying SageMaker Hub. Model Selection is an agent skill from awslabs/agent-plugins, published by the product's own GitHub organization. Selects a base model for the user's use case by querying SageMaker Hub.

When should I use Model Selection?

Model Selection fits situations like: the user asks which model to use; wants to select; change their base model; mentions a model name.

How do I install Model Selection in Claude Code?

Run `npx skills add awslabs/agent-plugins --skill model-selection -a claude-code`. Or copy the skill folder (plugins/sagemaker-ai/skills/model-selection in awslabs/agent-plugins) into .claude/skills/model-selection in your project. Claude Code loads it when a task matches its description.

How do I install Model Selection in Codex?

Run `npx skills add awslabs/agent-plugins --skill model-selection -a codex`. Or copy the skill folder (plugins/sagemaker-ai/skills/model-selection in awslabs/agent-plugins) into .agents/skills/model-selection in your project. Codex loads it when a task matches its description.

Can I use Model Selection in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add awslabs/agent-plugins --skill model-selection -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/model-selection, .gemini/skills/model-selection, .github/skills/model-selection and .opencode/skills/model-selection in your project.

What does Model Selection need to run?

Going by SKILL.md and its folder, Model Selection needs Python for the scripts in its folder and the command-line tools its instructions call (python and aws). Our summary lists: Python 3.

Does Model Selection access the network?

SKILL.md names 1 domain. As links in the text: huggingface.co. This is read from the text; nothing was executed.

Is Model Selection safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. The check reads SKILL.md only: the scripts in the folder are not scanned, so read them before running anything.

What licence does Model Selection use?

Model Selection is published under the Apache-2.0 licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Model Selection use?

About 844 tokens (SKILL.md is roughly 3.4k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 9.8k tokens, read only when the agent opens those files.

What are the alternatives to Model Selection?

Skills that share tags, products or a category with Model Selection: SageMaker Serving Image Selection (huggingface/skills, 11k stars), Python Environment Setup for SageMaker (huggingface/skills, 11k stars), SageMaker Deployment Planner (huggingface/skills, 11k stars) and Hf Cloud Serving Image Selection (waybarrios/opencode-power-pack, 533 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Model Selection?

awslabs (a GitHub organization, an official publisher) maintains it in awslabs/agent-plugins, which has 912 GitHub stars. The repository holds 33 skills in this directory. The repository was last updated on October 5, 2026.

Source: awslabs/agent-plugins on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.