Agent skill

Web Research With curl

by laude-institute in laude-institute/headlong

Looks things up online from the shell with curl: fetching pages, searching DuckDuckGo's HTML endpoint, stripping tags for readable text and pulling JSON APIs.

Apache-2.0Auto-check passedProductivity & Automation

Install Web Research With curl

skills CLI
$ npx skills add laude-institute/headlong --skill web-research -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install laude-institute/headlong web-research --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/laude-institute/headlong.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/web-research .claude/skills/web-research && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
web-research
GitHub stars
1.2k
Token cost
~362 tokens
SKILL.md length
106 words
Files
1
Skills in repo
10
Repo updated
First seen
Licence
Apache-2.0

At a glance

Looks things up online from the shell with curl: fetching pages, searching DuckDuckGo's HTML endpoint, stripping tags for readable text and pulling JSON APIs.

  • Looking up current information the agent does not have in context
  • SKILL.md covers When to use, Common operations and Tips
  • Calls curl and jq; reaches html.duckduckgo.com
  • Fetching a web page or JSON API from the command line

What it does

This is a short shell cookbook for questions that need current information. It shows how to fetch a page, run a DuckDuckGo HTML search and pull out result links, strip tags with sed for rough text, query a JSON API and filter it with jq, and download a file. The tips: use curl with -s and -L, cut output down with head or tail because full pages are huge, expect no dynamic content from JavaScript-heavy sites, rate-limit requests and check robots.txt before scraping at length.

When your agent uses it

  • Looking up current information the agent does not have in context
  • Fetching a web page or JSON API from the command line
  • Extracting readable text from a page without a browser

Example prompts

  • “Look up the latest stable release of Node.js and tell me what changed.”
  • “Search the web for how to rotate an SSH key, then summarize the top results.”
  • “Fetch https://api.github.com/repos/curl/curl and tell me how many open issues it has.”

Requirements

  • curl, jq and standard Unix text tools
  • Network access

What it can do on your machine

Read from SKILL.md and the folder at commit c43e597. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Shell commands in SKILL.md call:

    • curl
    • jq

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    Hosts in commands or code, which the agent is likely to contact:

    • html.duckduckgo.com

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Web Research With curl loads about 362 tokens when it runs. Until then it costs about 38 tokens; SKILL.md has 106 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~38
When it runs · the whole SKILL.md, loaded when a task matches
~362

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from laude-institute/headlong at commit c43e597, republished under its Apache-2.0 licence (© laude-institute). 106 words, ~362 tokens.

Download SKILL.mdSave it as .claude/skills/web-research/SKILL.md (or your agent's skills folder).
name
web-research
description
Research topics using the web via curl. Use when the user asks to look something up, find current information, or research a topic online.

web-research

When to use

User asks to look something up, find current information, research a topic, or answer a question that requires up-to-date knowledge beyond what you have in context.

Common operations

Fetch a web page
bash
curl -sL "https://example.com" | head -200
Search via DuckDuckGo (HTML)
bash
query="your search terms"
curl -sL "https://html.duckduckgo.com/html/?q=$(printf '%s' "$query" | jq -sRr @uri)" \
  | grep -oP 'href="https?://[^"]*"' | head -10
Fetch and extract text from HTML
bash
# Strip HTML tags for readable text
curl -sL "$url" | sed 's/<[^>]*>//g' | sed '/^$/d' | head -100
Fetch JSON APIs
bash
curl -s "https://api.example.com/endpoint" | jq '.results[]'
Download a file
bash
curl -sLO "https://example.com/file.txt"

Tips

  • Always use -s (silent) and -L (follow redirects) with curl
  • Pipe HTML through sed 's/<[^>]*>//g' for rough text extraction
  • Use head or tail to limit output — full web pages are huge
  • For JavaScript-heavy sites, raw curl won't render dynamic content
  • Rate-limit requests — don't hammer endpoints in tight loops
  • Check robots.txt before scraping extensively

© laude-institute, Apache-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in skills/web-research of laude-institute/headlong.

Open the folder on GitHubat commit c43e597

Compare with similar skills

Web Research With curl next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Web Research With curl compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Web Research With curl this skilllaude-institute/headlong1.2k—~362Automated safety check: PassApache-2.0
Agent ReachPanniantong/Agent-Reach95k—~1.4kAutomated safety check: PassMIT
Tavily Web Searchallenpeng0705/EnvoyMesh3.1k3 repos~2.5kAutomated safety check: NotesNone
Perplexity SearchescapeWu/perplexity-ai171—~1.8kAutomated safety check: PassMIT
Tavily Web SearchLichAmnesia/lich-skills234—~1kAutomated safety check: NotesMIT
Perplexity Search and Researchartwist-polyakov/polyakov-claude-skills208—~2.5kAutomated safety check: NotesMIT

Similar skills

  • Agent Reach

    Panniantong/Agent-Reach

    Routes web research and platform lookups across 16 sites, including Twitter, Reddit, YouTube, Bilibili, Xiaohongshu and GitHub, through one command-line tool.

    95k GitHub stars~1.4k tokensUpdated 3 days ago
    Productivity & AutomationAuto-check passed
  • Tavily Web Search

    allenpeng0705/EnvoyMesh

    Searches the web through the Tavily API with LLM-friendly output: clean structured results, optional AI-written answers, domain filters, news mode, images and raw content.

    3.1k GitHub starsUsed in 3 repos~2.5k tokens
    Productivity & AutomationAuto-check: notes
  • Perplexity Search

    escapeWu/perplexity-ai

    Searches the live web with citations through perplexity-mcp v2 tools or a bundled Python REST client, with focused ask, deep research and detached tasks.

    171 GitHub stars~1.8k tokensUpdated 13 days ago
    Productivity & AutomationAuto-check passed
  • Tavily Web Search

    LichAmnesia/lich-skills

    Runs headless web searches and single-page extraction through the Tavily API from a Python script, returning cited, summarized results without a browser.

    234 GitHub stars~1k tokensUpdated 4 mo ago
    Productivity & AutomationAuto-check: notes
  • Perplexity Search and Research

    artwist-polyakov/polyakov-claude-skills

    Shell scripts for web search and research through the Perplexity API: raw results, cited answers, background deep research and page fetching, with results cached on disk.

    208 GitHub stars~2.5k tokensUpdated 2 days ago
    Productivity & AutomationAuto-check: notes
  • Searches the web, code, companies and people through the Exa MCP server, with notes on setup, the web_search_exa tool and treating results as untrusted data.

    277k GitHub starsUsed in 4 repos~1.1k tokens
    Productivity & AutomationAuto-check passed

More from laude-institute/headlong

All 10 skills in this repo
  • Shellm Architecture Reference

    laude-institute/headlong

    Explains how shellm's bash-based recursive LLM shell fits together - its core engine, identity system, memory, skills and trajectory log.

    1.2k GitHub stars~2k tokensUpdated 2 days ago
    Auto-check: notes
  • ShellLM Skill Author

    laude-institute/headlong

    Scaffolds new ShellLM skills with the right frontmatter, directory layout and agent-facing writing style, so an agent can extend its own capabilities.

    1.2k GitHub stars~1.3k tokensUpdated 2 days ago
    Auto-check passed
  • Google Workspace CLI

    laude-institute/headlong

    Drives Google Drive, Gmail, Calendar, Sheets, Docs and Chat from the shell with the gws CLI, including its sign-in, flags, helper commands and safety rules.

    1.2k GitHub stars~1.2k tokensUpdated 2 days ago
    Auto-check passed
  • Obsidian

    laude-institute/headlong

    Read, write, search, and manage notes in an Obsidian vault. An agent skill from laude-institute/headlong.

    1.2k GitHub stars~880 tokensUpdated 2 days ago
    Auto-check passed
  • Slack

    laude-institute/headlong

    Talk with people on Slack — recognize slack- senders and reach them via chat reply

    1.2k GitHub stars~831 tokensUpdated 2 days ago
    Auto-check passed
  • Telegram

    laude-institute/headlong

    Send and receive messages via the Telegram Bot API using curl

    1.2k GitHub stars~623 tokensUpdated 2 days ago
    Auto-check passed

Works with

Questions about Web Research With curl

What does Web Research With curl do?

Looks things up online from the shell with curl: fetching pages, searching DuckDuckGo's HTML endpoint, stripping tags for readable text and pulling JSON APIs. This is a short shell cookbook for questions that need current information. It shows how to fetch a page, run a DuckDuckGo HTML search and pull out result links, strip tags with sed for rough text, query a JSON API and filter it with jq, and download a file.

When should I use Web Research With curl?

Web Research With curl fits situations like: looking up current information the agent does not have in context; fetching a web page or JSON API from the command line; extracting readable text from a page without a browser.

How do I install Web Research With curl in Claude Code?

Run `npx skills add laude-institute/headlong --skill web-research -a claude-code`. Or copy the skill folder (skills/web-research in laude-institute/headlong) into .claude/skills/web-research in your project. Claude Code loads it when a task matches its description.

How do I install Web Research With curl in Codex?

Run `npx skills add laude-institute/headlong --skill web-research -a codex`. Or copy the skill folder (skills/web-research in laude-institute/headlong) into .agents/skills/web-research in your project. Codex loads it when a task matches its description.

Can I use Web Research With curl in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add laude-institute/headlong --skill web-research -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/web-research, .gemini/skills/web-research, .github/skills/web-research and .opencode/skills/web-research in your project.

What does Web Research With curl need to run?

Going by SKILL.md and its folder, Web Research With curl needs the command-line tools its instructions call (curl and jq). Our summary lists: curl, jq and standard Unix text tools; Network access.

Does Web Research With curl access the network?

SKILL.md names 1 domain. In commands or code: html.duckduckgo.com; the agent is likely to contact it when it follows the instructions. This is read from the text; nothing was executed.

Is Web Research With curl safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Web Research With curl use?

Web Research With curl is published under the Apache-2.0 licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Web Research With curl use?

About 362 tokens (SKILL.md is roughly 1.4k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Web Research With curl?

Skills that share tags, products or a category with Web Research With curl: Agent Reach (Panniantong/Agent-Reach, 95k stars), Tavily Web Search (allenpeng0705/EnvoyMesh, 3.1k stars), Perplexity Search (escapeWu/perplexity-ai, 171 stars) and Tavily Web Search (LichAmnesia/lich-skills, 234 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Web Research With curl?

laude-institute (a GitHub organization) maintains it in laude-institute/headlong, which has 1,216 GitHub stars. The repository holds 10 skills in this directory. The repository was last updated on October 9, 2026.

Source: laude-institute/headlong on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.