Twitter Crawler
huangserva/servasyy_skills
Twitter 推文爬取器 - 指定用户名爬取推文，保存为 Markdown 格式，支持自定义数量和字段. An agent skill from huangserva/servasyy_skills.
A skill your agent uses when the user wants a single page rendered as clean Markdown plus structured metadata.
Install Scraping HTML To Markdown
$ npx skills add hashgraph-online/awesome-codex-plugins --skill scraping-html-to-markdown -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install hashgraph-online/awesome-codex-plugins scraping-html-to-markdown --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/hashgraph-online/awesome-codex-plugins.git skills-src && mkdir -p .claude/skills && cp -r skills-src/plugins/kreuzberg-dev/plugins/plugins/kreuzcrawl/skills/scraping-html-to-markdown .claude/skills/scraping-html-to-markdown && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "scraping-html-to-markdown" agent skill from https://github.com/hashgraph-online/awesome-codex-plugins/tree/main/plugins/kreuzberg-dev/plugins/plugins/kreuzcrawl/skills/scraping-html-to-markdown into .claude/skills/scraping-html-to-markdown/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "scraping-html-to-markdown", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/hashgraph-online/awesome-codex-plugins/tree/main/plugins/kreuzberg-dev/plugins/plugins/kreuzcrawl/skills/scraping-html-to-markdownType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add hashgraph-online/awesome-codex-plugins --skill scraping-html-to-markdown -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install hashgraph-online/awesome-codex-plugins scraping-html-to-markdown --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/hashgraph-online/awesome-codex-plugins.git skills-src && mkdir -p .agents/skills && cp -r skills-src/plugins/kreuzberg-dev/plugins/plugins/kreuzcrawl/skills/scraping-html-to-markdown .agents/skills/scraping-html-to-markdown && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "scraping-html-to-markdown" agent skill from https://github.com/hashgraph-online/awesome-codex-plugins/tree/main/plugins/kreuzberg-dev/plugins/plugins/kreuzcrawl/skills/scraping-html-to-markdown into .agents/skills/scraping-html-to-markdown/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "scraping-html-to-markdown", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add hashgraph-online/awesome-codex-plugins --skill scraping-html-to-markdown -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install hashgraph-online/awesome-codex-plugins scraping-html-to-markdown --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/hashgraph-online/awesome-codex-plugins.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/plugins/kreuzberg-dev/plugins/plugins/kreuzcrawl/skills/scraping-html-to-markdown .cursor/skills/scraping-html-to-markdown && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "scraping-html-to-markdown" agent skill from https://github.com/hashgraph-online/awesome-codex-plugins/tree/main/plugins/kreuzberg-dev/plugins/plugins/kreuzcrawl/skills/scraping-html-to-markdown into .cursor/skills/scraping-html-to-markdown/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "scraping-html-to-markdown", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/hashgraph-online/awesome-codex-plugins.git --path plugins/kreuzberg-dev/plugins/plugins/kreuzcrawl/skills/scraping-html-to-markdown--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add hashgraph-online/awesome-codex-plugins --skill scraping-html-to-markdown -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install hashgraph-online/awesome-codex-plugins scraping-html-to-markdown --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/hashgraph-online/awesome-codex-plugins.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/plugins/kreuzberg-dev/plugins/plugins/kreuzcrawl/skills/scraping-html-to-markdown .gemini/skills/scraping-html-to-markdown && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "scraping-html-to-markdown" agent skill from https://github.com/hashgraph-online/awesome-codex-plugins/tree/main/plugins/kreuzberg-dev/plugins/plugins/kreuzcrawl/skills/scraping-html-to-markdown into .gemini/skills/scraping-html-to-markdown/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "scraping-html-to-markdown", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install hashgraph-online/awesome-codex-plugins scraping-html-to-markdownInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add hashgraph-online/awesome-codex-plugins --skill scraping-html-to-markdown -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/hashgraph-online/awesome-codex-plugins.git skills-src && mkdir -p .github/skills && cp -r skills-src/plugins/kreuzberg-dev/plugins/plugins/kreuzcrawl/skills/scraping-html-to-markdown .github/skills/scraping-html-to-markdown && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "scraping-html-to-markdown" agent skill from https://github.com/hashgraph-online/awesome-codex-plugins/tree/main/plugins/kreuzberg-dev/plugins/plugins/kreuzcrawl/skills/scraping-html-to-markdown into .github/skills/scraping-html-to-markdown/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "scraping-html-to-markdown", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add hashgraph-online/awesome-codex-plugins --skill scraping-html-to-markdown -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install hashgraph-online/awesome-codex-plugins scraping-html-to-markdown --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/hashgraph-online/awesome-codex-plugins.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/plugins/kreuzberg-dev/plugins/plugins/kreuzcrawl/skills/scraping-html-to-markdown .opencode/skills/scraping-html-to-markdown && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "scraping-html-to-markdown" agent skill from https://github.com/hashgraph-online/awesome-codex-plugins/tree/main/plugins/kreuzberg-dev/plugins/plugins/kreuzcrawl/skills/scraping-html-to-markdown into .opencode/skills/scraping-html-to-markdown/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "scraping-html-to-markdown", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
Facts
- Skill name
scraping-html-to-markdown- GitHub stars
- 1.3k
- Token cost
- ~1.1k tokens
- SKILL.md length
- 400 words
- Files
- 1
- Skills in repo
- 714
- Repo updated
- First seen
- Licence
- Apache-2.0
At a glance
A skill your agent uses when the user wants a single page rendered as clean Markdown plus structured metadata.
- The user wants a single page rendered as clean Markdown plus structured metadata
- SKILL.md covers Quick recipe, Flag surface, Output shape and Common pitfalls, plus 2 more sections
- Calls jq
- Tasks that involve Web scraping
What it does
Scraping HTML To Markdown is an agent skill from hashgraph-online/awesome-codex-plugins. Use when the user wants a single page rendered as clean Markdown plus structured metadata. Covers kreuzcrawl scrape <url, JSON vs Markdown output, what metadata is returned, and how to handle JS-heavy pages.
Its SKILL.md is about 1.1k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.
It sits in Data & Analytics, covering Web scraping and Markdown. The repository describes itself as: A curated list of awesome OpenAI Codex / ChatGPT plugins, skills, and resources. The 1 Codex Marketplace. See live plugins at: https://hol.org/plugins/best-codex-plugins. The licence is Apache-2.0.
When your agent uses it
- The user wants a single page rendered as clean Markdown plus structured metadata
- Tasks that involve Web scraping
- Tasks that involve Markdown
Example prompts
- “/scraping-html-to-markdown”
What it can do on your machine
Read from SKILL.md and the folder at commit 9e7b281. It shows what the files ask for, not the result of running them.
Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
Shell commands in SKILL.md call:
jq
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Context cost
Scraping HTML To Markdown loads about 1.1k tokens when it runs. Until then it costs about 59 tokens; SKILL.md has 400 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
Safety
Auto-check passed
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.
SKILL.md
The full file from hashgraph-online/awesome-codex-plugins at commit 9e7b281, republished under its Apache-2.0 licence (© hashgraph-online). 400 words, ~1,109 tokens.
Download SKILL.mdSave it as
.claude/skills/scraping-html-to-markdown/SKILL.md (or your agent's skills folder).- name
- scraping-html-to-markdown
- description
- Use when the user wants a single page rendered as clean Markdown plus structured metadata. Covers `kreuzcrawl scrape <url>`, JSON vs Markdown output, what metadata is returned, and how to handle JS-heavy pages.
Scraping HTML to Markdown
kreuzcrawl scrape <url> is the right tool when the user has a single
page in mind. It returns Markdown plus a full structured payload (metadata,
links, images, JSON-LD, HTTP response info).
Quick recipe
kreuzcrawl scrape https://example.com/article --format markdownJSON form (default) when downstream needs metadata:
kreuzcrawl scrape https://example.com/article --format jsonFlag surface
| Flag | Default | Purpose |
|---|---|---|
--format | json | json or markdown. |
--timeout | 30000 | Per-request timeout in ms. |
--proxy | — | HTTP, HTTPS, or SOCKS5 proxy URL. |
--user-agent | — | Override request UA. |
--respect-robots-txt | off | Honour robots.txt. |
--browser-mode | auto | auto, always, never — see headless-fallback skill. |
--browser-endpoint | — | External CDP ws:// URL. |
--config | — | Inline JSON or @file.json for full CrawlConfig. |
Output shape
Markdown mode
Prints the rendered Markdown only. Use when piping to a file the user will read, or when the result becomes LLM context downstream.
JSON mode
Top-level PageResult with:
url,final_url(after redirects),status_code.markdown:{ content, fit_content, warnings }—fit_contentis a pruned LLM-optimised variant.metadata: Open Graph, Twitter Card, Dublin Core, article tags, JSON-LD, headings (H1–H6), feeds, favicons, hreflang.links: arrays forInternal,External,Anchor, andDocument.images:<img>,<picture>,srcset,og:image.tables: structured table data preserved separately from Markdown.response: HTTP headers, content type, charset, body size.
Read result.markdown.content for the Markdown string when scripting.
Common pitfalls
Empty or stub content
Static fetch returned a JS shell. Symptoms in JSON output:
markdown.contentis short or only contains nav/footer chrome.markdown.warningsmentions JS-render-required.metadata.headingsis empty when the page clearly has headings.
Re-run with --browser-mode always and see the headless-fallback skill.
Show full SKILL.md (160 more words)Show less
WAF block
Auto mode detects 8 WAF vendors and retries through headless Chrome
automatically. If you forced --browser-mode never, the WAF response will
fall through. Check response.status_code — 403/406/503 with WAF headers
(server: cloudflare, x-amz-cf-id, etc.) is the giveaway.
Robots.txt blocking the fetch
If --respect-robots-txt is set and the path is disallowed, the scrape
returns an error rather than partial content. Drop the flag only on hosts
you own or have authorisation for.
Wrong charset
Most pages declare UTF-8. Pages that lie about their charset surface as
mojibake in markdown.content. Override via --config '{"force_encoding":"latin-1"}'
or similar.
Examples
Scrape an article for downstream LLM context
kreuzcrawl scrape https://blog.example.com/post-123 --format markdown \
> /tmp/article.mdScrape with proxy and custom UA
kreuzcrawl scrape https://example.com \
--proxy http://proxy.internal:3128 \
--user-agent "kreuzcrawl (research@example.com)" \
--format jsonExtract just the OG metadata
kreuzcrawl scrape https://example.com --format json \
| jq '.metadata | {title: .og.title, description: .og.description, image: .og.image}'When to reach for crawl or interact instead
- The user wants the whole site, not one page →
crawling-a-siteskill. - The user needs to click, type, or scroll before extracting → use
kreuzcrawl interactwith the action list. - The user only wants the list of URLs →
kreuzcrawl map.
© hashgraph-online, Apache-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
Files
Just SKILL.md in plugins/kreuzberg-dev/plugins/plugins/kreuzcrawl/skills/scraping-html-to-markdown of hashgraph-online/awesome-codex-plugins.
Open the folder on GitHubat commit 9e7b281
Compare with similar skills
Scraping HTML To Markdown next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Scraping HTML To Markdown this skillhashgraph-online/awesome-codex-plugins | 1.3k | — | ~1.1k | Automated safety check: Pass | Apache-2.0 | |
| Twitter Crawlerhuangserva/servasyy_skills | 167 | — | ~672 | Automated safety check: Notes | None | |
| 9Router Web Fetchdecolua/9router | 30k | — | ~941 | Automated safety check: Pass | MIT | |
| Optimize Button Patternsduckduckgo/tracker-radar-collector | 169 | — | ~1.6k | Automated safety check: Pass | Custom licence | |
| Meowhub Browserzhaojiaqi/MeowHub | 111 | — | ~1.6k | Automated safety check: Pass | GPL-3.0 | |
| Fetch URL As MarkdownCodeAlive-AI/ai-driven-development | 158 | — | ~906 | Automated safety check: Pass | MIT |
Similar skills
9Router Web Fetch
decolua/9router
Fetches a web page through a 9Router server's web fetch endpoint and returns it as markdown, plain text or HTML, using one of several extraction providers.
Optimize Button Patterns
duckduckgo/tracker-radar-collector
Iteratively improves cookie-popup button regex patterns in button-patterns.js against labelled-button-texts.csv.
Meowhub Browser
zhaojiaqi/MeowHub
Browse the web using Browserless.io cloud browser service. An agent skill from zhaojiaqi/MeowHub.
Fetch URL As Markdown
CodeAlive-AI/ai-driven-development
Fetch a web page (URL) and return clean Markdown via local trafilatura, with Exa MCP as a fallback for JS-rendered or anti-bot pages.
Firecrawl
Prism-Shadow/penguin-harness
Search the web and scrape pages into clean markdown with the Firecrawl API — query-based discovery, single-URL extraction including public PDFs, driven by curl with a vault-stored API key.
More from hashgraph-online/awesome-codex-plugins
All 714 skills in this repoAnime Reaction Gif
hashgraph-online/awesome-codex-plugins
Create original anime-style reaction stickers as looping GIFs and MP4 previews, using generated character pose sheets and timed key poses.
Calibredb
hashgraph-online/awesome-codex-plugins
Manage and query Calibre libraries with the calibredb CLI (local paths or Calibre Content server URLs).
Rust API Test Harness
hashgraph-online/awesome-codex-plugins
A skill your agent uses when adding, changing, testing, or debugging Rust HTTP APIs and services, especially when Codex needs black-box integration tests, random-port app startup, real database test…
Art
hashgraph-online/awesome-codex-plugins
Make a studio's game look like something at build time — a cover from a real frame of the game (free), painted covers, backdrops, textures and character plates from image models through the…
Calle
hashgraph-online/awesome-codex-plugins
Use CALL-E from Codex through the calle CLI. An agent skill from hashgraph-online/awesome-codex-plugins.
Game Balance Economy
hashgraph-online/awesome-codex-plugins
Balance game difficulty, resources, rewards, probability, progression, economies, and dominant strategies.
Related
Categories
Questions about Scraping HTML To Markdown
What does Scraping HTML To Markdown do?
A skill your agent uses when the user wants a single page rendered as clean Markdown plus structured metadata. Scraping HTML To Markdown is an agent skill from hashgraph-online/awesome-codex-plugins. Use when the user wants a single page rendered as clean Markdown plus structured metadata.
When should I use Scraping HTML To Markdown?
Scraping HTML To Markdown fits situations like: the user wants a single page rendered as clean Markdown plus structured metadata; tasks that involve Web scraping; tasks that involve Markdown.
How do I install Scraping HTML To Markdown in Claude Code?
Run `npx skills add hashgraph-online/awesome-codex-plugins --skill scraping-html-to-markdown -a claude-code`. Or copy the skill folder (plugins/kreuzberg-dev/plugins/plugins/kreuzcrawl/skills/scraping-html-to-markdown in hashgraph-online/awesome-codex-plugins) into .claude/skills/scraping-html-to-markdown in your project. Claude Code loads it when a task matches its description.
How do I install Scraping HTML To Markdown in Codex?
Run `npx skills add hashgraph-online/awesome-codex-plugins --skill scraping-html-to-markdown -a codex`. Or copy the skill folder (plugins/kreuzberg-dev/plugins/plugins/kreuzcrawl/skills/scraping-html-to-markdown in hashgraph-online/awesome-codex-plugins) into .agents/skills/scraping-html-to-markdown in your project. Codex loads it when a task matches its description.
Can I use Scraping HTML To Markdown in Cursor, Gemini CLI or GitHub Copilot?
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add hashgraph-online/awesome-codex-plugins --skill scraping-html-to-markdown -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/scraping-html-to-markdown, .gemini/skills/scraping-html-to-markdown, .github/skills/scraping-html-to-markdown and .opencode/skills/scraping-html-to-markdown in your project.
What does Scraping HTML To Markdown need to run?
Going by SKILL.md and its folder, Scraping HTML To Markdown needs the command-line tools its instructions call (jq).
Does Scraping HTML To Markdown access the network?
SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.
Is Scraping HTML To Markdown safe to install?
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.
What licence does Scraping HTML To Markdown use?
Scraping HTML To Markdown is published under the Apache-2.0 licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.
How many tokens does Scraping HTML To Markdown use?
About 1.1k tokens (SKILL.md is roughly 4.4k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.
What are the alternatives to Scraping HTML To Markdown?
Skills that share tags, products or a category with Scraping HTML To Markdown: Twitter Crawler (huangserva/servasyy_skills, 167 stars), 9Router Web Fetch (decolua/9router, 30k stars), Optimize Button Patterns (duckduckgo/tracker-radar-collector, 169 stars) and Meowhub Browser (zhaojiaqi/MeowHub, 111 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Who maintains Scraping HTML To Markdown?
hashgraph-online (a GitHub organization) maintains it in hashgraph-online/awesome-codex-plugins, which has 1,255 GitHub stars. The repository holds 714 skills in this directory. The repository was last updated on October 9, 2026.
Source: hashgraph-online/awesome-codex-plugins on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.