Data Scraper Agent
affaan-m/ECC
Build a fully automated AI-powered data collection agent for any public source — job boards, prices, news, GitHub, sports, anything.
Configure CI/CD pipelines for Apify Actor builds and deployments.
$ npx skills add jeremylongshore/tons-of-skills-marketplace --skill apify-ci-integration -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install jeremylongshore/tons-of-skills-marketplace apify-ci-integration --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/jeremylongshore/tons-of-skills-marketplace.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/.curated/apify-ci-integration .claude/skills/apify-ci-integration && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "apify-ci-integration" agent skill from https://github.com/jeremylongshore/tons-of-skills-marketplace/tree/main/skills/.curated/apify-ci-integration into .claude/skills/apify-ci-integration/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "apify-ci-integration", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/jeremylongshore/tons-of-skills-marketplace/tree/main/skills/.curated/apify-ci-integrationType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add jeremylongshore/tons-of-skills-marketplace --skill apify-ci-integration -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install jeremylongshore/tons-of-skills-marketplace apify-ci-integration --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/jeremylongshore/tons-of-skills-marketplace.git skills-src && mkdir -p .agents/skills && cp -r skills-src/skills/.curated/apify-ci-integration .agents/skills/apify-ci-integration && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "apify-ci-integration" agent skill from https://github.com/jeremylongshore/tons-of-skills-marketplace/tree/main/skills/.curated/apify-ci-integration into .agents/skills/apify-ci-integration/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "apify-ci-integration", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add jeremylongshore/tons-of-skills-marketplace --skill apify-ci-integration -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install jeremylongshore/tons-of-skills-marketplace apify-ci-integration --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/jeremylongshore/tons-of-skills-marketplace.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/skills/.curated/apify-ci-integration .cursor/skills/apify-ci-integration && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "apify-ci-integration" agent skill from https://github.com/jeremylongshore/tons-of-skills-marketplace/tree/main/skills/.curated/apify-ci-integration into .cursor/skills/apify-ci-integration/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "apify-ci-integration", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/jeremylongshore/tons-of-skills-marketplace.git --path skills/.curated/apify-ci-integration--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add jeremylongshore/tons-of-skills-marketplace --skill apify-ci-integration -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install jeremylongshore/tons-of-skills-marketplace apify-ci-integration --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/jeremylongshore/tons-of-skills-marketplace.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/skills/.curated/apify-ci-integration .gemini/skills/apify-ci-integration && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "apify-ci-integration" agent skill from https://github.com/jeremylongshore/tons-of-skills-marketplace/tree/main/skills/.curated/apify-ci-integration into .gemini/skills/apify-ci-integration/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "apify-ci-integration", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install jeremylongshore/tons-of-skills-marketplace apify-ci-integrationInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add jeremylongshore/tons-of-skills-marketplace --skill apify-ci-integration -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/jeremylongshore/tons-of-skills-marketplace.git skills-src && mkdir -p .github/skills && cp -r skills-src/skills/.curated/apify-ci-integration .github/skills/apify-ci-integration && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "apify-ci-integration" agent skill from https://github.com/jeremylongshore/tons-of-skills-marketplace/tree/main/skills/.curated/apify-ci-integration into .github/skills/apify-ci-integration/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "apify-ci-integration", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add jeremylongshore/tons-of-skills-marketplace --skill apify-ci-integration -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install jeremylongshore/tons-of-skills-marketplace apify-ci-integration --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/jeremylongshore/tons-of-skills-marketplace.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/skills/.curated/apify-ci-integration .opencode/skills/apify-ci-integration && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "apify-ci-integration" agent skill from https://github.com/jeremylongshore/tons-of-skills-marketplace/tree/main/skills/.curated/apify-ci-integration into .opencode/skills/apify-ci-integration/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "apify-ci-integration", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
apify-ci-integrationConfigure CI/CD pipelines for Apify Actor builds and deployments.
Apify CI Integration is an agent skill from jeremylongshore/tons-of-skills-marketplace. Configure CI/CD pipelines for Apify Actor builds and deployments. Use when automating Actor deployment via GitHub Actions, running integration tests against the live Apify API, or building CI/CD for scrapers that push to the Apify platform. Trigger with "apify CI", "apify GitHub Actions", "apify automated deploy", "CI apify", "apify pipeline", "auto deploy actor".
Its SKILL.md is about 1.7k tokens, which your agent loads only when the skill is triggered. The skill folder holds 3 other files, including reference files (for example `references/integration-tests.md` and `references/workflows.md`). Compatibility notes: Designed for Claude Code
It sits in DevOps & Cloud, covering Web scraping, CI/CD and Deployment. It works with Apify, GitHub Actions and GitHub. The repository describes itself as: Model-agnostic agent-skills platform with a harness-free canonical layer, verified adapters, and the ccpi package manager. Explore at tonsofskills.com. The licence is MIT.
5 steps, taken from the step headings in SKILL.md.
Read from SKILL.md and the folder at commit cfae287. It shows what the files ask for, not the result of running them.
Pre-approves these tools, so the agent can use them without asking each time:
WriteBash(gh:*)Bash(npm:*)From allowed-tools in the SKILL.md frontmatter.
Shell commands in SKILL.md call:
ghnpmdockerFrom the folder's file list and the shell code blocks in SKILL.md.
Hosts in commands or code, which the agent is likely to contact:
api.apify.comAlso links to:
docs.apify.comdocs.github.comFrom URLs in SKILL.md, links to its own repository left out.
Names these keys or tokens, usually read from environment variables:
APIFY_TOKENFrom names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Designed for Claude Code
From compatibility in the SKILL.md frontmatter.
Apify CI Integration loads about 1.7k tokens when it runs, and up to ~3.4k if it reads all its reference files. Until then it costs about 97 tokens; SKILL.md has 588 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.
The full file from jeremylongshore/tons-of-skills-marketplace at commit cfae287, republished under its MIT licence (© jeremylongshore). 588 words, ~1,682 tokens.
.claude/skills/apify-ci-integration/SKILL.md (or your agent's skills folder). This skill also uses 2 other files; get the full folder from GitHub.Automate Apify Actor builds, tests, and deployments using GitHub Actions — test-on-PR, deploy-on-merge, live-API integration testing, and Docker build verification. Three workflow files do the work (test, deploy, verify-build); full copy-paste-ready definitions live in references/workflows.md.
CI authenticates to Apify with a personal API token, never a hard-coded credential:
gh secret set APIFY_TOKEN),
and expose it to a job only via env: APIFY_TOKEN: ${{ secrets.APIFY_TOKEN }}.apify login --token $APIFY_TOKEN; the REST API and
apify-client read it from the APIFY_TOKEN environment variable.APIFY_TOKEN_TEST / APIFY_TOKEN_PROD secrets so integration
runs never touch production data. Get tokens from Apify Console → Settings →
Integrations.# Store Apify token for CI
gh secret set APIFY_TOKEN --body "apify_api_YOUR_CI_TOKEN"
# Optional: separate tokens for test vs production
gh secret set APIFY_TOKEN_TEST --body "apify_api_test_token"
gh secret set APIFY_TOKEN_PROD --body "apify_api_prod_token"Add .github/workflows/apify-test.yml with two jobs — unit-tests on every PR
and integration-tests gated to push (merge) events. The integration job
proves connectivity before running tests:
- name: Verify Apify connection
run: |
curl -sf -H "Authorization: Bearer $APIFY_TOKEN" \
https://api.apify.com/v2/users/me | jq '.data.username'Full two-job workflow: references/workflows.md (Test Workflow).
Add .github/workflows/apify-deploy.yml — triggered on merges that touch
src/**, package.json, or .actor/** (plus workflow_dispatch). It builds,
tests, installs the Apify CLI, apify pushes, then runs a minimal smoke call to
confirm the new build actually starts. Full definition:
references/workflows.md (Deploy Workflow).
Gate live-API tests on the token so they skip cleanly when it is absent:
const SKIP_INTEGRATION = !process.env.APIFY_TOKEN;
describe.skipIf(SKIP_INTEGRATION)('Apify Integration', () => {
it('should authenticate successfully', async () => {
const user = await client.user().get();
expect(user.username).toBeTruthy();
});
});The complete suite (auth, live Actor run, create/delete a named dataset with cleanup) is in references/integration-tests.md.
Add a verify-build.yml that docker builds the Actor image and boots it once
to confirm the entry point loads. Optionally add branch-protection rules that
require the CI contexts before merge. Both blocks:
references/workflows.md (Actor Build Verification).
Applying this skill produces committed CI/CD infrastructure in the repository:
.github/workflows/apify-test.yml — unit tests on every PR, integration tests
on merge to main..github/workflows/apify-deploy.yml — apify push + post-deploy smoke test on
merge, plus manual workflow_dispatch..github/workflows/verify-build.yml — Docker build + entry-point check on PRs.tests/integration/apify.test.ts — token-gated live-API integration suite.APIFY_TOKEN, optional _TEST / _PROD variants)
and, optionally, branch-protection requiring the CI checks to pass.Once merged, every PR runs unit tests, every merge deploys and smoke-tests the
Actor, and a failed deploy surfaces a ::error:: annotation in the run log.
| Issue | Cause | Solution |
|---|---|---|
APIFY_TOKEN not set | Secret not configured | gh secret set APIFY_TOKEN |
| Integration test timeout | Slow Actor run | Increase timeout, use smaller input |
| Docker build fails in CI | Local-only deps | Commit package-lock.json |
apify push fails | Not logged in | Add apify login --token step |
| Flaky integration tests | External service issues | Add retries, use test.retry(2) |
Minimal test-on-PR gate — the smallest useful setup is Step 1 (store the
secret) plus the unit-tests job from the test workflow. Every PR then runs
npm ci && npm run build && npm test before it can merge.
Full deploy pipeline — add the deploy workflow so a merge to main that
touches src/** builds, tests, runs apify push, then smoke-tests the new
build:
- name: Push Actor to Apify
run: apify push
- name: Verify deployment
run: |
ACTOR_ID=$(jq -r '.name' .actor/actor.json)
apify actors call $ACTOR_ID \
--input='{"startUrls":[{"url":"https://example.com"}],"maxItems":1}' \
--timeout=120apify-client app (not Actor dev) — for an app that calls Actors rather than
publishing one, mock apify-client in unit tests and gate a real-token
integration job to main. Full workflow:
references/workflows.md (CI Configuration for apify-client Apps).
See references/workflows.md for every full workflow and references/integration-tests.md for the complete integration suite.
For runtime deployment patterns beyond CI wiring — release channels, versioned
Actor builds, and rollback — see the apify-deploy-integration skill in this
pack.
© jeremylongshore, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
SKILL.md and 2 other files (references) in skills/.curated/apify-ci-integration of jeremylongshore/tons-of-skills-marketplace.
Open the folder on GitHubat commit cfae287
Apify CI Integration next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Apify CI Integration this skilljeremylongshore/tons-of-skills-marketplace | 2.8k | — | ~1.7k | Automated safety check: Pass | MIT | |
| Data Scraper Agentaffaan-m/ECC | 277k | 5 repos | ~6.3k | Automated safety check: Notes | MIT | |
| Data Scraper Agentaffaan-m/ECC | 276k | 1 repos | ~5.2k | Automated safety check: Notes | MIT | |
| Data Scraper Agentaffaan-m/ECC | 276k | — | ~377 | Automated safety check: Pass | MIT | |
| AI News RadarLearnPrompt/ai-news-radar | 1.8k | — | ~2.5k | Automated safety check: Notes | MIT | |
| Use Vercel Actionamondnet/vercel-action | 764 | — | ~2.7k | Automated safety check: Pass | MIT |
affaan-m/ECC
Build a fully automated AI-powered data collection agent for any public source — job boards, prices, news, GitHub, sports, anything.
affaan-m/ECC
构建一个全自动化的AI驱动数据收集代理,适用于任何公共来源——招聘网站、价格信息、新闻、GitHub、体育赛事等任何内容。按计划进行抓取,使用免费LLM(Gemini Flash)丰富数据,将结果存储在Notion/Sheets/Supabase中,并从用户反馈中学习。完全免费在GitHub Actions上运行。适用于用户希望自动监控、收集或跟踪任何公共数据的场景。
affaan-m/ECC
任意のパブリックソース(ジョブボード、価格、ニュース、GitHub、スポーツなど)用の完全自動化されたAI搭載データ収集エージェントを構築します。スケジュールでスクレイプし、無料LLM(Gemini Flash)でデータを豊かにし、Notion/Sheets/Supabaseに結果を保存し、ユーザーフィードバックから学習します。GitHub…
LearnPrompt/ai-news-radar
A skill your agent uses when working on AI News Radar, 24 小时 AI 更新雷达, AI 更新雷达, 伯乐Skill, or Scout Skill: finding high-signal AI/tech sources, adding RSS/OPML/GitHub feeds, checking source health…
amondnet/vercel-action
Wire amondnet/vercel-action into a GitHub Actions workflow to deploy Vercel projects from CI.
LubomirGeorgiev/cloudflare-workers-nextjs-saas-template
Source-of-truth runbook for preparing this Vinext Cloudflare Workers SaaS template for production deployment.
jeremylongshore/tons-of-skills-marketplace
Execute this skill enables AI assistant to conduct a security-focused code review using the security-agent plugin.
jeremylongshore/tons-of-skills-marketplace
Build this skill automates the adaptation of pre-trained machine learning models using transfer learning techniques.
jeremylongshore/tons-of-skills-marketplace
Execute proactive auto-loading: automatically detects and loads agents.md files.
jeremylongshore/tons-of-skills-marketplace
Aggregate and centralize performance metrics from applications, systems, databases, caches, and services.
jeremylongshore/tons-of-skills-marketplace
Execute this skill enables AI assistant to analyze capacity requirements and plan for future growth.
jeremylongshore/tons-of-skills-marketplace
Process use when you need to work with database indexing. An agent skill from jeremylongshore/tons-of-skills-marketplace.
Works with
Configure CI/CD pipelines for Apify Actor builds and deployments. Apify CI Integration is an agent skill from jeremylongshore/tons-of-skills-marketplace. Configure CI/CD pipelines for Apify Actor builds and deployments.
Apify CI Integration fits situations like: automating Actor deployment via GitHub Actions; running integration tests against the live Apify API; building CI/CD for scrapers that push to the Apify platform; apify GitHub Actions.
Run `npx skills add jeremylongshore/tons-of-skills-marketplace --skill apify-ci-integration -a claude-code`. Or copy the skill folder (skills/.curated/apify-ci-integration in jeremylongshore/tons-of-skills-marketplace) into .claude/skills/apify-ci-integration in your project. Claude Code loads it when a task matches its description.
Run `npx skills add jeremylongshore/tons-of-skills-marketplace --skill apify-ci-integration -a codex`. Or copy the skill folder (skills/.curated/apify-ci-integration in jeremylongshore/tons-of-skills-marketplace) into .agents/skills/apify-ci-integration in your project. Codex loads it when a task matches its description.
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add jeremylongshore/tons-of-skills-marketplace --skill apify-ci-integration -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/apify-ci-integration, .gemini/skills/apify-ci-integration, .github/skills/apify-ci-integration and .opencode/skills/apify-ci-integration in your project.
Going by SKILL.md and its folder, Apify CI Integration needs the command-line tools its instructions call (gh, npm and docker) and credentials named APIFY_TOKEN. Our summary lists: Docker; A credential in APIFY_TOKEN. Its frontmatter pre-approves these tools: Write, Bash(gh:*), Bash(npm:*). Compatibility (from SKILL.md): Designed for Claude Code.
SKILL.md names 3 domains. In commands or code: api.apify.com; the agent is likely to contact it when it follows the instructions. As links in the text: docs.apify.com and docs.github.com. This is read from the text; nothing was executed.
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.
Apify CI Integration is published under the MIT licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.
About 1.7k tokens (SKILL.md is roughly 6.7k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 1.7k tokens, read only when the agent opens those files.
Skills that share tags, products or a category with Apify CI Integration: Data Scraper Agent (affaan-m/ECC, 277k stars), Data Scraper Agent (affaan-m/ECC, 276k stars), Data Scraper Agent (affaan-m/ECC, 276k stars) and AI News Radar (LearnPrompt/ai-news-radar, 1.8k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
jeremylongshore (a GitHub user) maintains it in jeremylongshore/tons-of-skills-marketplace, which has 2,827 GitHub stars. The repository holds 3,342 skills in this directory. The repository was last updated on October 10, 2026.
Source: jeremylongshore/tons-of-skills-marketplace on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.