Crawl4AI Web Scraping
smallnest/goclaw
Scrapes sites, handles JavaScript-heavy pages and extracts structured data with Crawl4AI, through its crwl CLI or Python SDK, including schema-based extraction without an LLM.
Agent skill
by jeremylongshore in jeremylongshore/tons-of-skills-marketplace
Design a production Foundry architecture spanning data pipelines, Ontology, OSDK or Platform APIs, deployment, governance, and operations.
$ npx skills add jeremylongshore/tons-of-skills-marketplace --skill palantir-reference-architecture -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install jeremylongshore/tons-of-skills-marketplace palantir-reference-architecture --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/jeremylongshore/tons-of-skills-marketplace.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/.curated/palantir-reference-architecture .claude/skills/palantir-reference-architecture && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "palantir-reference-architecture" agent skill from https://github.com/jeremylongshore/tons-of-skills-marketplace/tree/main/skills/.curated/palantir-reference-architecture into .claude/skills/palantir-reference-architecture/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "palantir-reference-architecture", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/jeremylongshore/tons-of-skills-marketplace/tree/main/skills/.curated/palantir-reference-architectureType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add jeremylongshore/tons-of-skills-marketplace --skill palantir-reference-architecture -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install jeremylongshore/tons-of-skills-marketplace palantir-reference-architecture --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/jeremylongshore/tons-of-skills-marketplace.git skills-src && mkdir -p .agents/skills && cp -r skills-src/skills/.curated/palantir-reference-architecture .agents/skills/palantir-reference-architecture && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "palantir-reference-architecture" agent skill from https://github.com/jeremylongshore/tons-of-skills-marketplace/tree/main/skills/.curated/palantir-reference-architecture into .agents/skills/palantir-reference-architecture/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "palantir-reference-architecture", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add jeremylongshore/tons-of-skills-marketplace --skill palantir-reference-architecture -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install jeremylongshore/tons-of-skills-marketplace palantir-reference-architecture --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/jeremylongshore/tons-of-skills-marketplace.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/skills/.curated/palantir-reference-architecture .cursor/skills/palantir-reference-architecture && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "palantir-reference-architecture" agent skill from https://github.com/jeremylongshore/tons-of-skills-marketplace/tree/main/skills/.curated/palantir-reference-architecture into .cursor/skills/palantir-reference-architecture/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "palantir-reference-architecture", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/jeremylongshore/tons-of-skills-marketplace.git --path skills/.curated/palantir-reference-architecture--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add jeremylongshore/tons-of-skills-marketplace --skill palantir-reference-architecture -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install jeremylongshore/tons-of-skills-marketplace palantir-reference-architecture --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/jeremylongshore/tons-of-skills-marketplace.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/skills/.curated/palantir-reference-architecture .gemini/skills/palantir-reference-architecture && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "palantir-reference-architecture" agent skill from https://github.com/jeremylongshore/tons-of-skills-marketplace/tree/main/skills/.curated/palantir-reference-architecture into .gemini/skills/palantir-reference-architecture/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "palantir-reference-architecture", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install jeremylongshore/tons-of-skills-marketplace palantir-reference-architectureInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add jeremylongshore/tons-of-skills-marketplace --skill palantir-reference-architecture -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/jeremylongshore/tons-of-skills-marketplace.git skills-src && mkdir -p .github/skills && cp -r skills-src/skills/.curated/palantir-reference-architecture .github/skills/palantir-reference-architecture && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "palantir-reference-architecture" agent skill from https://github.com/jeremylongshore/tons-of-skills-marketplace/tree/main/skills/.curated/palantir-reference-architecture into .github/skills/palantir-reference-architecture/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "palantir-reference-architecture", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add jeremylongshore/tons-of-skills-marketplace --skill palantir-reference-architecture -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install jeremylongshore/tons-of-skills-marketplace palantir-reference-architecture --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/jeremylongshore/tons-of-skills-marketplace.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/skills/.curated/palantir-reference-architecture .opencode/skills/palantir-reference-architecture && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "palantir-reference-architecture" agent skill from https://github.com/jeremylongshore/tons-of-skills-marketplace/tree/main/skills/.curated/palantir-reference-architecture into .opencode/skills/palantir-reference-architecture/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "palantir-reference-architecture", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
palantir-reference-architectureDesign a production Foundry architecture spanning data pipelines, Ontology, OSDK or Platform APIs, deployment, governance, and operations.
Palantir Reference Architecture is an agent skill from jeremylongshore/tons-of-skills-marketplace. Design a production Foundry architecture spanning data pipelines, Ontology, OSDK or Platform APIs, deployment, governance, and operations. Use when defining a new platform integration or reviewing an existing one. Trigger with "Palantir architecture".
Its SKILL.md is about 1.4k tokens, which your agent loads only when the skill is triggered. The skill folder holds 2 other files, including reference files (for example `references/official-docs.md`). Compatibility notes: Requires current Palantir Foundry documentation and approved access for any live resource, permission, data, build, application, or deployment change
It sits in Data & Analytics, covering Data pipelines and ETL. The repository describes itself as: Model-agnostic agent-skills platform with a harness-free canonical layer, verified adapters, and the ccpi package manager. Explore at tonsofskills.com. The licence is MIT.
5 steps, taken from the first numbered list in SKILL.md.
Read from SKILL.md and the folder at commit cfae287. It shows what the files ask for, not the result of running them.
Pre-approves these tools, so the agent can use them without asking each time:
ReadGlobGrepWriteEditFrom allowed-tools in the SKILL.md frontmatter.
No scripts in the folder and no shell commands in SKILL.md.
From the folder's file list and the shell code blocks in SKILL.md.
No URLs in SKILL.md.
From URLs in SKILL.md, links to its own repository left out.
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Requires current Palantir Foundry documentation and approved access for any live resource, permission, data, build, application, or deployment change
From compatibility in the SKILL.md frontmatter.
Palantir Reference Architecture loads about 1.4k tokens when it runs, and up to ~1.6k if it reads all its reference files. Until then it costs about 71 tokens; SKILL.md has 568 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.
The full file from jeremylongshore/tons-of-skills-marketplace at commit cfae287, republished under its MIT licence (© jeremylongshore). 568 words, ~1,391 tokens.
.claude/skills/palantir-reference-architecture/SKILL.md (or your agent's skills folder). This skill also uses 1 other file; get the full folder from GitHub.Start from business decisions and data authority, then map each responsibility to a Foundry primitive. Keep ingestion, transformation, Ontology semantics, application access, deployment, and telemetry independently governable.
references/official-docs.md and resolve target-enrollment feature availability with platform owners.Define user-delegated and service authentication at each application boundary. Record OAuth grants, principals, scopes, Developer Console restrictions, secret storage, rotation, and revocation without placing credential values in the architecture artifact.
Define capabilities, owners, authoritative data, side effects, latency/freshness objectives, and failure domains.
Place ingestion and transformations into projects with explicit inputs, outputs, lineage, expectations, and compute choices.
Model domain semantics and writeback in the Ontology, including object/property security and Action validation.
Choose generated OSDK or Platform SDK per application boundary and constrain OAuth scopes, restrictions, and principal permissions.
Choose DevOps product, Developer Console application, or Compute Module deployment, then add metrics, governed logs, audit evidence, and rollback.
Domain and data owners approve semantics and writeback; security owners approve boundaries and controls; platform owners approve deployment primitives; operations owners approve service objectives and recovery.
A decision record, context/component/data-flow diagrams, responsibility and authority matrix, data contracts, access model, API/SDK choice, deployment topology, failure analysis, observability design, and staged rollout.
| Condition | Response |
|---|---|
| One service owns ingestion, policy, writeback, and release | Separate responsibilities and failure domains before scaling the design. |
| The architecture depends on handwritten Ontology API names | Anchor it to generated OSDK/application metadata. |
| Granular security is assumed to propagate downstream | Add mandatory controls or redesign downstream handling. |
| No rollback exists for schema or Ontology changes | Stage compatibility and dual-read/write behavior before promotion. |
Design a supplier-risk workflow with connector ingestion, incremental transforms, governed datasets, supplier Ontology objects, validated Actions, a generated OSDK application, and release-managed environments.
Review a container integration by deciding whether it belongs in a Compute Module, mapping OAuth/resource restrictions, data flows, replica behavior, logs, and prior-image rollback.
© jeremylongshore, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
SKILL.md and 1 other file (references) in skills/.curated/palantir-reference-architecture of jeremylongshore/tons-of-skills-marketplace.
Open the folder on GitHubat commit cfae287
Palantir Reference Architecture next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Palantir Reference Architecture this skilljeremylongshore/tons-of-skills-marketplace | 2.8k | — | ~1.4k | Automated safety check: Pass | MIT | |
| Crawl4AI Web Scrapingsmallnest/goclaw | 599 | 1 repos | ~2.5k | Automated safety check: Pass | MIT | |
| Glue 09 10 Migrationaws-samples/aws-glue-samples | 1.5k | — | ~2.4k | Automated safety check: Pass | MIT-0 | |
| Migrate Glue Devendpoint To Interactive Sessionsaws-samples/aws-glue-samples | 1.5k | — | ~3.6k | Automated safety check: Pass | MIT-0 | |
| Dbt Databricks PR Readydatabricks/dbt-databricks | 380 | — | ~2.8k | Automated safety check: Pass | Apache-2.0 | |
| Apache Spark EngineerJeffallan/claude-skills | 12k | 1 repos | ~1.7k | Automated safety check: Pass | MIT |
smallnest/goclaw
Scrapes sites, handles JavaScript-heavy pages and extracts structured data with Crawl4AI, through its crwl CLI or Python SDK, including schema-based extraction without an LLM.
aws-samples/aws-glue-samples
Upgrade an AWS Glue ETL job from Glue version 0.9 or 1.0 to Glue 4.0.
aws-samples/aws-glue-samples
Migrate a legacy AWS Glue development endpoint to a Glue interactive session, following the official AWS migration checklist.
databricks/dbt-databricks
A skill your agent uses for an open dbt-databricks pull request, including your own PR or a fork PR, to assess merge readiness and optionally repair selected gaps on the PR head branch.
Jeffallan/claude-skills
Guides writing and tuning Apache Spark jobs: DataFrame and RDD code, Spark SQL, partitioning, caching, shuffle tuning and structured streaming.
MaterializeInc/materialize
Cut a dbt-materialize PyPI release: bump the version in version.py and setup.py, date the Unreleased CHANGELOG entry, and open the release PR with a Ship: <url body.
jeremylongshore/tons-of-skills-marketplace
Execute this skill enables AI assistant to conduct a security-focused code review using the security-agent plugin.
jeremylongshore/tons-of-skills-marketplace
Build this skill automates the adaptation of pre-trained machine learning models using transfer learning techniques.
jeremylongshore/tons-of-skills-marketplace
Execute proactive auto-loading: automatically detects and loads agents.md files.
jeremylongshore/tons-of-skills-marketplace
Aggregate and centralize performance metrics from applications, systems, databases, caches, and services.
jeremylongshore/tons-of-skills-marketplace
Execute this skill enables AI assistant to analyze capacity requirements and plan for future growth.
jeremylongshore/tons-of-skills-marketplace
Process use when you need to work with database indexing. An agent skill from jeremylongshore/tons-of-skills-marketplace.
Categories
Design a production Foundry architecture spanning data pipelines, Ontology, OSDK or Platform APIs, deployment, governance, and operations. Palantir Reference Architecture is an agent skill from jeremylongshore/tons-of-skills-marketplace. Design a production Foundry architecture spanning data pipelines, Ontology, OSDK or Platform APIs, deployment, governance, and operations.
Palantir Reference Architecture fits situations like: defining a new platform integration; reviewing an existing one; with Palantir architecture.
Run `npx skills add jeremylongshore/tons-of-skills-marketplace --skill palantir-reference-architecture -a claude-code`. Or copy the skill folder (skills/.curated/palantir-reference-architecture in jeremylongshore/tons-of-skills-marketplace) into .claude/skills/palantir-reference-architecture in your project. Claude Code loads it when a task matches its description.
Run `npx skills add jeremylongshore/tons-of-skills-marketplace --skill palantir-reference-architecture -a codex`. Or copy the skill folder (skills/.curated/palantir-reference-architecture in jeremylongshore/tons-of-skills-marketplace) into .agents/skills/palantir-reference-architecture in your project. Codex loads it when a task matches its description.
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add jeremylongshore/tons-of-skills-marketplace --skill palantir-reference-architecture -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/palantir-reference-architecture, .gemini/skills/palantir-reference-architecture, .github/skills/palantir-reference-architecture and .opencode/skills/palantir-reference-architecture in your project.
SKILL.md names no scripts, command-line tools or credentials: Palantir Reference Architecture is instructions for the agent only. Our summary lists: Python 3. Its frontmatter pre-approves these tools: Read, Glob, Grep, Write, Edit. Compatibility (from SKILL.md): Requires current Palantir Foundry documentation and approved access for any live resource, permission, data, build, application, or deployment change.
SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.
Palantir Reference Architecture is published under the MIT licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.
About 1.4k tokens (SKILL.md is roughly 5.6k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 259 tokens, read only when the agent opens those files.
Skills that share tags, products or a category with Palantir Reference Architecture: Crawl4AI Web Scraping (smallnest/goclaw, 599 stars), Glue 09 10 Migration (aws-samples/aws-glue-samples, 1.5k stars), Migrate Glue Devendpoint To Interactive Sessions (aws-samples/aws-glue-samples, 1.5k stars) and Dbt Databricks PR Ready (databricks/dbt-databricks, 380 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
jeremylongshore (a GitHub user) maintains it in jeremylongshore/tons-of-skills-marketplace, which has 2,827 GitHub stars. The repository holds 3,342 skills in this directory. The repository was last updated on October 10, 2026.
Source: jeremylongshore/tons-of-skills-marketplace on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.