New: 98,000+ skills indexed from GitHub repos with 100+ stars · every skill safety-checkedBrowse top skillsTop skills
Esc to closeBrowse all skills
Submit a skillBrowse skills
Submit a skillBrowse skills

Agent skill

Setting Up Log Aggregation

by jeremylongshore in jeremylongshore/tons-of-skills-marketplace

Execute use when setting up log aggregation solutions using ELK, Loki, or Splunk.

MITAuto-check passedDevOps & Cloud

Install Setting Up Log Aggregation

skills CLI
$ npx skills add jeremylongshore/tons-of-skills-marketplace --skill setting-up-log-aggregation -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install jeremylongshore/tons-of-skills-marketplace setting-up-log-aggregation --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/jeremylongshore/tons-of-skills-marketplace.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/.curated/setting-up-log-aggregation .claude/skills/setting-up-log-aggregation && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
setting-up-log-aggregation
GitHub stars
2.8k
Token cost
~1.1k tokens
SKILL.md length
512 words
Files
13 (incl. scripts, references, assets)
Skills in repo
3,342
Repo updated
First seen
Licence
MIT

At a glance

Execute use when setting up log aggregation solutions using ELK, Loki, or Splunk.

  • Works in 9 steps: Select the log aggregation platform: ELK… → Deploy the storage backend:… → Configure log shippers on sources:… → …
  • Setting up log aggregation solutions using ELK
  • SKILL.md covers Overview, Prerequisites, Instructions and Output, plus 3 more sections
  • Runs Python scripts from its folder

What it does

Setting Up Log Aggregation is an agent skill from jeremylongshore/tons-of-skills-marketplace. Execute use when setting up log aggregation solutions using ELK, Loki, or Splunk. Trigger with phrases like "setup log aggregation", "deploy ELK stack", "configure Loki", or "install Splunk". Generates production-ready configurations for data ingestion, processing, storage, and visualization with proper security and scalability.

Its SKILL.md is about 1.1k tokens, which your agent loads only when the skill is triggered. The skill folder holds 15 other files, including scripts, reference files and assets (for example `assets/README.md`, `assets/dashboard_elk.json` and `assets/dashboard_loki.json`). Compatibility notes: Designed for Claude Code

It sits in DevOps & Cloud, covering Observability. It works with Splunk, Elasticsearch and Kubernetes. The repository describes itself as: Model-agnostic agent-skills platform with a harness-free canonical layer, verified adapters, and the ccpi package manager. Explore at tonsofskills.com. The licence is MIT.

When your agent uses it

  • Setting up log aggregation solutions using ELK
  • With phrases like setup log aggregation
  • Deploy ELK stack

Example prompts

  • “setup log aggregation”
  • “deploy ELK stack”
  • “configure Loki”
  • “/setting-up-log-aggregation”

Requirements

  • Python 3
  • Docker
  • Compatibility (from SKILL.md): Designed for Claude Code
  • Pre-approved tools (allowed-tools): Read, Write, Edit, Grep, Glob, Bash(docker:*), Bash(kubectl:*)

Workflow steps

9 steps, taken from the first numbered list in SKILL.md.

  1. Select the log aggregation platform: ELK for full-text search and complex queries, Loki for lightweight Kubernetes-native logging, Splunk…
  2. Deploy the storage backend: Elasticsearch cluster, Loki with object storage (S3/GCS), or Splunk indexers
  3. Configure log shippers on sources: Filebeat for ELK, Promtail for Loki, Fluentd/Fluent Bit for multi-destination
  4. Define parsing rules: Logstash grok patterns for unstructured logs, JSON parsing for structured logs, multiline handling for stack traces
  5. Set retention policies: Index Lifecycle Management (ILM) for Elasticsearch, chunk retention for Loki, index rotation for Splunk
  6. Deploy visualization: Kibana dashboards for ELK, Grafana dashboards for Loki, Splunk Search for Splunk
  7. Configure alerting: define log-based alerts for error spikes, application exceptions, and security events
  8. Implement RBAC: restrict dashboard access and log visibility by team and environment
  9. Test the full pipeline: generate test logs, verify ingestion, confirm parsing, and validate dashboard display

What it can do on your machine

Read from SKILL.md and the folder at commit cfae287. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves these tools, so the agent can use them without asking each time:

    • Read
    • Write
    • Edit
    • Grep
    • Glob
    • Bash(docker:*)
    • Bash(kubectl:*)

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Ships 4 files in scripts/ (Python), which the agent can run.

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    Links to these hosts (documentation or services it may open):

    • grafana.com
    • elastic.co
    • docs.fluentd.org

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

  • Compatibility

    Designed for Claude Code

    From compatibility in the SKILL.md frontmatter.

Context cost

Setting Up Log Aggregation loads about 1.1k tokens when it runs, and up to ~1.2k if it reads all its reference files. Until then it costs about 89 tokens; SKILL.md has 512 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~89
When it runs · the whole SKILL.md, loaded when a task matches
~1.1k
With references · SKILL.md plus every file in references/, read only if the agent opens them
~1.2k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.

SKILL.md

The full file from jeremylongshore/tons-of-skills-marketplace at commit cfae287, republished under its MIT licence (© jeremylongshore). 512 words, ~1,150 tokens.

Download SKILL.mdSave it as .claude/skills/setting-up-log-aggregation/SKILL.md (or your agent's skills folder). This skill also uses 12 other files; get the full folder from GitHub.
name
setting-up-log-aggregation
description
Execute use when setting up log aggregation solutions using ELK, Loki, or Splunk. Trigger with phrases like "setup log aggregation", "deploy ELK stack", "configure Loki", or "install Splunk". Generates production-ready configurations for data ingestion, processing, storage, and visualization with proper security and scalability.
allowed-tools
Read, Write, Edit, Grep, Glob, Bash(docker:*), Bash(kubectl:*)
compatibility
Designed for Claude Code
version
1.26.0
author
Jeremy Longshore <jeremy@intentsolutions.io>
license
MIT
tags
devops, deployment, security, scaling

Setting Up Log Aggregation

Overview

Deploy centralized log aggregation platforms (ELK Stack, Grafana Loki, Splunk) with ingestion pipelines, structured parsing, retention policies, visualization dashboards, and alerting. Configure log shippers (Filebeat, Promtail, Fluentd) to collect from applications, containers, and system logs with proper security and scalability.

Prerequisites

  • Target infrastructure identified: Kubernetes, Docker Compose, or VMs
  • Storage requirements calculated: estimate daily log volume and multiply by retention period
  • Network connectivity between log sources and aggregation platform (typically ports 9200, 3100, 8088)
  • Authentication mechanism defined (LDAP, OAuth, API tokens, or basic auth)
  • Resource allocation planned: Elasticsearch needs significant heap memory (minimum 4GB per node)

Instructions

  1. Select the log aggregation platform: ELK for full-text search and complex queries, Loki for lightweight Kubernetes-native logging, Splunk for enterprise with advanced analytics
  2. Deploy the storage backend: Elasticsearch cluster, Loki with object storage (S3/GCS), or Splunk indexers
  3. Configure log shippers on sources: Filebeat for ELK, Promtail for Loki, Fluentd/Fluent Bit for multi-destination
  4. Define parsing rules: Logstash grok patterns for unstructured logs, JSON parsing for structured logs, multiline handling for stack traces
  5. Set retention policies: Index Lifecycle Management (ILM) for Elasticsearch, chunk retention for Loki, index rotation for Splunk
  6. Deploy visualization: Kibana dashboards for ELK, Grafana dashboards for Loki, Splunk Search for Splunk
  7. Configure alerting: define log-based alerts for error spikes, application exceptions, and security events
  8. Implement RBAC: restrict dashboard access and log visibility by team and environment
  9. Test the full pipeline: generate test logs, verify ingestion, confirm parsing, and validate dashboard display

Output

  • Docker Compose or Kubernetes manifests for the log aggregation stack
  • Log shipper configuration files (Filebeat YAML, Promtail config, Fluentd conf)
  • Parsing and field extraction rules (Logstash pipeline, grok patterns)
  • Retention policy configuration (ILM, lifecycle rules)
  • Dashboard JSON exports for Kibana or Grafana
  • Alert rule definitions for error rate monitoring
Show full SKILL.md (209 more words)Show less

Error Handling

ErrorCauseSolution
Elasticsearch heap space exhaustedJVM heap too small for index volumeIncrease ES_JAVA_OPTS heap size (set to 50% of available RAM, max 32GB) or add nodes
Cannot connect to ElasticsearchNetwork issue or Elasticsearch not startedVerify Elasticsearch is running and healthy; check firewall rules and bind address
Failed to create indexDisk space full or index template misconfiguredCheck disk usage with df -h; review index template settings and shard allocation
Failed to parse log lineGrok pattern mismatch or unexpected log formatTest grok patterns with Kibana Grok Debugger; add fallback pattern for unmatched lines
Promtail: too many open filesSystem file descriptor limit too low for log tailingIncrease ulimit -n to 65536; reduce the number of watched paths

Examples

  • "Deploy an ELK stack on Docker Compose with Filebeat collecting Nginx and application logs, Logstash parsing with grok, and a Kibana dashboard for 5xx error monitoring."
  • "Set up Loki + Promtail on Kubernetes with 14-day retention, basic auth, and a Grafana dashboard showing logs per namespace."
  • "Configure Fluentd to ship logs from 20 application servers to both Elasticsearch (hot storage, 7 days) and S3 (cold storage, 1 year)."

Resources

© jeremylongshore, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 12 other files (scripts, references, assets) in skills/.curated/setting-up-log-aggregation of jeremylongshore/tons-of-skills-marketplace.

  • SKILL.md
  • assets/README.md
  • assets/dashboard_elk.json
  • assets/dashboard_loki.json
  • assets/dashboard_splunk.json
  • assets/example_log_data.json
  • assets/loki_config_template.yaml
  • assets/splunk_config_template.conf
  • references/README.md
  • scripts/README.md
  • scripts/setup_elk.py
  • scripts/setup_loki.py
  • scripts/setup_splunk.py

Open the folder on GitHubat commit cfae287

Compare with similar skills

Setting Up Log Aggregation next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Setting Up Log Aggregation compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Setting Up Log Aggregation this skilljeremylongshore/tons-of-skills-marketplace2.8k—~1.1kAutomated safety check: PassMIT
Observability K8s Investigationelastic/agent-skills592—~8.3kAutomated safety check: PassApache-2.0
Observability Sre Triageelastic/agent-skills592—~7.4kAutomated safety check: PassApache-2.0
Kratos Developmentaide-family/moon253—~1.5kAutomated safety check: PassNone
Implementing Log Forwarding With Fluentdmukul975/Anthropic-Cybersecurity-Skills34k—~697Automated safety check: PassApache-2.0
Kubeshark KFL2 Filter Referencekubeshark/kubeshark12k—~3.6kAutomated safety check: PassApache-2.0

Similar skills

  • Observability K8s Investigation

    elastic/agent-skills

    Official

    Investigate Kubernetes workload, node, and control-plane issues using OTel telemetry (EDOT).

    592 GitHub stars~8.3k tokensUpdated 3 days ago
    DevOps & CloudAuto-check passed

  • Observability Sre Triage

    elastic/agent-skills

    Official

    Triage a degraded or suspect service end to end: read SLO status and burn rate, check active alerting rules and ML anomalies, measure throughput, latency, and error rate, assess dependency health…

    592 GitHub stars~7.4k tokensUpdated 3 days ago
    DevOps & CloudAuto-check passed

  • Kratos Development

    aide-family/moon

    Develops Go microservices with Kratos v2 following official design philosophy, DDD/Clean Architecture layout, Protobuf API, error/config/middleware patterns, and observability.

    253 GitHub stars~1.5k tokensUpdated 3 mo ago
    Backend & APIsAuto-check passed

  • Implementing Log Forwarding With Fluentd

    mukul975/Anthropic-Cybersecurity-Skills

    Configures Fluent Bit as an endpoint log forwarder and Fluentd as the central aggregator for centralized log collection, routing, filtering, and enrichment, covering input plugins for…

    34k GitHub stars~697 tokensUpdated 1 mo ago
    Backend & APIsAuto-check passed

  • Kubeshark KFL2 Filter Reference

    kubeshark/kubeshark

    Syntax reference for KFL2, the CEL-based display filter language used to search Kubernetes network traffic captured by Kubeshark, loaded before any filter is written.

    12k GitHub stars~3.6k tokensUpdated today
    DevOps & CloudAuto-check passed

  • UModel Root Cause Analysis

    alibaba/UnifiedModel

    Investigates a service incident to its root cause by querying a UModel object graph alongside metrics, logs, topology and recent deployments.

    415 GitHub stars~1.9k tokensUpdated 16 days ago
    DevOps & CloudAuto-check passed

More from jeremylongshore/tons-of-skills-marketplace

All 3,342 skills in this repo

  • Performing Security Code Review

    jeremylongshore/tons-of-skills-marketplace

    Execute this skill enables AI assistant to conduct a security-focused code review using the security-agent plugin.

    2.8k GitHub starsUsed in 2 repos~1.3k tokens
    Auto-check: notes

  • Adapting Transfer Learning Models

    jeremylongshore/tons-of-skills-marketplace

    Build this skill automates the adaptation of pre-trained machine learning models using transfer learning techniques.

    2.8k GitHub stars~1.1k tokensUpdated today
    Auto-check passed

  • Agent Context Loader

    jeremylongshore/tons-of-skills-marketplace

    Execute proactive auto-loading: automatically detects and loads agents.md files.

    2.8k GitHub stars~1.1k tokensUpdated today
    Auto-check passed

  • Aggregating Performance Metrics

    jeremylongshore/tons-of-skills-marketplace

    Aggregate and centralize performance metrics from applications, systems, databases, caches, and services.

    2.8k GitHub stars~1.2k tokensUpdated today
    Auto-check passed

  • Analyzing Capacity Planning

    jeremylongshore/tons-of-skills-marketplace

    Execute this skill enables AI assistant to analyze capacity requirements and plan for future growth.

    2.8k GitHub stars~947 tokensUpdated today
    Auto-check passed

  • Analyzing Database Indexes

    jeremylongshore/tons-of-skills-marketplace

    Process use when you need to work with database indexing. An agent skill from jeremylongshore/tons-of-skills-marketplace.

    2.8k GitHub stars~2k tokensUpdated today
    Auto-check passed

Works with

Categories

Questions about Setting Up Log Aggregation

What does Setting Up Log Aggregation do?

Execute use when setting up log aggregation solutions using ELK, Loki, or Splunk. Setting Up Log Aggregation is an agent skill from jeremylongshore/tons-of-skills-marketplace. Execute use when setting up log aggregation solutions using ELK, Loki, or Splunk.

When should I use Setting Up Log Aggregation?

Setting Up Log Aggregation fits situations like: setting up log aggregation solutions using ELK; with phrases like setup log aggregation; deploy ELK stack.

How do I install Setting Up Log Aggregation in Claude Code?

Run `npx skills add jeremylongshore/tons-of-skills-marketplace --skill setting-up-log-aggregation -a claude-code`. Or copy the skill folder (skills/.curated/setting-up-log-aggregation in jeremylongshore/tons-of-skills-marketplace) into .claude/skills/setting-up-log-aggregation in your project. Claude Code loads it when a task matches its description.

How do I install Setting Up Log Aggregation in Codex?

Run `npx skills add jeremylongshore/tons-of-skills-marketplace --skill setting-up-log-aggregation -a codex`. Or copy the skill folder (skills/.curated/setting-up-log-aggregation in jeremylongshore/tons-of-skills-marketplace) into .agents/skills/setting-up-log-aggregation in your project. Codex loads it when a task matches its description.

Can I use Setting Up Log Aggregation in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add jeremylongshore/tons-of-skills-marketplace --skill setting-up-log-aggregation -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/setting-up-log-aggregation, .gemini/skills/setting-up-log-aggregation, .github/skills/setting-up-log-aggregation and .opencode/skills/setting-up-log-aggregation in your project.

What does Setting Up Log Aggregation need to run?

Going by SKILL.md and its folder, Setting Up Log Aggregation needs Python for the scripts in its folder. Our summary lists: Python 3; Docker. Its frontmatter pre-approves these tools: Read, Write, Edit, Grep, Glob, Bash(docker:*), Bash(kubectl:*). Compatibility (from SKILL.md): Designed for Claude Code.

Does Setting Up Log Aggregation access the network?

SKILL.md names 3 domains. As links in the text: grafana.com, elastic.co and docs.fluentd.org. This is read from the text; nothing was executed.

Is Setting Up Log Aggregation safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. The check reads SKILL.md only: the scripts in the folder are not scanned, so read them before running anything.

What licence does Setting Up Log Aggregation use?

Setting Up Log Aggregation is published under the MIT licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Setting Up Log Aggregation use?

About 1.1k tokens (SKILL.md is roughly 4.6k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 16 tokens, read only when the agent opens those files.

What are the alternatives to Setting Up Log Aggregation?

Skills that share tags, products or a category with Setting Up Log Aggregation: Observability K8s Investigation (elastic/agent-skills, 592 stars), Observability Sre Triage (elastic/agent-skills, 592 stars), Kratos Development (aide-family/moon, 253 stars) and Implementing Log Forwarding With Fluentd (mukul975/Anthropic-Cybersecurity-Skills, 34k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Setting Up Log Aggregation?

jeremylongshore (a GitHub user) maintains it in jeremylongshore/tons-of-skills-marketplace, which has 2,827 GitHub stars. The repository holds 3,342 skills in this directory. The repository was last updated on October 10, 2026.

Source: jeremylongshore/tons-of-skills-marketplace on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.