New: 98,000+ skills indexed from GitHub repos with 100+ stars · every skill safety-checkedBrowse top skillsTop skills
Esc to closeBrowse all skills
Submit a skillBrowse skills
Submit a skillBrowse skills

Agent skill

Clickhouse System Log Disk Exhaustion

by divinevideo in divinevideo/divine-mobile

Fix ClickHouse "Code: 243 Cannot reserve 1.00 MiB, not enough space" errors caused by system log tables (textlog, tracelog, processorsprofilelog, querylog) filling the disk.

MPL-2.0Auto-check passedDatabases

Install Clickhouse System Log Disk Exhaustion

skills CLI
$ npx skills add divinevideo/divine-mobile --skill clickhouse-system-log-disk-exhaustion -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install divinevideo/divine-mobile clickhouse-system-log-disk-exhaustion --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/divinevideo/divine-mobile.git skills-src && mkdir -p .claude/skills && cp -r skills-src/.agents/skills/clickhouse-system-log-disk-exhaustion .claude/skills/clickhouse-system-log-disk-exhaustion && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
clickhouse-system-log-disk-exhaustion
GitHub stars
266
Token cost
~2k tokens
SKILL.md length
468 words
Files
1
Skills in repo
103
Repo updated
First seen
Licence
MPL-2.0

At a glance

Fix ClickHouse "Code: 243 Cannot reserve 1.00 MiB, not enough space" errors caused by system log tables (textlog, tracelog, processorsprofilelog, querylog) filling the disk.

  • Works in 3 steps: ClickHouse Cloud Console… → Get the default user password from the… → ClickHouse Cloud Support: Request system…
  • ClickHouse inserts fail with NOTENOUGHSPACE error
  • SKILL.md covers Problem, Context / Trigger Conditions, Solution and Verification, plus 2 more sections
  • Calls kubectl

What it does

Clickhouse System Log Disk Exhaustion is an agent skill from divinevideo/divine-mobile. Fix ClickHouse "Code: 243 Cannot reserve 1.00 MiB, not enough space" errors caused by system log tables (textlog, tracelog, processorsprofilelog, querylog) filling the disk. Use when: (1) ClickHouse inserts fail with NOTENOUGHSPACE error, (2) Disk is 100% full but application tables are small, (3) system database is 10-100x larger than user databases, (4) Sentry shows batch insert/commit failures across multiple tables simultaneously. Covers both self-hosted ClickHouse (Altinity operator on K8s) and ClickHouse…

Its SKILL.md is about 2k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

It sits in Databases, covering Data warehousing and Container orchestration. It works with ClickHouse, Kubernetes and Sentry. The licence is MPL-2.0.

When your agent uses it

  • ClickHouse inserts fail with NOTENOUGHSPACE error
  • Disk is 100% full but application tables are small
  • System database is 10-100x larger than user databases
  • Sentry shows batch insert/commit failures across multiple tables simultaneously

Example prompts

  • “Code: 243 Cannot reserve 1.00 MiB, not enough space”
  • “/clickhouse-system-log-disk-exhaustion”

Workflow steps

3 steps, taken from the first numbered list in SKILL.md.

  1. ClickHouse Cloud Console (https://clickhouse.cloud/): Go to service settings and
  2. Get the default user password from the console, then run the ALTER TTL commands
  3. ClickHouse Cloud Support: Request system log cleanup

What it can do on your machine

Read from SKILL.md and the folder at commit c3d6f7e. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Shell commands in SKILL.md call:

    • kubectl

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    Links to these hosts (documentation or services it may open):

    • clickhouse.com
    • clickhouse.cloud

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Clickhouse System Log Disk Exhaustion loads about 2k tokens when it runs. Until then it costs about 152 tokens; SKILL.md has 468 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~152
When it runs · the whole SKILL.md, loaded when a task matches
~2k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from divinevideo/divine-mobile at commit c3d6f7e, republished under its MPL-2.0 licence (© divinevideo). 468 words, ~1,971 tokens.

Download SKILL.mdSave it as .claude/skills/clickhouse-system-log-disk-exhaustion/SKILL.md (or your agent's skills folder).
name
clickhouse-system-log-disk-exhaustion
description
Fix ClickHouse "Code: 243 Cannot reserve 1.00 MiB, not enough space" errors caused by system log tables (text_log, trace_log, processors_profile_log, query_log) filling the disk. Use when: (1) ClickHouse inserts fail with NOT_ENOUGH_SPACE error, (2) Disk is 100% full but application tables are small, (3) system database is 10-100x larger than user databases, (4) Sentry shows batch insert/commit failures across multiple tables simultaneously. Covers both self-hosted ClickHouse (Altinity operator on K8s) and ClickHouse Cloud with different remediation paths for each.
author
Claude Code
version
1.0.0
date
2026-03-01

ClickHouse System Log Disk Exhaustion

Problem

ClickHouse internal system log tables (text_log, trace_log, processors_profile_log, query_log, metric_log, asynchronous_metric_log) grow unbounded with a default TTL of 180 days, eventually filling the entire disk. This causes all INSERT operations to fail with Code: 243 - Cannot reserve 1.00 MiB, not enough space, cascading across all application tables simultaneously.

Context / Trigger Conditions

  • Error message: Code: 243. DB::Exception: Cannot reserve 1.00 MiB, not enough space. (NOT_ENOUGH_SPACE)
  • Symptoms: All writes fail simultaneously across multiple tables; reads may still work
  • Sentry pattern: Multiple batch insert/commit failure issues appearing at the same time
  • Diagnosis query: SELECT database, formatReadableSize(sum(total_bytes)) FROM system.tables WHERE total_bytes > 0 GROUP BY database ORDER BY sum(total_bytes) DESC
    • If system database is 10x+ larger than application databases, this is the cause
  • ClickHouse Cloud additional symptom: Numbered suffix tables (trace_log_16, text_log_19) from decommissioned server nodes accumulate and never get cleaned up within the TTL window

Solution

Diagnosis
sql
-- Check database sizes
SELECT database, formatReadableSize(sum(total_bytes)) as size
FROM system.tables WHERE total_bytes > 0
GROUP BY database ORDER BY sum(total_bytes) DESC;

-- Find biggest system tables
SELECT name, formatReadableSize(total_bytes) as size
FROM system.tables
WHERE database = 'system' AND total_bytes > 100000000
ORDER BY total_bytes DESC LIMIT 20;

-- Check disk usage
SELECT name, formatReadableSize(free_space) as free, formatReadableSize(total_space) as total
FROM system.disks WHERE name = 'default';
Fix: Self-Hosted ClickHouse (kubectl access)

Step 1: If disk is 100% full (TRUNCATE itself fails with NOT_ENOUGH_SPACE)

TRUNCATE needs some temporary disk space. When disk is truly 100% full, you must free space at the filesystem level first:

bash
# Find and remove orphaned _N suffix tables (from old replicas)
kubectl exec $CH_POD -- du -sh /var/lib/clickhouse/data/system/*_0/

# Detach them first (important!), then remove data
kubectl exec $CH_POD -- clickhouse-client --query "DETACH TABLE system.text_log_0 PERMANENTLY"
kubectl exec $CH_POD -- bash -c "rm -rf /var/lib/clickhouse/data/system/text_log_0/"
# Repeat for other _N tables until you have ~100MB+ free

Step 2: Drop partitions or truncate tables

System log tables are partitioned by toYYYYMM(event_date). Drop older partitions first (smaller operations), then truncate:

sql
-- Check partitions
SELECT partition, formatReadableSize(sum(bytes_on_disk)) as size
FROM system.parts
WHERE database = 'system' AND table = 'text_log' AND active
GROUP BY partition ORDER BY partition;

-- Drop old partitions one at a time
ALTER TABLE system.text_log DROP PARTITION 202601 SETTINGS max_partition_size_to_drop = 0;
ALTER TABLE system.text_log DROP PARTITION 202602 SETTINGS max_partition_size_to_drop = 0;

-- Or truncate entire tables (needs max_table_size_to_drop override if > 50GB)
TRUNCATE TABLE system.text_log SETTINGS max_table_size_to_drop = 0;
TRUNCATE TABLE system.trace_log SETTINGS max_table_size_to_drop = 0;
TRUNCATE TABLE system.processors_profile_log SETTINGS max_table_size_to_drop = 0;
TRUNCATE TABLE system.query_log SETTINGS max_table_size_to_drop = 0;
TRUNCATE TABLE system.metric_log SETTINGS max_table_size_to_drop = 0;
TRUNCATE TABLE system.asynchronous_metric_log SETTINGS max_table_size_to_drop = 0;

Step 3: Set TTL to prevent recurrence

sql
-- Set 3-day TTL on all major system log tables
ALTER TABLE system.text_log MODIFY TTL event_date + INTERVAL 3 DAY SETTINGS materialize_ttl_after_modify = 0;
ALTER TABLE system.trace_log MODIFY TTL event_date + INTERVAL 3 DAY SETTINGS materialize_ttl_after_modify = 0;
ALTER TABLE system.processors_profile_log MODIFY TTL event_date + INTERVAL 3 DAY SETTINGS materialize_ttl_after_modify = 0;
ALTER TABLE system.query_log MODIFY TTL event_date + INTERVAL 7 DAY SETTINGS materialize_ttl_after_modify = 0;
ALTER TABLE system.metric_log MODIFY TTL event_date + INTERVAL 3 DAY SETTINGS materialize_ttl_after_modify = 0;
ALTER TABLE system.asynchronous_metric_log MODIFY TTL event_date + INTERVAL 3 DAY SETTINGS materialize_ttl_after_modify = 0;
ALTER TABLE system.part_log MODIFY TTL event_date + INTERVAL 3 DAY SETTINGS materialize_ttl_after_modify = 0;
ALTER TABLE system.query_views_log MODIFY TTL event_date + INTERVAL 3 DAY SETTINGS materialize_ttl_after_modify = 0;
Fix: ClickHouse Cloud (no shell access)

ClickHouse Cloud revokes all modify permissions on system.* from user-created roles:

REVOKE INSERT, ALTER, CREATE TABLE, DROP TABLE, TRUNCATE, OPTIMIZE ON system.* FROM default_role

You CANNOT fix this via SQL. The only options are:

  1. ClickHouse Cloud Console (https://clickhouse.cloud/): Go to service settings and reduce system log TTLs to 3-7 days
  2. Get the default user password from the console, then run the ALTER TTL commands
  3. ClickHouse Cloud Support: Request system log cleanup

To identify ClickHouse Cloud: check for numbered suffix tables (e.g., trace_log_16, text_log_19) which are from rotated server pods. Also, system.disks will show system-tables/mergetree/ paths with 16 EiB (object storage).

Show full SKILL.md (156 more words)Show less

Verification

sql
-- After cleanup, verify disk usage
-- Self-hosted:
SELECT formatReadableSize(free_space) FROM system.disks WHERE name = 'default';

-- Both: verify system database is now small
SELECT database, formatReadableSize(sum(total_bytes)) as size
FROM system.tables WHERE total_bytes > 0 GROUP BY database;

-- Test that writes work
INSERT INTO your_table (...) VALUES (...);

Notes

  • The max_table_size_to_drop safety limit defaults to 50 GB. Tables larger than this require SETTINGS max_table_size_to_drop = 0 to truncate/drop.
  • When disk is truly 100% full, even DDL operations fail. You MUST free space at the filesystem level first (detach+remove orphaned tables, or delete tmp files).
  • DETACH TABLE ... PERMANENTLY is important before removing data files - it prevents ClickHouse from trying to access the removed files.
  • materialize_ttl_after_modify = 0 prevents ClickHouse from immediately trying to rewrite all data to apply TTL (which would need disk space you don't have).
  • On ClickHouse Cloud, each server node rotation leaves behind numbered system log tables (e.g., _16, _19) that accumulate over time. With 65+ nodes over months, this adds up to tens of GB.
  • The text_log is typically the largest offender because it logs at trace/debug level by default and includes every log line from the ClickHouse server.

References

© divinevideo, MPL-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in .agents/skills/clickhouse-system-log-disk-exhaustion of divinevideo/divine-mobile.

Open the folder on GitHubat commit c3d6f7e

Compare with similar skills

Clickhouse System Log Disk Exhaustion next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Clickhouse System Log Disk Exhaustion compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Clickhouse System Log Disk Exhaustion this skilldivinevideo/divine-mobile266—~2kAutomated safety check: PassMPL-2.0
Opensourcefaqdigoal/blog8.6k—~966Automated safety check: PassGPL-2.0
Cloud SaaS Modechmonitor/chmonitor300—~2.5kAutomated safety check: NotesGPL-3.0
Adapter Alignmentevloghq/evlog1.9k—~1.3kAutomated safety check: PassMIT
Archestra Dev Investigatearchestra-ai/archestra4.4k—~831Automated safety check: PassCustom licence
Gke Cost Analysisgoogle/skills21k—~1.5kAutomated safety check: PassApache-2.0

Similar skills

  • Opensourcefaq

    digoal/blog

    解答与开源产品有关的深度技术问题，输出图文并茂的 Markdown 技术文章。触发条件：用户提出与开源项目（如 PostgreSQL、Redis、Kafka、Kubernetes、ClickHouse、Flink 等）相关的技术问题，并提供源码目录或 URL、deepwiki repo 名称。即使用户只说"帮我解答这个开源问题"或"分析一下这个项目的某个机制"，也应使用本…

    8.6k GitHub stars~966 tokensUpdated 2 days ago
    DatabasesAuto-check passed

  • Cloud SaaS Mode

    chmonitor/chmonitor

    Work on chmonitor's Cloud (SaaS) vs self-hosted (OSS) behaviour from ONE codebase.

    300 GitHub stars~2.5k tokensUpdated 5 days ago
    DatabasesAuto-check: notes

  • Adapter Alignment

    evloghq/evlog

    Twice-monthly check that evlog's drain adapters still send what each provider's own client sends.

    1.9k GitHub stars~1.3k tokensUpdated today
    DevelopmentAuto-check passed

  • Archestra Dev Investigate

    archestra-ai/archestra

    A skill your agent uses when investigating Archestra bugs or incidents — staging issues, backend 50x errors, Drizzle failed queries, DB connection pressure, deploy regressions, or Kubernetes/runtime…

    4.4k GitHub stars~831 tokensUpdated today
    DevOps & CloudAuto-check passed

  • Gke Cost Analysis

    google/skills

    Official

    Answer natural language questions and perform analysis on GKE cluster and workload costs using BigQuery billing exports, cost allocation data, and live cluster monitoring metrics.

    21k GitHub stars~1.5k tokensUpdated yesterday
    DevOps & CloudAuto-check passed

  • Cluster Operations

    chmonitor/chmonitor

    Cluster management: distributed tables, ON CLUSTER DDL, node lifecycle, resharding, load balancing, and Keeper migration.

    300 GitHub stars~634 tokensUpdated 5 days ago
    DevOps & CloudAuto-check passed

More from divinevideo/divine-mobile

All 103 skills in this repo

  • Argocd Externalsecret Namespace Permission

    divinevideo/divine-mobile

    Fix ArgoCD ExternalSecret deployment failing with "namespace X is not permitted in project Y".

    266 GitHub stars~931 tokensUpdated today
    Auto-check passed

  • Art Direct

    divinevideo/divine-mobile

    Art direction for any content — reads text, PDF, Word, HTML, PPT, then proposes 2-3 creative directions with photography style, mood, and visual language.

    266 GitHub stars~4.8k tokensUpdated today
    Auto-check passed

  • Async Await Null Race Condition

    divinevideo/divine-mobile

    Fix "Null check operator used on a null value" errors when an object is set to null during an async await.

    266 GitHub stars~881 tokensUpdated today
    Auto-check passed

  • AWS V4 Signing Custom Headers Gcs

    divinevideo/divine-mobile

    Add custom metadata headers (x-amz-meta-) to AWS v4 signed requests for GCS S3-compatible API.

    266 GitHub stars~1k tokensUpdated today
    Auto-check passed

  • Bash Herestring Newline Secrets

    divinevideo/divine-mobile

    Fix password/secret authentication failures caused by trailing newlines when creating Google Cloud secrets (or similar) with bash here-strings.

    266 GitHub stars~791 tokensUpdated today
    Auto-check passed

  • Blossom Content Addressed URL Filtering

    divinevideo/divine-mobile

    Fix silent video/media processing failures caused by URL extraction code that filters on file extensions (.mp4, .webm, .webp).

    266 GitHub stars~1.1k tokensUpdated today
    Auto-check passed

Works with

Categories

Questions about Clickhouse System Log Disk Exhaustion

What does Clickhouse System Log Disk Exhaustion do?

Fix ClickHouse "Code: 243 Cannot reserve 1.00 MiB, not enough space" errors caused by system log tables (textlog, tracelog, processorsprofilelog, querylog) filling the disk. Clickhouse System Log Disk Exhaustion is an agent skill from divinevideo/divine-mobile.00 MiB, not enough space" errors caused by system log tables (textlog, tracelog, processorsprofilelog, querylog) filling the disk.

When should I use Clickhouse System Log Disk Exhaustion?

Clickhouse System Log Disk Exhaustion fits situations like: clickHouse inserts fail with NOTENOUGHSPACE error; disk is 100% full but application tables are small; system database is 10-100x larger than user databases; sentry shows batch insert/commit failures across multiple tables simultaneously.

How do I install Clickhouse System Log Disk Exhaustion in Claude Code?

Run `npx skills add divinevideo/divine-mobile --skill clickhouse-system-log-disk-exhaustion -a claude-code`. Or copy the skill folder (.agents/skills/clickhouse-system-log-disk-exhaustion in divinevideo/divine-mobile) into .claude/skills/clickhouse-system-log-disk-exhaustion in your project. Claude Code loads it when a task matches its description.

How do I install Clickhouse System Log Disk Exhaustion in Codex?

Run `npx skills add divinevideo/divine-mobile --skill clickhouse-system-log-disk-exhaustion -a codex`. Or copy the skill folder (.agents/skills/clickhouse-system-log-disk-exhaustion in divinevideo/divine-mobile) into .agents/skills/clickhouse-system-log-disk-exhaustion in your project. Codex loads it when a task matches its description.

Can I use Clickhouse System Log Disk Exhaustion in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add divinevideo/divine-mobile --skill clickhouse-system-log-disk-exhaustion -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/clickhouse-system-log-disk-exhaustion, .gemini/skills/clickhouse-system-log-disk-exhaustion, .github/skills/clickhouse-system-log-disk-exhaustion and .opencode/skills/clickhouse-system-log-disk-exhaustion in your project.

What does Clickhouse System Log Disk Exhaustion need to run?

Going by SKILL.md and its folder, Clickhouse System Log Disk Exhaustion needs the command-line tools its instructions call (kubectl).

Does Clickhouse System Log Disk Exhaustion access the network?

SKILL.md names 2 domains. As links in the text: clickhouse.com and clickhouse.cloud. This is read from the text; nothing was executed.

Is Clickhouse System Log Disk Exhaustion safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Clickhouse System Log Disk Exhaustion use?

Clickhouse System Log Disk Exhaustion is published under the MPL-2.0 licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Clickhouse System Log Disk Exhaustion use?

About 2k tokens (SKILL.md is roughly 7.9k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Clickhouse System Log Disk Exhaustion?

Skills that share tags, products or a category with Clickhouse System Log Disk Exhaustion: Opensourcefaq (digoal/blog, 8.6k stars), Cloud SaaS Mode (chmonitor/chmonitor, 300 stars), Adapter Alignment (evloghq/evlog, 1.9k stars) and Archestra Dev Investigate (archestra-ai/archestra, 4.4k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Clickhouse System Log Disk Exhaustion?

divinevideo (a GitHub organization) maintains it in divinevideo/divine-mobile, which has 266 GitHub stars. The repository holds 103 skills in this directory. The repository was last updated on October 10, 2026.

Source: divinevideo/divine-mobile on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.