New: 97,000+ skills indexed from GitHub repos with 100+ stars · every skill safety-checkedBrowse top skillsTop skills
Esc to closeBrowse all skills
Submit a skillBrowse skills
Submit a skillBrowse skills

Agent skill

Psycopg2 Batch Insert Optimization

by divinevideo in divinevideo/divine-mobile

Optimize slow PostgreSQL inserts in Python using psycopg2. An agent skill from divinevideo/divine-mobile.

MPL-2.0Auto-check passedDatabases

Install Psycopg2 Batch Insert Optimization

skills CLI
$ npx skills add divinevideo/divine-mobile --skill psycopg2-batch-insert-optimization -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install divinevideo/divine-mobile psycopg2-batch-insert-optimization --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/divinevideo/divine-mobile.git skills-src && mkdir -p .claude/skills && cp -r skills-src/.agents/skills/psycopg2-batch-insert-optimization .claude/skills/psycopg2-batch-insert-optimization && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
psycopg2-batch-insert-optimization
GitHub stars
265
Token cost
~1.2k tokens
SKILL.md length
285 words
Files
1
Skills in repo
103
Repo updated
First seen
Licence
MPL-2.0

At a glance

Optimize slow PostgreSQL inserts in Python using psycopg2. An agent skill from divinevideo/divine-mobile.

  • Row-by-row inserts are taking too long over network
  • SKILL.md covers Problem, Context / Trigger Conditions, Solution and Verification, plus 3 more sections
  • Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md
  • Executemany() isnt providing speedup

What it does

Psycopg2 Batch Insert Optimization is an agent skill from divinevideo/divine-mobile. Optimize slow PostgreSQL inserts in Python using psycopg2. Use when: (1) Row-by-row inserts are taking too long over network, (2) executemany() isn't providing speedup, (3) Migrating large datasets to PostgreSQL, (4) Network latency making individual INSERT statements impractical. The key is using executevalues() from psycopg2.extras instead of executemany() or individual execute() calls.

Its SKILL.md is about 1.2k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

It sits in Databases. It works with PostgreSQL, Python and SQLite. The licence is MPL-2.0.

When your agent uses it

  • Row-by-row inserts are taking too long over network
  • Executemany() isnt providing speedup
  • Migrating large datasets to PostgreSQL
  • Network latency making individual INSERT statements impractical

Example prompts

  • “/psycopg2-batch-insert-optimization”

Requirements

  • Python 3

What it can do on your machine

Read from SKILL.md and the folder at commit 8d5fbf2. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    No scripts in the folder and no shell commands in SKILL.md (its code samples are python).

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    Links to these hosts (documentation or services it may open):

    • psycopg.org
    • postgresql.org

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Psycopg2 Batch Insert Optimization loads about 1.2k tokens when it runs. Until then it costs about 107 tokens; SKILL.md has 285 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~107
When it runs · the whole SKILL.md, loaded when a task matches
~1.2k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from divinevideo/divine-mobile at commit 8d5fbf2, republished under its MPL-2.0 licence (© divinevideo). 285 words, ~1,209 tokens.

Download SKILL.mdSave it as .claude/skills/psycopg2-batch-insert-optimization/SKILL.md (or your agent's skills folder).
name
psycopg2-batch-insert-optimization
description
Optimize slow PostgreSQL inserts in Python using psycopg2. Use when: (1) Row-by-row inserts are taking too long over network, (2) executemany() isn't providing speedup, (3) Migrating large datasets to PostgreSQL, (4) Network latency making individual INSERT statements impractical. The key is using execute_values() from psycopg2.extras instead of executemany() or individual execute() calls.
author
Claude Code
version
1.0.0
date
2025-01-20

psycopg2 Batch Insert Optimization

Problem

When inserting thousands of rows into PostgreSQL over a network connection, row-by-row inserts are extremely slow. Each INSERT requires a round-trip, and with network latency of ~50-100ms, inserting 10,000 rows takes 10+ minutes.

The naive approach of using cursor.executemany() doesn't help much—it still sends individual statements.

Context / Trigger Conditions

  • Inserting >100 rows into PostgreSQL via psycopg2
  • Each insert taking ~1 second or more
  • Network latency to database (especially Cloud SQL, RDS, remote databases)
  • Migration scripts running for hours
  • executemany() not providing expected speedup

Solution

Use execute_values() from psycopg2.extras:

python
from psycopg2.extras import execute_values

# Instead of this (SLOW):
for row in data:
    cursor.execute("INSERT INTO table (a, b, c) VALUES (%s, %s, %s)", row)

# Or this (STILL SLOW):
cursor.executemany("INSERT INTO table (a, b, c) VALUES (%s, %s, %s)", data)

# Use this (FAST):
execute_values(cursor, """
    INSERT INTO table (a, b, c)
    VALUES %s
    ON CONFLICT (id) DO NOTHING
""", data, page_size=500)
conn.commit()
Key Parameters:
  • page_size: Number of rows per batch (default 100, try 500-1000)
  • The VALUES %s placeholder is replaced with multiple value tuples
For UPSERT operations:
python
execute_values(cursor, """
    INSERT INTO users (user_id, username, email)
    VALUES %s
    ON CONFLICT (user_id) DO UPDATE SET
        username = EXCLUDED.username,
        email = COALESCE(EXCLUDED.email, users.email)
""", user_data, page_size=500)
Progress Monitoring for Long Migrations:
python
import sys
sys.stdout.reconfigure(line_buffering=True)  # Force unbuffered output

BATCH_SIZE = 500
for i in range(0, len(data), BATCH_SIZE):
    batch = data[i:i+BATCH_SIZE]
    execute_values(cursor, query, batch)
    conn.commit()
    print(f"Processed {min(i+BATCH_SIZE, len(data))}/{len(data)} rows...")

Verification

  • Migration that previously took hours completes in minutes
  • You can see batches being processed in real-time with progress output
  • Check row counts after: SELECT COUNT(*) FROM table

Example

Real-world migration of 9,563 users from SQLite to PostgreSQL:

python
from psycopg2.extras import execute_values
import sys

sys.stdout.reconfigure(line_buffering=True)
BATCH_SIZE = 500

# Fetch from SQLite
sqlite_cur.execute('SELECT user_id, username, avatar_url, verified FROM users')
rows = sqlite_cur.fetchall()
data = [(r['user_id'], r['username'], r['avatar_url'], bool(r['verified']))
        for r in rows]

# Batch insert to PostgreSQL
for i in range(0, len(data), BATCH_SIZE):
    batch = data[i:i+BATCH_SIZE]
    execute_values(pg_cur, '''
        INSERT INTO users (user_id, username, avatar_url, verified)
        VALUES %s
        ON CONFLICT (user_id) DO UPDATE SET
            username = COALESCE(EXCLUDED.username, users.username),
            avatar_url = COALESCE(EXCLUDED.avatar_url, users.avatar_url)
    ''', batch)
    pg_conn.commit()
    print(f"Processed {min(i+BATCH_SIZE, len(data))}/{len(data)} users...")

Result: 9,563 users migrated in ~20 seconds instead of ~2.5 hours.

Notes

  • execute_values() constructs a single INSERT with multiple VALUES, drastically reducing round-trips
  • executemany() is deceptively slow—it still sends individual statements
  • For very large datasets (>100k rows), consider COPY command or copy_expert()
  • The page_size parameter controls memory usage vs. batch efficiency
  • Always commit after each batch for long migrations (allows progress tracking and partial recovery)
SQLite to PostgreSQL Syntax Differences:

When migrating, also watch for these SQL differences:

  • INSERT OR IGNORE → ON CONFLICT DO NOTHING
  • INSERT OR REPLACE → ON CONFLICT DO UPDATE SET ...
  • MAX(a, b) (SQLite) → GREATEST(a, b) (PostgreSQL)
  • MIN(a, b) (SQLite) → LEAST(a, b) (PostgreSQL)
  • ? placeholders → %s placeholders
  • AUTOINCREMENT → SERIAL or GENERATED ALWAYS AS IDENTITY

References

© divinevideo, MPL-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in .agents/skills/psycopg2-batch-insert-optimization of divinevideo/divine-mobile.

Open the folder on GitHubat commit 8d5fbf2

Compare with similar skills

Psycopg2 Batch Insert Optimization next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Psycopg2 Batch Insert Optimization compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Psycopg2 Batch Insert Optimization this skilldivinevideo/divine-mobile265—~1.2kAutomated safety check: PassMPL-2.0
MoviePilot Database Operationjxxghp/MoviePilot12k—~7.1kAutomated safety check: PassGPL-3.0
Cognee Session Memory and Improvetopoteretes/cognee32k—~3kAutomated safety check: PassApache-2.0
SQL Database Support for pRESTprest/prest4.6k—~1.6kAutomated safety check: PassMIT
Saleor Django Migration Rulessaleor/saleor23k—~1.6kAutomated safety check: PassBSD-3-Clause
Chdb SQLvemetric/vemetric3941 repos~1.2kAutomated safety check: PassApache-2.0

Similar skills

  • MoviePilot Database Operation

    jxxghp/MoviePilot

    Inspects, queries and carefully modifies the MoviePilot SQLite or PostgreSQL database through a bundled script that reads connection settings itself, without needing the password in the prompt.

    12k GitHub stars~7.1k tokensUpdated today
    DatabasesAuto-check passed

  • Cognee Session Memory and Improve

    topoteretes/cognee

    Explains how cognee stores session memory by session_id and bridges it into the permanent graph with improve(), including the stages, results and settings.

    32k GitHub stars~3k tokensUpdated today
    Agent WorkflowsAuto-check passed

  • SQL Database Support for pREST

    prest/prest

    Guides classifying, gap-analyzing and scaffolding support for a new SQL database in pREST, from Postgres-compatible variants to entirely new dialects.

    4.6k GitHub stars~1.6k tokensUpdated yesterday
    DatabasesAuto-check passed

  • Saleor Django Migration Rules

    saleor/saleor

    Rules for writing Django migrations in Saleor that avoid long table locks and stay compatible with zero-downtime rolling deploys.

    23k GitHub stars~1.6k tokensUpdated yesterday
    DatabasesAuto-check passed

  • Chdb SQL

    vemetric/vemetric

    A skill your agent uses when the user wants to run SQL — especially analytical SQL — on local files (parquet/csv/json), URLs, S3 paths, or remote databases (Postgres, MySQL, MongoDB, ClickHouse…

    394 GitHub starsUsed in 1 repo~1.2k tokens
    DatabasesAuto-check passed

  • Migration

    gocronx-team/gocron

    Create, review, or verify gocron database migrations across SQLite, MySQL, and PostgreSQL.

    808 GitHub stars~1.1k tokensUpdated 5 days ago
    DatabasesAuto-check passed

More from divinevideo/divine-mobile

All 103 skills in this repo

  • Argocd Externalsecret Namespace Permission

    divinevideo/divine-mobile

    Fix ArgoCD ExternalSecret deployment failing with "namespace X is not permitted in project Y".

    265 GitHub stars~931 tokensUpdated today
    Auto-check passed

  • Art Direct

    divinevideo/divine-mobile

    Art direction for any content — reads text, PDF, Word, HTML, PPT, then proposes 2-3 creative directions with photography style, mood, and visual language.

    265 GitHub stars~4.8k tokensUpdated today
    Auto-check passed

  • Async Await Null Race Condition

    divinevideo/divine-mobile

    Fix "Null check operator used on a null value" errors when an object is set to null during an async await.

    265 GitHub stars~881 tokensUpdated today
    Auto-check passed

  • AWS V4 Signing Custom Headers Gcs

    divinevideo/divine-mobile

    Add custom metadata headers (x-amz-meta-) to AWS v4 signed requests for GCS S3-compatible API.

    265 GitHub stars~1k tokensUpdated today
    Auto-check passed

  • Bash Herestring Newline Secrets

    divinevideo/divine-mobile

    Fix password/secret authentication failures caused by trailing newlines when creating Google Cloud secrets (or similar) with bash here-strings.

    265 GitHub stars~791 tokensUpdated today
    Auto-check passed

  • Blossom Content Addressed URL Filtering

    divinevideo/divine-mobile

    Fix silent video/media processing failures caused by URL extraction code that filters on file extensions (.mp4, .webm, .webp).

    265 GitHub stars~1.1k tokensUpdated today
    Auto-check passed

Works with

Categories

Questions about Psycopg2 Batch Insert Optimization

What does Psycopg2 Batch Insert Optimization do?

Optimize slow PostgreSQL inserts in Python using psycopg2. An agent skill from divinevideo/divine-mobile. Psycopg2 Batch Insert Optimization is an agent skill from divinevideo/divine-mobile. Optimize slow PostgreSQL inserts in Python using psycopg2.

When should I use Psycopg2 Batch Insert Optimization?

Psycopg2 Batch Insert Optimization fits situations like: row-by-row inserts are taking too long over network; executemany() isnt providing speedup; migrating large datasets to PostgreSQL; network latency making individual INSERT statements impractical.

How do I install Psycopg2 Batch Insert Optimization in Claude Code?

Run `npx skills add divinevideo/divine-mobile --skill psycopg2-batch-insert-optimization -a claude-code`. Or copy the skill folder (.agents/skills/psycopg2-batch-insert-optimization in divinevideo/divine-mobile) into .claude/skills/psycopg2-batch-insert-optimization in your project. Claude Code loads it when a task matches its description.

How do I install Psycopg2 Batch Insert Optimization in Codex?

Run `npx skills add divinevideo/divine-mobile --skill psycopg2-batch-insert-optimization -a codex`. Or copy the skill folder (.agents/skills/psycopg2-batch-insert-optimization in divinevideo/divine-mobile) into .agents/skills/psycopg2-batch-insert-optimization in your project. Codex loads it when a task matches its description.

Can I use Psycopg2 Batch Insert Optimization in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add divinevideo/divine-mobile --skill psycopg2-batch-insert-optimization -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/psycopg2-batch-insert-optimization, .gemini/skills/psycopg2-batch-insert-optimization, .github/skills/psycopg2-batch-insert-optimization and .opencode/skills/psycopg2-batch-insert-optimization in your project.

What does Psycopg2 Batch Insert Optimization need to run?

SKILL.md names no scripts, command-line tools or credentials: Psycopg2 Batch Insert Optimization is instructions for the agent only. Our summary lists: Python 3.

Does Psycopg2 Batch Insert Optimization access the network?

SKILL.md names 2 domains. As links in the text: psycopg.org and postgresql.org. This is read from the text; nothing was executed.

Is Psycopg2 Batch Insert Optimization safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Psycopg2 Batch Insert Optimization use?

Psycopg2 Batch Insert Optimization is published under the MPL-2.0 licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Psycopg2 Batch Insert Optimization use?

About 1.2k tokens (SKILL.md is roughly 4.8k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Psycopg2 Batch Insert Optimization?

Skills that share tags, products or a category with Psycopg2 Batch Insert Optimization: MoviePilot Database Operation (jxxghp/MoviePilot, 12k stars), Cognee Session Memory and Improve (topoteretes/cognee, 32k stars), SQL Database Support for pREST (prest/prest, 4.6k stars) and Saleor Django Migration Rules (saleor/saleor, 23k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Psycopg2 Batch Insert Optimization?

divinevideo (a GitHub organization) maintains it in divinevideo/divine-mobile, which has 265 GitHub stars. The repository holds 103 skills in this directory. The repository was last updated on October 7, 2026.

Source: divinevideo/divine-mobile on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.