Investigate Forum

Investigate Forum

Sam runs this about once a day and comes back every few hours to approve things. The job: for each forum topic still waiting on Sam, do the investigation he would do, get a fix as far as a reviewed PR when it is safe to, and leave a reply he can paste.

Ground rules

These hold for every topic, every run. Do not reinterpret them mid-run.

1. At most three topics per run, one at a time, newest activity first. The queue is cut at three (--limit, default 3); the rest wait for the next run so open PRs get merged before more pile up. Finish a topic (state recorded) before opening the next. No parallel subagents across topics. A subagent may help inside one topic (for example, a second opinion on a fix) but the loop stays sequential.
2. Production is read-only. Pre-authorized: Django ORM reads, mongoengine reads, redis read commands (GET, HGETALL, ZRANGE, SMEMBERS, KEYS on a narrow pattern, INFO), log greps, sentry-cli issues list, Play Console crash reads. Never .save(), .update(), .delete(), .create(), bulk writes, redis writes, container restarts, deploys, or make deploy/make celery. Anything that changes production state is tier 2 and needs an explicit yes through AskUserQuestion, then run exactly the approved command and nothing more.
3. Never post to the forum. Replies are drafted for Sam to paste. Never call any Discourse write endpoint.
4. Never merge or push to main. PRs are opened ready for review with the forum label. Sam merges, or says "merge" in the session, in which case the merge procedure in the commit-pr skill applies (order, bringing branches up to date, the test-file conflict pattern).
5. Tier 1 means no input needed. The bug is reproducible, the fix is local to the code, no product decision is involved, no user account is touched. Everything else is tier 2.
6. Reproduce before fixing. Per CLAUDE.md: write the failing test first, then fix, then show it passing. If it cannot be reproduced, say so in the PR and the reply rather than guessing.
7. Reply-only topics are tier 1. A how-to question, a known limitation, a duplicate, a "works as designed": draft the reply, record it, move on. No interview needed.
8. Ask with AskUserQuestion, never plain text. Tier 2 decisions, and anything mid-fix that could go two materially different ways.
9. When the auto-mode classifier blocks an approved prod write, hand it to Sam. Write the exact script or SQL to a file at the repo root, print the one-line command that runs it, record tier2-pending with that command in the note, and move on. Never look for another route to the same write.
10. Before any prod merge_feeds, check for branches. Feed.objects.filter(branch_from_feed=<duplicate>) must be empty or re-parented first; until PR #2133 is deployed, a merge that deletes a feed with branches deletes the branches and their subscriptions too (CBC incident, 2026-09-14). Prefer force=False so the heavier feed survives. Once #2133 is deployed, every merge logs MERGE_FEEDS_INVENTORY JSON lines and a bad one is reversed with manage.py restore_merged_feed --log FILE --feed-id ID --dry-run first, then without --dry-run; grep the lines from docker logs task-celery on the worker that ran the merge.
11. A PR is done only when /commit-pr says so. Green CI on the current head, zero unresolved Claude or Codex review threads, marked ready for review. The push-to-clean loop lives in the commit-pr skill; this skill never re-implements it. That loop is capped at three review rounds; a fix that keeps growing under review (locks, leases, queues, a new store) has become a subsystem and goes back to Sam through AskUserQuestion before another round, with the PR's size next to its first commit's.

Arguments

/investigate-forum [flags] [forum URL ...]

Files in this skill

Run both scripts from the repo root with python3 .agents/skills/investigate-forum/<script>.

Step 0: Preflight

From the main repo checkout on main:

git status --short            # expect clean, or only state.json changes
git fetch origin && git status -sb | head -1   # note if main is behind origin
docker ps --format '{{.Names}}' | grep -c newsblur_db_   # shared DBs must be up (4)
gh auth status 2>&1 | head -2

If main has uncommitted changes other than state.json, stop and tell Sam; worktrees branch from main and a dirty main means the branch point is ambiguous. If the shared DBs are down, run make in the main repo first.

Step 1: Fetch the queue

python3 .agents/skills/investigate-forum/fetch_topics.py --summary          # headers
python3 .agents/skills/investigate-forum/fetch_topics.py                    # headers + full raw threads
python3 .agents/skills/investigate-forum/fetch_topics.py --topic 13833      # one topic

Pass --days, --since, --all, --limit straight through. The output is newest activity first and already excludes topics whose last post is Sam's (samuelclay) and topics handled in state.json at their current last post. A topic that was handled but has a newer post since comes back marked FOLLOW-UP; treat it as new, but read the state entry first so the previous PR or reply is not duplicated.

Before starting the loop, also list any state.json entries with "status": "tier2-pending". Those are questions Sam has not answered yet. Re-ask each one at the top of the run (Step 5) so he can clear them before new work begins.

Print the queue as a short numbered list so Sam can see the plan, then start with item 1.

Step 2: Investigate one topic

Read the whole thread. Note who posted, what platform (web, iOS, Android, Mac, API), which account features are involved (premium tier, newsletters, archive, briefing, Ask AI), and whether another forum member already answered correctly.

Classify

Pick exactly one:

When unsure between tier 1 and tier 2, it is tier 2. When unsure between reply-only and skip, reply.

Investigation toolbox

Codebase. Grep first. The web frontend is Backbone under media/js/newsblur/, Django apps under apps/, iOS under clients/ios/, Android under clients/android/. Check git log -S'<phrase>' --since='3 months ago' for recent changes near the symptom; many forum reports follow a deploy.

Local reproduction. The main stack is https://localhost (make to start). Dev users: chrome, samuel, archive14, free3, free4, archive11. Autologin at /reader/dev/autologin/<user>/. API calls via make api URL=/reader/feeds. Subscription state is set through the Django shell recipe in CLAUDE.md under "User State".

Production, read-only. All through ./utils/ssh_hz.sh -n <server> "<command>". Shell examples (each is a one-shot Django shell; escape inner quotes as shown):

# Who is this forum user on NewsBlur? Match on username or email from the thread.
./utils/ssh_hz.sh -n happ-web-01 "docker exec -t newsblur_web python manage.py shell -c \"
from django.contrib.auth.models import User
for u in User.objects.filter(username__iexact='mtaylor') | User.objects.filter(email__iexact='someone@example.com'):
    p = u.profile
    print(u.pk, u.username, u.email, u.last_login, 'premium' if p.is_premium else 'free', 'archive' if p.is_archive else '', 'pro' if p.is_pro else '', p.premium_expire)
\""

# A feed by URL or id: fetch state, error counts, schedule, subscribers.
./utils/ssh_hz.sh -n happ-web-01 "docker exec -t newsblur_web python manage.py shell -c \"
from apps.rss_feeds.models import Feed, MFetchHistory
for f in Feed.objects.filter(feed_address__icontains='tmz.com')[:10]:
    print(f.pk, f.feed_address, f.active, f.num_subscribers, f.last_update, f.next_scheduled_update, f.fetched_once, f.has_feed_exception, f.has_page_exception)
    print(MFetchHistory.feed(f.pk)['feed_fetch_history'][:5])
\""

# Newsletter address for a user (username-secret@newsletters.newsblur.com) and their newsletter feeds.
./utils/ssh_hz.sh -n happ-web-01 "docker exec -t newsblur_web python manage.py shell -c \"
from django.contrib.auth.models import User
from apps.rss_feeds.models import Feed
u = User.objects.get(username='mtaylor')
print(u.profile.secret_token)
for f in Feed.objects.filter(feed_address__startswith='newsletter:', usersubscription__user=u)[:20]:
    print(f.pk, f.feed_address, f.feed_title, f.last_update, f.num_subscribers)
\""

# Recent web or task logs around a symptom.
./utils/ssh_hz.sh -n happ-web-01 "docker logs newsblur_web --since 24h 2>&1 | grep -i 'newsletter' | tail -50"
./utils/ssh_hz.sh -n htask-celery-01 "docker logs task-celery --since 24h 2>&1 | grep -i 'tmz' | tail -50"

# Redis, read commands only.
./utils/ssh_hz.sh -n hdb-redis-story-1 "redis-cli hgetall feed:12345"

Server names live in ansible/inventories/hetzner.ini. Sentry: sentry-cli --url https://sentry.newsblur.com issues list -o newsblur -p web --status unresolved --query "<keyword>" (projects: web, task, node, monitor). Android crashes: the Play Developer Reporting recipe in CLAUDE.md. Email delivery (newsletters, notifications) goes through ImprovMX into apps/newsletters/views.py:newsletter_receive; bounce text quoted by a user is the best evidence there.

Never paste a user's email address or secret token into the PR or the reply. Refer to them by forum username.

Step 3: Fix PR flow (tier 1, or tier 2 after Sam approves the fix)

Branch and worktree are named forum-<topic id>-<three or four word slug>, for example forum-13833-newsletter-spam-score.

1. Create the worktree from main and start its stack.

   git worktree add .worktree/forum-13833-newsletter-spam-score -b forum-13833-newsletter-spam-score main
   cd .worktree/forum-13833-newsletter-spam-score && make worktree

   make worktree prints the workspace URLs and ports. Containers are newsblur_web_<name>, newsblur_celery_<name>, etc. Restart newsblur_celery_<name> after touching anything a Celery task imports.

2. Reproduce with a failing test first. Test classes are Test_*, methods test_*. Run make test SCOPE=apps.<app> ARGS="-v 2" from the worktree and keep the failing output for the PR. For client bugs (iOS, Android) follow the platform notes in CLAUDE.md and clients/ios/CLAUDE.md; a unit test where one is feasible, otherwise a clear manual repro.

3. Before screenshot when the symptom is visible in the web UI. Put screenshots in .worktree/<name>/screenshots/ (that directory is inside the gitignored worktree, so nothing is committed).

   python3 ../../.agents/skills/investigate-forum/screenshot.py --workspace forum-13833-newsletter-spam-score \
       --user chrome --js "NEWSBLUR.reader.open_river_stories()" --wait 4000 --out screenshots/before.png

   --path, --selector, --theme dark, --full-page, and repeated --js are available; see the script docstring. Read the PNG afterwards to confirm it shows the bug. Backend-only fixes skip screenshots and show the failing then passing test output instead.

4. Fix. Smallest change that makes the test pass. Match the code style rules in CLAUDE.md (Black 110, snake_case JS, comments that name the file, no TODOs). Restart the worktree Celery container if the change runs inside a task.

5. Prove it. Test passes. After screenshot with the identical command as before, then read it. Run make lint from the worktree if Python changed.

6. Commit. One focused commit per logical change, subject in the imperative naming the user-visible fix with the topic in parentheses, attribution lines from the current session appended. Stage specific files, not git add -A.

   git add apps/rss_feeds/models.py utils/feed_fetcher.py apps/rss_feeds/test_rss_feeds.py
   git commit -m "$(cat <<'EOF'
   Never ration a forbidden feed shared by active readers by their proxy budget (forum #13832)
   
   <why, wrapped at 72 characters>
   
   Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
   Claude-Session: <session url>
   EOF
   )"

7. Hand off to /commit-pr. Invoke the commit-pr skill from the worktree with: the title (same as the commit subject), the label forum, the screenshot attachments, and the PR body below. That skill pushes, opens the PR as a draft, watches CI, fixes failures, promotes the PR to ready for review, waits for the Claude and Codex review bots, fixes every review thread, and returns only when the PR is verifiably clean or it is blocked. Do not duplicate any of that here; if it reports a blocker, carry the blocker into the state entry and the report.

   PR body to pass in:

   ## Forum topic
   https://forum.newsblur.com/t/<slug>/<id>
   <one sentence: what @username reported>
   
   ## Cause
   <two or three sentences, name the file and function>
   
   ## Fix
   <what changed and why this is the smallest safe change>
   
   ## Reproduction
   - Failing test before: `make test SCOPE=apps.x ARGS="-v 2"` (output excerpt)
   - Passing after (output excerpt)
   - Before / after screenshots attached
   
   ## Deploy notes
   <make deploy, make celery, both, or none; migrations; Celery restart>
   
   ## Forum reply draft
   **Reply for [#<id> <title>](https://forum.newsblur.com/t/<slug>/<id>)** (<username>):
   
   <the reply from Step 6, plain text>
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   
   <session url>

8. Stop the stack, keep the worktree. make worktree-stop from the worktree, then cd back to the main repo. Sam runs make worktree-close after merging.

9. Record state (Step 7) with pr-open, the PR URL, branch, worktree path, and the reply.