Shopify Review Triage

Shopify review triage - public low-star reviews to a P0-P3 brief

What this does

Takes rows of public Shopify App Store review text and produces one prioritized brief a product or support owner can act on: what kind of problem each review describes, how badly it can hurt, what to do first, and where the original wording came from.

It is built for independent Shopify app teams and the agencies that run their support - the case where low-star reviews arrive scattered across several listings plus a few watched competitors, and the failure mode is treating them all as equally urgent.

The rubric below is not invented here. It reproduces a publicly published rule set verbatim, so a manual pass and this skill sort the same row the same way. See Provenance for the source.

Hard rules

These are not style preferences. Breaking one makes the output worse than nothing.

1. Public review text only. Never accept, request, or copy support tickets, merchant emails, order data, personal contact details, internal telemetry, or anything else not already public on a listing page. If such data appears in the input, stop, say which rows are affected, and ask for them to be removed before continuing.
2. Never invent evidence. Do not write a review, a rating, a date, an app name, or a source URL that was not supplied. A row with no link gets source: not captured - never a guessed one.
3. Keyword output is a sort, not a verdict. Everything produced by the rubric alone is labeled first pass - not human-checked. Only a person who read the review and checked it against their own systems may relabel an item human-checked.
4. Reviews are customer reports, not verified defects. Write "the reviewer reports the editor showed a blank screen", never "the editor is broken". The distinction survives into the brief.
5. No coverage claims. The brief covers exactly the rows supplied and says so. Make no claim of exhaustive coverage of a listing, a period, or an app.
6. No promises. No revenue impact, no outcome, no ranking effect, no legal or compliance advice. Suggest actions; do not predict results.
7. Draft only - never contact anyone. Do not send email, post a developer reply, open a support ticket, message a reviewer, or publish anything. Hand the draft back to the team and let a person decide what to send.
8. Reviewers are people. Refer to "the reviewer". Do not name, profile, or speculate about them.

1. Collect the rows

Ask for one review per line. The full form keeps the source link, which the brief needs:

rating | app name | review date | public reviews URL | review text

A shorter three-field form is also accepted - treat field 1 as the rating when it is a bare 1-5 (optionally followed by star or stars), otherwise as the app name:

rating | app name | review text

Rules for this step:

• Lines starting with # are comments. Blank lines are skipped.
• If a row lacks a source URL, carry source: not captured through to the brief. Do not drop the row and do not fabricate a link.
• Do not go and fetch anything yourself. This skill needs no network access; the person you are helping pastes the public rows they already opened.
• The trigger this rubric is tuned for is a new 1-3-star review. Higher-rated rows still classify correctly (a 5-star review often lands in feature requests or needs-human-read), so keep them if they were supplied, but never present them as low-star signal.

2. First pass - apply the rubric

Lower-case the review text and normalize curly apostrophes (’ to ') before matching, so a pasted "won’t load" still matches won't load. Also match every keyword below with the apostrophe dropped entirely: merchants routinely type these contractions without one, and the apostrophe-free spelling must classify exactly the same as the contracted form.

Five buckets. Each row gets exactly one primary bucket - the first dimension below, in this order, with any matching keyword. Further matches are recorded as secondary, never as a second brief item.

P0 - Incident risk

The purchase path, app activation, or merchant data may be at stake right now. Left alone it costs the merchant money and the team installs.

Suggested action. Try to reproduce on a test store today. If confirmed, treat it as an incident: fix or mitigate first, then reply to the reviewer with what changed.

Signal keywords. won't load, won't open, won't close, can't close, cannot close, blank screen, broken, crash, stopped working, not working, doesn't work, does not work, checkout, losing sales, lost sales, error

P1 - Repeated friction

The product works, but the same struggle keeps showing up across reviews or against an open support theme. Repetition is the signal, not volume of adjectives.

Suggested action. Log it against the matching support theme. If the same complaint repeats across rows, schedule a UX fix ahead of new feature work.

Signal keywords. confusing, unclear, hard to, difficult, complicated, clunky, slow, couldn't figure, could not figure, annoying, had to contact support, setup took, too many steps

P2 - Pricing confusion

What the merchant expected to pay and what happened diverged. Usually a copy problem in the listing, the plan limits, or the upgrade prompts - not a code problem.

Suggested action. Compare what the reviewer expected with the listing's pricing section and in-app upgrade prompts; clarify the copy where they diverge.

Signal keywords. pricing, price, charged, charge, billing, billed, expensive, free plan, trial, refund, hidden fee, hidden cost, paywall

P3 - Feature request

The merchant wants something the app does not do, or could not find. Valuable as a log entry, rarely urgent on its own.

Suggested action. Add it to the feature-request log with a link to the review. If the capability already exists, reply to the reviewer with where to find it.

Signal keywords. wish, would be great, would love, please add, feature request, missing, if only, would like, no option to, needs an option, hope you add, add support for

Needs human read

No keyword matched. Vague frustration, sarcasm, mixed praise, or a story that needs context.

Suggested action. No keyword matched. Read the full review yourself and file it manually - the heuristic makes no guess here.

Priority. The rubric labels this bucket P2 and sorts it last. Treat that label as provisional placement in the queue, not as a severity judgment - nothing has been judged yet.

Tie-breaks and escalation

1. Most severe wins. A row naming both a broken checkout and a billing surprise files under P0 with pricing noted as secondary. Never split one review across two brief items.
2. Repetition escalates. If the same friction or pricing theme appears in three or more reviews within about 60 days, move it up one level and say how many rows drove the change.
3. Age discounts. A review older than a year is background, not evidence of a current problem, unless a recent row corroborates it. Cite it as context, never as the headline.
4. Competitor reviews never create a P0 for you. A competitor's incident is roadmap, positioning, or copy input - it belongs in the competitor watch section.
5. When unsure, choose needs human read. The bucket exists so the rubric never launders uncertainty into a priority label.