Mem0 Test Integration

mem0-test-integration

Verifies what /mem0-integrate produced. Runs in the same workspace, on the same feature branch. Loose coupling — fast, catches compile and runtime bugs, does not catch logical errors.

Canonical sources (use these, not ambient knowledge)

All static checks and smoke-test shapes validate against these URLs. WebFetch each before running step 3.

• Scope-tagged docs index: https://docs.mem0.ai/llms.txt
• OpenAPI (Platform REST): https://docs.mem0.ai/openapi.json
• Published SDK skill (canonical call patterns): https://raw.githubusercontent.com/mem0ai/mem0/main/skills/mem0/SKILL.md
• Vercel AI SDK skill (if the target repo uses @ai-sdk/*): https://raw.githubusercontent.com/mem0ai/mem0/main/skills/mem0-vercel-ai-sdk/SKILL.md
• SDK source (cross-check version against frontmatter mem0_tested_versions):
  • Repo root: https://github.com/mem0ai/mem0
  • Python: https://github.com/mem0ai/mem0/tree/main/mem0
  • TypeScript: https://github.com/mem0ai/mem0/tree/main/mem0-ts

Read the Delegated skill: field in .mem0-integration/plan.md — if it names a skill URL, fetch that skill and use its example blocks as the reference for both static checks (step 3) and the smoke test (step 5).

Non-invasiveness contract

Every check in this skill assumes the integration is additive and feature-flagged (see /mem0-integrate "Integration principles"). Specifically:

• product.json must contain a feature_flag field.
• Steps 4–6 run in two passes:
  • Pass A — flag unset. All pre-existing tests must pass, smoke/E2E skip. The repo must behave like main. Any failure here is a hard fail — do not let the self-heal loop attempt a patch.
  • Pass B — flag set. New tests must pass, smoke and E2E run.
• If Pass A fails, the scorecard marks non_invasive: false and sets overall: fail with a distinct reason code the integrator's heal loop refuses to touch.

Preconditions

Refuse to start unless ALL of the following are true:

• .mem0-integration/ directory exists in the repo root.
• .mem0-integration/product.json, goal.md, and plan.md are readable and internally consistent (JSON parses, docs non-empty).
• Current branch name begins with mem0-integrate/ (set by the companion skill). Prevents accidental runs on unrelated branches.
• Working tree is clean. The skill never modifies source files; any dirty state means the integration is mid-edit and not ready to verify.
• The same API key the integration used is available in the environment (MEM0_API_KEY for Platform, OPENAI_API_KEY for OSS — read which from product.json). Interactive mode asks if missing; CI mode exits 2.

Exit with a written rationale on any precondition failure. Never attempt to "fix up" state.

Pipeline

1. Read the contract

Load:

• product.json → which language, which product (Platform vs OSS), which mem0 version, write_site, read_site.
• plan.md → the mechanical contract (write pattern, read pattern, preserved behavior).
• goal.md → the intent (displayed in the scorecard only; not tested).

2. Install dependencies

Route by language from product.json:

If install fails → exit code 2 with stderr tail. Never move to testing if dependencies don't resolve.

3. Static sanity checks (fast, local, no API calls)

• Import check: does the write-site file import the expected Mem0 surface? Authoritative list comes from ## Identify the User's Setup in https://docs.mem0.ai/llms.txt:

  • Platform Python → from mem0 import MemoryClient
  • Platform TS → import MemoryClient from "mem0ai"
  • OSS Python → from mem0 import Memory
  • OSS TS → import { Memory } from "mem0ai/oss"

  If plan.md names a delegated skill (e.g., Vercel AI), use that skill's import signature instead of the list above. Mismatch → fail with line number.

• Version check: installed mem0ai version falls in the range from this skill's mem0_tested_versions. Out of range → warn but continue.

• Type check (TS tracks only): run tsc --noEmit or tsup --dts. Non-zero → fail.

• Lint (if the repo has a linter configured): run the repo's own lint command. Lint failures from this skill's changes → fail; pre-existing lint failures → surface as a warning.

• Eager-init check: grep the write_site and read_site files (paths from product.json) for MemoryClient( or Memory( at module scope — i.e., not inside a function, method, or class body. MemoryClient() validates the API key in __init__ (Python pings the API, TS throws on a blank key and pings in the background) and OSS Memory() can eagerly initialize embedding/LLM providers — module-level instantiation hits the wire on import and breaks Pass A's test collection whenever the key is unset. Hit → fail with file:line and the lazy-init guidance from /mem0-integrate step 8 constraint #7.

4. Run the repo's native test suite (two passes)

Pass A — feature_flag unset. Run the entire pre-existing suite (excluding the new test_mem0_* files). Must be 100% green. Any failure here marks non_invasive: false in the scorecard and is a hard fail — the integrator's self-heal loop refuses to touch it.

Pass B — feature_flag set (value from product.json). Run the full suite including the new tests. All must pass.

Isolate integration-introduced failures using git diff main..HEAD --name-only. A test file that exists on main and fails only under the integration branch (flag set or unset) counts against the scorecard regardless of pass. A test file that already failed on main is surfaced as pre_existing_unrelated and does not count — but is still reported so the user can clean it up.

Capture output to .mem0-integration/test-stdout-flag-off.log and .mem0-integration/test-stdout-flag-on.log. Scorecard reports pass/fail per pass.

5. Smoke test (real API call, shortest round-trip)

Scripted end-to-end flow tailored to product.json. The call shapes below are the minimal ones; if plan.md names a delegated skill, use that skill's minimal example verbatim instead — it is the canonical shape for the detected stack.

Platform (Python):

import os
import time
from mem0 import MemoryClient
c = MemoryClient()                               # uses MEM0_API_KEY
uid = f"mem0-test-integration-{os.urandom(4).hex()}"
c.add([{"role": "user", "content": "I prefer aisle seats"}], user_id=uid)
for _ in range(10):
    hits = c.search("seat preference", filters={"user_id": uid})["results"]
    if hits:
        break
    time.sleep(2)
assert any("aisle" in h.get("memory", "") for h in hits), hits
c.delete_all(user_id=uid)                        # clean up

add is asynchronous on Platform (it returns status: "PENDING" with an event_id), so the search is retried for a bounded time. Entity IDs go in filters and results come back under ["results"].

Platform (TS): same shape with new MemoryClient({ apiKey: process.env.MEM0_API_KEY }) from "mem0ai" (the TS client does not read the env var itself), client.search("seat preference", { filters: { user_id: uid } }) with hits under .results, and client.deleteAll({ userId: uid }).

OSS (Python / TS): uses Memory() / new Memory() with default config (OpenAI LLM via OPENAI_API_KEY; Python defaults to an embedded on-disk Qdrant, TS to an in-memory vector store, so no service is needed unless the repo's own config points at one). If the repo ships a docker-compose.yml with the configured vector store service, the skill starts it first and tears it down after. If the configured backing store is not reachable → fail with a clear message naming the fix.

The smoke test always uses a disposable random user_id prefixed with mem0-test-integration- so a failed cleanup doesn't pollute the user's real data. A background tidy step lists Platform entities with client.users(), picks the type: "user" entries whose name starts with that prefix and whose created_at is older than 24 hours, and calls delete_all(user_id=...) for each on the next run (there is no server-side prefix delete).

Capture output to .mem0-integration/smoke-stdout.log.