Mfs Find

MFS — find / read across configured sources

1. What MFS is

A retrieval layer that exposes many kinds of content as a unified path tree and makes that tree searchable through one hybrid index:

• One CLI (mfs), one mental model. Local dir, Postgres, GitHub repo, Slack workspace, S3 bucket, BigQuery dataset — all addressed as paths under their <scheme>:// URI. Same verbs everywhere: ls / tree / cat / head / tail / grep / search / export.
• One hybrid index. Dense vectors (semantic) + BM25 (keyword) fused per query — covers conceptual recall and exact-token recall in one call.
• POSIX-style locators. Every search hit carries a locator that reopens the exact unit: {"lines":[s,e]} for text/code, a PK dict for rows/issues/threads.

2. When to use MFS — and when NOT to

Rule: use the smallest tool that answers the question. MFS pays off when the scope is too big for rg.

Borderline — ASK the user:

3. Pre-flight — confirm the source is indexed

Before running any query, especially on cross-source asks:

mfs status                  # server up? any connectors registered?
mfs connector inspect <uri> # this connector's object/job summary
mfs ls <uri> --json         # per-entry capabilities + indexable / search_status

• Server unreachable → tell user to start it (mfs serve start if self-hosted), or this skill can't proceed.
• connectors empty → user hasn't ingested anything yet. Redirect to mfs-ingest — don't try to search nothing.
• search_status: unavailable for the target URI → only grep / ls / cat work; offer those or redirect to mfs-ingest for a re-sync.
• building → sync in flight; fall back to mfs grep (works without an index) until done.
• partial → recall incomplete but usable; flag the caveat to the user.

4. The core workflow: search → locate → browse

        search                 locate                browse
  ┌──────────────────┐   ┌──────────────────┐   ┌─────────────────────┐
  │  semantic + BM25 │ → │ result has lines │ → │  cat --range / cat  │
  │  finds candidates│   │  or a locator    │   │  --peek to confirm  │
  └──────────────────┘   └──────────────────┘   └─────────────────────┘

On large corpora this loop is the whole point: read only the part that matters. On small corpora it's still fine, just lighter.

Concrete:

1. Search:

   mfs search "<what the user actually wants>" <path-or-uri> --top-k 10

2. Locate — every hit's envelope carries locator:
   • text/code → {"lines":[start,end]} → mfs cat <source> --range start:end
   • structured (row/issue/thread) → PK dict → mfs cat <source> --locator '{...}'
   • once-per-object (dir/schema summary, image VLM) → null → mfs cat <source>
3. Browse — verify only what's needed:

   mfs cat --peek <file>       # outline (headings / function signatures)
   mfs cat --skim <file>       # peek + one-line summaries per section
   mfs head -n 20 <uri>        # first records of a structured object
   mfs tree <uri> -L 2         # subtree shape

5. Index requirement rules of thumb

• mfs search requires an index.
• mfs grep works without — pushdown → BM25 → linear scan fallback.
• mfs ls / tree / cat / head / tail browse without an index.

6. Search modes

mfs search defaults to hybrid. Override only when you know why.

Other useful flags:

• --top-k N — default 10; raise to 20-30 on a weak first round.
• --all — search every registered connector. Otherwise scope to a path/URI prefix.
• --kind <list> — restrict chunk kinds (row_text, thread_aggregate, body, summary, vlm_description, …).
• --collapse — keep only the top-scoring chunk per object; later chunks from the same source are dropped, not merged. Recall stays as-is (the query still hits the same candidates), but the visible result count can fall below --top-k — collapse is a post-filter, not a re-rank. If you want N distinct objects, raise --top-k (e.g. --top-k 30 --collapse).

--all: when yes, when no

• ✅ Cross-source recall — "any past tickets / commits / RFCs / slack about X".
• ❌ You know the source — scope to slack://; postgres + jira + docs together aren't comparable.
• ⚠ More than ~5 registered connectors — ASK the user whether to fan out widely or scope to the 2-3 likeliest sources first.

7. Decision tree — pick the smallest useful tool

mfs search requires an explicit scope or --all.

8. Command cheat sheet

Search

mfs search "<query>" <path-or-uri>             # default: hybrid, top-k=10
mfs search "<query>" --all                     # whole namespace
mfs search "<query>" <path> --top-k 20         # more candidates
mfs search "<query>" <path> --mode semantic    # dense-only
mfs search "<query>" <path> --mode keyword     # BM25-only
mfs search "<query>" <path> --kind row_text    # restrict chunk kinds
mfs search "<query>" <path> --collapse         # keep top hit per object (post-filter, may return < top-k)

Grep

mfs grep "<pattern>" <path>          # pushdown -> BM25 -> linear

mfs grep is not grep. The three-tier dispatch is:

1. Pushdown — for structured connectors (postgres, mongo, jira, …) the pattern is shipped to the source as a LIKE / regex filter. Literal-exact, token-level; no regex on most structured connectors.
2. BM25 over indexed objects — for body/code/document chunks already in Milvus, the pattern is fed through the same sparse index search --mode keyword uses. That is a tokenized, ranked lookup, not a literal substring scan: an analyzer split like getUserId → get, user, id will rank userId as a hit; a CJK pattern with no analyzer match returns nothing even when the literal bytes are present. If you need "does this exact byte string appear anywhere?", mfs export the object and run rg locally — mfs grep has no "force linear over indexed objects" flag.
3. Linear scan — only for not-indexed files in scope (file connector before mfs add). True substring / regex.

For exact-exhaustive on a huge structured object, mfs export then local grep / rg.