Sift Kg

sift-kg: AI Second Brain

The knowledge graph is your persistent, structured memory of the user's world — entities, relationships, communities, and how they all connect. It is built from the user's documents and persists across sessions.

Treat the graph as your persistent understanding of the user's world. When the user asks about their projects, domain, or knowledge — check the graph first. It grounds your responses in real entities and relationships instead of guessing. You don't need to query it for every interaction, but for anything related to the user's work or knowledge, the graph should inform your answer.

Session Start: Orient Yourself

At the start of every session, orient from the graph:

sift info --json -o output/

This returns: domain, entity_types, entities (count), relations (count), documents_processed, merge_proposals and relation_review status (if dedup has been run), and narrative_generated. If no graph exists yet, go to Build the Graph below.

Then load the structural map:

sift topology -o output/

This returns:

• Communities — clusters of related entities with member counts and top entity IDs
• Bridges — entities connecting multiple communities, sorted by cross-community edge count
• Community connections — which clusters are linked and how strongly (shared edges, bridge count)
• Isolated — entities with no substantive connections

Keep this topology in mind for the session. Community labels and bridge entities help you navigate the user's knowledge. When relevant questions come up, you already have the structural context.

Note on community labels: Labels are auto-generated as "Community 1", "Community 2" unless sift narrate has been run (which generates descriptive LLM labels like "Palm Beach Elite Network"). To understand what a generically-labeled community represents, look at its top_entities list — these are entity IDs (e.g., person:harry_boyte, program:civic_architecture), not display names. Query interesting IDs with sift query to get full context.

Note on top_entities: These are entity IDs, not display names. Read them by convention: person:harry_boyte = a person named Harry Boyte, company:palantir_technologies = a company named Palantir Technologies.

Querying the Graph

Explore an Entity

sift query "topic or name" -o output/             # fuzzy name search
sift query "person:exact_entity_id" -o output/     # exact ID lookup
sift query "topic" --depth 2 -o output/            # 2-hop neighborhood
sift query "topic" -t PERSON -o output/            # filter by entity type

Returns: matched entity with community membership and bridge status, plus the full subgraph (nodes + edges) around it. If multiple entities match, the top result (by connection count) is returned with an other_matches list — re-query with the exact entity ID for a specific one.

Lightweight Lookup

sift search "name" --json -o output/                    # find entities
sift search "name" --json --relations -o output/         # include direct relations
sift search "name" --json --description -o output/       # include descriptions

Use search when you just need to find an entity or check if something exists. Use query when you need the full neighborhood subgraph.

Reasoning Patterns

Pattern: Ground Responses in the Graph

When the user asks about their work — "what do I know about X?", "tell me about my project Y" — always query the graph first before responding:

sift query "X" -o output/

Read the match info (community, bridge status, connections) and the subgraph. Base your answer on actual entities and relationships from the graph. If the entity isn't in the graph, say so — don't invent connections.

Pattern: Link Knowledge Islands

This is the highest-value reasoning pattern. It identifies opportunities to connect knowledge areas that are structurally separated.

Step 1: Get topology and identify disconnected or weakly connected community pairs.

sift topology -o output/

Look at community_connections. Community pairs with 0 shared_edges are completely disconnected — knowledge islands. Pairs with low shared_edges (1-3) are weakly connected.

Step 2: For each disconnected pair, examine what each community contains.

Read the top_entities (entity IDs) from each community in the topology output. Look at entity types — if both communities contain overlapping types (e.g., both have CONCEPT entities, or both have ORGANIZATION entities), there may be a semantic connection that isn't structurally represented yet.

Step 3: Query top entities from each side to find shared concepts.

sift query "top_entity_from_community_A" -o output/
sift query "top_entity_from_community_B" -o output/

Compare the two subgraphs. Look for:

• Shared relation types — if both subgraphs have FUNDED_BY or LOCATED_IN relations, the domains share a structural pattern
• Common neighbor entities — entities that appear in both neighborhoods are natural bridge candidates
• Conceptual overlap — entities in one subgraph that could logically relate to entities in the other

Step 4: Articulate the connection for the user.

Tell them: "Your [Community A topic] and [Community B topic] are currently disconnected in your knowledge graph. But [entity X] in the first area and [entity Y] in the second share [specific relationship or concept]. Connecting these could [specific value]."

The insight is not that the connection exists — the user might already sense it vaguely. The value is identifying which specific entities to connect and why now, grounded in the actual graph structure.