Searching Documents

Searching documents with gaik

pip install "gaik[pg-vector-store,embedder,ranker]"
pip install "gaik[finnish-rag]"   # Finnish lemmatization

Hybrid search runs two arms that fail differently. The vector arm finds passages that answer in other words; the keyword arm finds exact tokens an embedding blurs — a section number, a standard's code, a product name. Reciprocal Rank Fusion (RRF) combines them by rank position, so a cosine similarity and a ts_rank_cd never have to share a scale.

Building that takes an afternoon. What costs weeks is that either arm can stop working without an error, and the page still fills with plausible results because the other arm keeps answering. Most of this skill is about making that failure visible.

Keep vector-only as a mode and make hybrid earn its place

Do not assume hybrid wins. On one 92-question set, vector-only matched hybrid's recall from top-10 upwards with a slightly better MRR at an eighth of the latency; hybrid won only at top-5. That metric asks whether one chunk was found, so it cannot see the case hybrid exists for — an exact code or section number — and on other corpora the keyword arm was the only thing that found those at all.

So ship three modes behind one parameter — hybrid, vector, keyword — and judge hybrid against vector-only on a query set that contains exact tokens. The modes double as the cheapest health check there is (below).

The gaik recipe

from gaik.software_components.config import get_openai_config
from gaik.software_components.RAG.embedder import Embedder
from gaik.software_components.RAG.pg_vector_store import PgVectorStore
from gaik.software_components.RAG.ranker import Ranker

# model= is the deployment name on Azure; its output size must equal embedding_dim
embedder = Embedder(get_openai_config(use_azure=True), model="text-embedding-3-small")

store = PgVectorStore(
    "postgresql://postgres:postgres@localhost:5432/search",
    embedding_dim=1536,
    fts_language="finnish",    # "simple" when a text_processor supplies lemmas (below)
    tsquery_mode="or",         # a sentence query still matches (default from 0.8.4)
    hnsw_ef_search=100,
)
store.setup()                                  # table, HNSW + GIN indexes, SQL functions
embeddings, docs = embedder.embed(chunks)      # chunks: list[langchain Document]
store.add(docs, embeddings)

query_vec = embedder.embed_query(query)
semantic = store.search_semantic(query_vec, top_k=50, threshold=0.0)
keyword = store.search_keyword(query, top_k=50)
hits = Ranker(expose_ranks=True).fuse(
    semantic, keyword, names=("semantic", "keyword"), weights=(1.0, 1.0), top_k=10
)

Embedder also accepts get_llm_config() from gaik.software_components.llm for native google/vertex, aitta, openai_compatible, or optional litellm. Native Google/Vertex needs gaik[llm-google]; LiteLLM needs gaik[llm-litellm] and an explicit provider-prefixed embedding_model (or LITELLM_EMBEDDING_MODEL). Aitta requires AITTA_API_KEY (also accepts AITTA_API_TOKEN or AITTA_TOKEN) and an explicit AITTA_EMBEDDING_MODEL, or an embedding_model config override naming an embedding model available on Aitta. A chat model is not an embedding model. Other compatible servers need their endpoint, API key, and an explicit embedding model. Use the same embedding model for indexing and queries, and match embedding_dim to its output. Native Anthropic does not provide embeddings.

For the Chroma-based end-to-end module, choose the three model stages independently:

from gaik.software_components.llm import get_llm_config
from gaik.software_modules.RAG_workflow import RAGWorkflow

workflow = RAGWorkflow(
    parser_config=get_llm_config("openai", model="gpt-6-luna"),
    embedding_config=get_llm_config("google", embedding_model="gemini-embedding-001"),
    answer_config=get_llm_config("aitta"),
)

Install gaik[rag-workflow,llm-google] for this example. The parser model must accept images; answer generation only needs chat. Each omitted stage uses shared api_config or the legacy OpenAI/Azure default. When all stages are explicit, no unrelated default credentials are loaded. RAGWorkflow uses Chroma; the PostgreSQL recipe above remains the path for PgVectorStore.

Fuse the two lists yourself instead of calling store.search_hybrid():

• the semantic list keeps its cosine similarities, which the relevance gate needs — search_hybrid() returns only RRF scores;
• expose_ranks=True writes rank_semantic and rank_keyword into every hit's metadata, so an arm that contributes nothing shows up per result;
• through gaik 0.7.2 search_hybrid() ignores tsquery_mode and always parses in websearch mode, which ANDs every term, and search_hybrid_weighted() fails on every call with column reference "id" is ambiguous.

From gaik 0.8.4 search_hybrid() also writes semantic_similarity (cosine) into every hit's metadata, so one call is enough when the ranks it returns are all you need.

Tune the weights and the RRF constant, do not assume them. Where the keyword arm alone reached 33.7% recall@15, equal weights at k=60 scored 8.7 points below vectors weighted 3× at k=20 (Ranker(rrf_k=...) sets k).

embedding_dim must equal the model's output, and at most 2,000. setup() builds an HNSW index on vector(N), which pgvector refuses above 2,000 dimensions. Without model=, Embedder falls back to a default that depends on the config helper, and two of them are too large: get_openai_config() gives text-embedding-3-large (3,072) and get_llm_config("google") or "vertex" gives gemini-embedding-001 (3,072). get_llm_config("openai") or "azure" gives EMBEDDING_MODEL, else text-embedding-3-small (1,536). Pass model= explicitly, and for a model above 2,000 pass vector_type="halfvec", which indexes up to 4,000 (gaik 0.8.4+; before that, pick a smaller model or build the schema yourself: references/postgres-without-gaik.md).

Three ways the keyword arm returns nothing, silently

1. The tsquery ANDs a sentence. websearch_to_tsquery and plainto_tsquery conjoin every term, so a nine-word question only matches a passage containing all nine stems — on real prose, none. Rewrite & to | and let ts_rank_cd rank by how many terms matched and how close together. In gaik that is tsquery_mode="or".
2. The index column is empty. On one production system the tsvector column existed as a plain nullable column instead of the GENERATED ALWAYS AS (…) STORED column the migration declared. Every row was NULL for about six months, the GIN index indexed nothing, and "hybrid" was vector search the whole time. Users noticed first.
3. The query is in another language than the index. Postgres full-text cannot cross languages: an English question against a finnish index took the keyword arm's recall@15 from 33.7% to 5.4%, and hybrid became vector-only. Translate the query into the corpus language as a question, not a keyword list — a keyword list helped the keyword arm but wrecked the vector arm and ended 14 points below not translating at all.

A fourth, rarer: zero-width characters (U+200B) pasted in from a CMS glue onto words. Postgres does not treat them as whitespace, so the word stays unstemmed and never matches. Strip them at ingest and at query time.

Check the arms are alive, against the live database. A unit test cannot see a column the migration declared correctly and the database never got. With gaik 0.8.4+, report = store.health() does it in one call: report.ok, and str(report) names each problem (a text_search column nothing fills, another fts_language than the index, lemmas built with other settings, rows either arm cannot see, missing indexes, session settings that did not stick). By hand, from a health endpoint or a startup check:

• rows with a NULL tsvector: 0, or exactly the rows deliberately left unindexed;
• a word copied out of a random stored chunk, searched in keyword mode, returns that chunk;
• one query in hybrid and in vector mode: identical result lists mean the keyword arm is contributing nothing.

Finnish and other inflected languages

Postgres' finnish snowball stemmer is inconsistent across a word family — tilinpäätös stems to tilinpäätös, tilinpäätöksen to tilinpäätöks — so a search for one form misses a passage containing the other, and nothing reports the miss. Lemmatize both sides with the same analyser. Measured on sentences queried in a different inflection than the document used: snowball matched 1 of 4, every prefix strategy 0–1 of 4, lemmas on both sides 4 of 4.

from gaik.software_components.RAG.finnish_text_processor import FinnishTextProcessor

# A named backend raises ImportError when it is missing; only "auto" falls back
# silently, to "simple", a tokenizer rather than a lemmatizer.
processor = FinnishTextProcessor(backend="pyvoikko", decompound=False)

store = PgVectorStore(dsn, embedding_dim=1536, fts_language="simple",
                      text_processor=processor, tsquery_mode="or")

Read references/finnish.md before indexing Finnish. The rules that each cost a debugging session:

• Pin the backend. backend="auto" picks native libvoikko where it is installed and something else where it is not, so a laptop indexes different lemmas than the container queries with, and the arm silently stops matching. pyvoikko is pure Python and runs everywhere, including images where libvoikko is not packaged.
• Start with decompound=False — gaik defaults to True. With both sides lemmatized, whole compounds already match, and splitting adds noise: arvonlisävero yields arvo, which reaches arvopaperi. Index and query must agree; after changing it, store.relemmatize() (0.8.4+) rebuilds the lemmas without re-embedding. Through 0.8.3 backend="auto" ignored decompound=False and split anyway.
• Long questions flood the lemma arm. A 25-word question became ~20 OR-ed lemmas that matched 78–79% of one corpus, and Hit@1 fell from 3/5 to 2/5 while short-term queries improved sharply. Split long questions into short topic queries before searching.
• No unaccent. In Finnish, ä/a and ö/o are different letters: tähti/tahti, sää/saa.