Extracting Structured Data

Extracting structured data with gaik

pip install "gaik[extract]"
pip install "gaik[vision-extract]"   # PDF/image extraction
# Add llm-google for shared Google/Vertex configs, or llm-litellm for LiteLLM.

Two entry points, and the choice is about where the document is in its lifecycle:

• VisionExtractor — PDF or image straight to structured data in one call. It renders the pages, so it sees layout, stamps, and handwriting.
• DataExtractor — takes document text you already parsed. Pair it with the parsing-documents skill.

from gaik.software_components.vision_extractor import VisionExtractor
from gaik.software_components.llm import get_llm_config

config = get_llm_config("azure")   # or another supported vision/structured-output model
result = VisionExtractor(api_config=config).extract(
    file_paths=["invoice.pdf"],
    user_requirements="Supplier name, invoice number, and every line item with quantity and unit price.",
)
result.data          # dict
result.usage         # None on the shared structured-output path; do not invent token cost

Arguments are keyword-only. With no extraction_model, gaik generates the schema from the prose (an extra LLM call); pass schema_dir=... and it caches the generated schema to disk and reuses it, which makes runs both cheaper and reproducible.

For text extraction, DataExtractor(config=get_llm_config(provider)) accepts the shared configuration from gaik.software_components.llm. Choose azure, openai, google, vertex, anthropic, anthropic_foundry, aitta, openai_compatible, or optional litellm; the model must support structured output. Native Google/Vertex and Anthropic need the llm-google and llm-anthropic extras respectively. LiteLLM requires gaik[llm-litellm] and a provider-prefixed model identifier. Aitta uses AITTA_API_KEY (aliases AITTA_API_TOKEN, AITTA_TOKEN) and AITTA_MODEL, defaulting to google/gemma-4-31b-it. Other compatible servers require OPENAI_API_KEY, OPENAI_BASE_URL, and OPENAI_MODEL, or equivalent explicit config overrides.

VisionExtractor(api_config=config) uses the same interface, renders PDF pages as PNG images, and generates schemas with that provider. Its model must support both images and structured output. SchemaGenerator(config=config) supports the shared provider path; use a hand-built Pydantic schema and ExtractionRequirements when you already know the schema. LLMJudge(config=config) also accepts the shared config; visual judging requires images, while judge_text_pair(...) only needs a chat model. Keep credentials in runtime configuration or environment variables, not generated source code.

Choose providers separately for audio, vision and extraction

The modules accept stage configs so transcription can stay on a supported audio service while text extraction uses Aitta or another chat provider:

pip install "gaik[audio-to-structured-data,documents-to-structured-data]"

from gaik.software_modules.audio_to_structured_data import AudioToStructuredData
from gaik.software_modules.documents_to_structured_data import DocumentsToStructuredData

audio_pipeline = AudioToStructuredData(
    transcription_config=get_llm_config("azure"),
    extraction_config=get_llm_config("aitta"),
)
document_pipeline = DocumentsToStructuredData(
    parser_config=get_llm_config("openai", model="gpt-6-luna"),
    extraction_config=get_llm_config("aitta"),
)

Omitted stage configs use api_config; without a shared config, missing stages load the legacy OpenAI/Azure default. Supplying every stage config avoids loading unrelated default credentials. Audio transcription remains OpenAI/Azure-only: a chat provider selection, including LiteLLM, does not make toolkit transcription support that provider.

Fix the schema, not the prompt

This is the highest-leverage thing to know, because the instinct is the opposite.

Five deliberately different extraction prompts — stripped-down, "read the characters first", a role frame, "walk the pages before answering" — produced identical F1 on every document and every repeat. One of them had been written specifically to counter a known corpus error and did not move a single value. The verbose variants cost about 30% more per document for exactly the same answer.

The reason is that a well-specified schema has already pinned the task: a description on each field, a closed object, and structured output constraining the response shape regardless of what the prose asks for. Prompt wording only has room to matter when the schema is vague — and then fixing the schema is still the better move.

So when extraction is wrong: add or sharpen field descriptions, tighten types, close the object. Do not spend the afternoon rewording the instruction.

The binding limit is property count, not schema size

Providers reject large schemas, but the thing being counted is the number of properties, not characters. Measured on one provider: 200 properties passed and 250 failed; a 300,000-character schema passed while carrying 100 properties.

$ref reuse does not help. The natural assumption is that a definition appearing once in $defs and referenced eight times counts once — it does not. Tested directly: 8 × 50 properties inline (18,263 characters) and the same thing via $ref (2,663 characters) were rejected identically. The schema is expanded before counting, so the refactor that shrinks it sevenfold on disk changes nothing.

The numbers are provider-specific and they move. The method is what to carry: generate synthetic schemas at increasing property counts and bisect until one is rejected. That takes about ten minutes and replaces a guess with a number.

Three failures that look like degrees of one problem

They are not, and the fix for two of them does nothing for the third:

Splitting the schema cannot help the third: however you divide the fields, one call is still asked to return the whole list. Staged and one-shot pipelines fail identically at a thousand rows. That one needs the document chunked into windows and results concatenated.

Only put fields in the schema that the document actually contains

Two traps, both of which produce numbers that look like model errors:

A field with no ground truth does not belong in the schema. Adding a unit field with expected value null because the source system does not record it means a correctly extracted "KG" scores as a mistake. Leave it out.

A field the document does not carry measures enrichment, not extraction. If the truth holds an internal part number and the page shows the supplier's code, reaching one from the other needs a lookup table. Scoring it grades the wrong system.

Both are caught the same cheap way: check every expected value against the document text when building the evaluation set.