Gaik Toolkit

GAIK Toolkit

Current PyPI version: !python ${CLAUDE_SKILL_DIR}/scripts/fetch_pypi_readme.py --version

Python toolkit for knowledge extraction, capture, and generation. Use when working with:

• Structured data extraction from documents, PDFs, images, or audio
• Schema generation from natural language requirements
• Document parsing (PDF, DOCX, images)
• Audio/video transcription with Whisper + local Whisper backends (Finnish fine-tuned model)
• Transcript enhancement — two-pass LLM error correction
• Parallel transcription with FFmpeg chunking
• Text-to-speech generation
• Document classification
• Text-to-SQL: natural-language querying of PostgreSQL databases and CSV/Excel/Parquet files (DuckDB)
• RAG pipelines: embedder, vector store (Chroma / PostgreSQL), retriever, answer generator
• End-to-end pipelines: AudioToStructuredData, DocumentsToStructuredData, RAGWorkflow

Related Skills

This skill is the overview / reference. Two sibling skills handle workflows:

Quick Links

• Documentation: https://gaik-project.github.io/gaik-toolkit/
• Live Demo: https://gaik-demo.2.rahtiapp.fi/ (registration required)
• GitHub: https://github.com/GAIK-project/gaik-toolkit
• Source Code: implementation_layer/src/gaik/
• PyPI: https://pypi.org/project/gaik/

Repository Structure

Toolkit Demo App

Interactive web app at implementation_layer/toolkit_demo_app/. Next.js 16 + FastAPI (bun + uv).

• Live: https://gaik-demo.2.rahtiapp.fi/ (registration required)
• Dev: bun run dev:all (runs both frontend and API)
• See: Demo App Reference for full architecture, routes, and conventions

Documentation Website

Fumadocs/Next.js site at guidance_layer/website/. Content in .mdx files under content/docs/.

• Live: https://gaik-project.github.io/gaik-toolkit/
• Dev: pnpm dev (from guidance_layer/website/ -- uses pnpm, not bun)
• See: Docs Website Reference for content structure and editing guide

Installation

Install via pip with optional extras: pip install "gaik[extract]", pip install "gaik[all-cpu]", etc. See Installation Reference for all available extras and setup.

Environment Variables

Azure OpenAI (recommended):

AZURE_API_KEY=your-key
AZURE_ENDPOINT=https://your-resource.openai.azure.com/
AZURE_DEPLOYMENT=gpt-6-luna        # default when unset
AZURE_API_VERSION=2025-03-01-preview

OpenAI:

OPENAI_API_KEY=your-key
OPENAI_MODEL=gpt-6-luna            # default when unset

Other providers read their own variables (AITTA_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY, LITELLM_MODEL …); LLM_PROVIDER is the default for get_llm_config() called without a provider. See Building Blocks Reference.

Configuration Pattern

Two parallel surfaces. Pick the simpler one for OpenAI/Azure-only use cases; pick the multi-provider one for any other provider or when the same code must switch providers. Components take either dict in their config argument (config, api_config, openai_config …).

Legacy surface (OpenAI/Azure only — keeps working unchanged):

from gaik.software_components.config import get_openai_config, create_openai_client

config = get_openai_config(use_azure=True)   # Azure OpenAI
config = get_openai_config(use_azure=False)  # Standard OpenAI
client = create_openai_client(config)        # OpenAI/AzureOpenAI client

Multi-provider surface:

from gaik.software_components.llm import get_llm_config, create_llm_client

config = get_llm_config("aitta")   # openai, azure, aitta, openai_compatible, google, vertex,
                                   # anthropic, anthropic_foundry, litellm
client = create_llm_client(config) # ProviderClient with chat/chat_parsed/chat_stream/embed

• No extra: openai, azure, aitta (CSC's OpenAI-compatible API; default model google/gemma-4-31b-it, 600 s timeout for cold starts) and openai_compatible (explicit base_url and model, e.g. vLLM or Ollama).
• Extras: gaik[llm-anthropic], gaik[llm-google]; gaik[llm-litellm] for the optional litellm backend, which needs a provider-prefixed model such as azure/<deployment>. Native adapters stay the default; gaik[llm-all] installs all three.
• Audio (Transcriber, ParallelTranscriber, TextToSpeech) accepts only OpenAI/Azure and raises NotImplementedError for every other provider, Aitta and openai_compatible included.
• Vision (VisionParser, MultimodalParser, VisionExtractor) accepts any provider config; the model must accept images.
• Legacy dicts win: a config with use_azure keeps its backend whatever LLM_PROVIDER says, and a bare dict without provider or use_azure keeps its pre-0.8 routing (resolution rules).
• gpt-6-* are reasoning models: sampling options (temperature, top_p) work only with reasoning_effort="none", which gpt-6-astra does not offer, and the token limit is max_completion_tokens. Components translate this themselves; when calling a raw SDK client, pass the options through normalize_chat_kwargs from gaik.software_components.llm.parameters.

Building Blocks

Core classes in gaik.software_components.*. For detailed API and constructor parameters, see Building Blocks Reference.

Parser notes

Every parser ships a class and a module-level convenience function, and the two do not agree on return type — the class method gives you the text, the function gives you a metadata dict. Reaching for the shorter name is the easy mistake:

parser = PyMuPDFParser()
text = parser.parse_pdf("doc.pdf")        # -> str
result = parse_pdf("doc.pdf")             # -> dict, text lives under result["text_content"]

The same split applies to DocxParser.parse_docx / parse_docx, and every parse_document variant returns a dict on both the class and the function.

Transcriber notes

• Models: "whisper", "whisper-1", "gpt-4o-transcribe", "whisper_local"
• enhanced_transcript=True runs output through TranscriptEnhancer (two-pass LLM correction)
• whisper_local requires local_api_base + local_api_key; language="fi" selects Finnish fine-tuned model
• ParallelTranscriber uses FFmpeg chunking; requires ffmpeg + ffprobe on $PATH

SRT/VTT Utilities

from gaik.software_components.transcriber import segments_to_srt, segments_to_vtt, parse_srt, chunk_segments

Video Search Helpers

from gaik.software_components.RAG.pg_vector_store import PgVectorStore, ingest_video_segments, format_search_results