Ak Cloud Deploy

Deploy to Cloud

Use this skill to deploy your Agent Kernel project to AWS, Azure, or GCP.

Instructions for the Agent

When the user wants cloud deployment, follow this workflow.

Step 1: Identify the Project

Check for:

• pyproject.toml with agentkernel dependency
• agent entry file (app.py, lambda.py, or similar)
• config.yaml

If missing, suggest ak-init first.

Step 2: Ask Deployment Questions

1. Target platform: AWS, Azure, GCP, or Kubernetes (on-prem / baremetal / EKS via the Helm chart; see the On-Prem / Kubernetes section)?
2. Runtime mode:

• Serverless
• Containerized

3. Execution pattern (AWS only):

• Synchronous HTTP (rest_sync, supports standard or queue/scalable mode; AWS serverless or containerized)
• Asynchronous REST (rest_async, queue/scalable mode; AWS serverless or containerized)
• WebSocket full-response (async, works with or without queue mode — queue_mode = false runs the agent inline, queue_mode = true enqueues to a separately-scalable Agent Runner) — AWS serverless or AWS containerized/ECS
• WebSocket token streaming (stream, works with or without queue mode, same as async above) — AWS serverless or AWS containerized/ECS; also available via SSE (POST /api/v1/chat with execution.mode: stream, no Terraform changes required) wherever the built-in FastAPI REST server runs: local/self-hosted, AWS ECS single-container REST, Azure Container Apps, GCP Cloud Run — not on AWS Lambda or Azure Functions, which use WebSocket instead

4. Scalability (AWS serverless only): standard or queue/scalable mode?
5. Session store: Redis, Valkey (AWS only), DynamoDB (AWS), Cosmos DB (Azure), Firestore (GCP)?
6. Security: custom authorizer required (AWS serverless only)?
7. Resource naming: a single prefix applied to every resource name (e.g. myapp-dev-chat).

Step 3: Choose the Correct Terraform Module

Use official modules:

• AWS serverless: yaalalabs/ak-serverless/aws
• AWS containerized: yaalalabs/ak-containerized/aws
• Azure serverless: yaalalabs/ak-serverless/azurerm
• Azure containerized: yaalalabs/ak-containerized/azurerm
• GCP serverless: yaalalabs/ak-serverless/google
• GCP containerized: yaalalabs/ak-containerized/google

Use current module version (0.9.3) unless user requests another.

Kubernetes does not use Terraform: the Helm chart lives at ak-deployment/ak-k8s/chart in the Agent Kernel repository and is published as an OCI artifact (oci://ghcr.io/yaalalabs/charts/agent-kernel, versioned with each release). Install the published chart, as in the Kubernetes section below, not a repository checkout.

All modules are provider-agnostic: they declare required_providers but do not configure them internally. Configure each provider (aws/docker, azurerm, or google/google-beta/docker) in the root module and pass it explicitly via the module's providers = { ... } argument, as shown in the examples below. Azure's containerized module builds and pushes its image via a nested submodule with its own internal docker provider, so no docker provider needs to be configured or passed by the caller there.

AWS-only features in this skill:

• execution_mode
• queue_mode
• authorizer

Azure examples use the provider module's standard top-level inputs (function_name, package_path, container_port, publisher_email) rather than AWS serverless execution modes.

GCP modules use Cloud Run for both serverless (scale-to-zero, min_instance_count=0) and containerized (always-on, min_instance_count≥1) modes. Both use agentkernel.gcp.CloudRun as the entry point.

Step 4: Align App Dependencies and Session Config

When the user selects a session store, always update both app dependencies and config.yaml.

Redis sessions

• Dependency extras: include redis
• Typical dependency example:

dependencies = [
  "agentkernel[openai,api,redis]>=0.9.3"
]

• config.yaml session block:

session:
  type: redis
  namespace: chat
  redis:
    host: ${REDIS_HOST}
    port: 6379
    db: 0

Valkey sessions (AWS)

Valkey is the open-source, Linux Foundation-governed fork of Redis. It is wire-compatible with Redis and available on AWS ElastiCache at a lower price point than the Redis OSS engine. Agent Kernel treats it as a first-class session and response store backend on AWS.

• Dependency extras: include valkey
• Typical dependency example:

dependencies = [
  "agentkernel[openai,api,aws,valkey]>=0.9.3"
]

• config.yaml session block:

session:
  type: valkey
  valkey:
    url: ${AK_SESSION__VALKEY__URL}  # valkeys:// for SSL
    ttl: 604800
    prefix: "ak:sessions:"

• Terraform provisions the cluster with create_valkey_cluster = true (serverless and containerized) and injects AK_SESSION__VALKEY__URL — session.type: valkey must still be set in config.yaml since the type itself is never injected.
• For queue-mode response storage, set create_valkey_response_store = true (serverless only) and execution.response_store.type: valkey in config.yaml.

DynamoDB sessions (AWS)

• Dependency extras: include aws
• Typical dependency example:

dependencies = [
  "agentkernel[openai,api,aws]>=0.9.3"
]

• config.yaml session block:

session:
  type: dynamodb
  dynamodb:
    table_name: ${DYNAMODB_MEMORY_TABLE}
    region: ${AWS_REGION}

Cosmos DB sessions (Azure)

• Dependency extras: include azure
• Typical dependency example:

dependencies = [
  "agentkernel[openai,api,azure]>=0.9.3"
]

• config.yaml session block:

session:
  type: cosmosdb
  cosmosdb:
    endpoint: ${AZURE_COSMOS_ENDPOINT}
    database_name: ${AZURE_COSMOS_DATABASE}
    container_name: ${AZURE_COSMOS_CONTAINER}

Firestore sessions (GCP)

• Dependency extras: include gcp
• Typical dependency example:

dependencies = [
  "agentkernel[openai,api,gcp]>=0.9.3"
]

• config.yaml session block:

session:
  type: firestore
  firestore:
    collection_name: ${AK_SESSION__FIRESTORE__COLLECTION_NAME}
    project_id: ${PROJECT_ID}
    ttl: 604800

• The Terraform module injects AK_SESSION__TYPE=firestore and AK_SESSION__FIRESTORE__COLLECTION_NAME automatically when Firestore is enabled.
• A TTL policy must be set on the Firestore collection pointing to the expiry_time field for automatic document expiry.

If the Terraform module creates the backing store (for example create_redis_cluster = true, create_valkey_cluster = true, or create_dynamodb_memory_table = true), make sure output values are wired to the app environment variables used by config.yaml.

Deploying Conversation Thread Storage

Conversation threads (ak-add-capabilities) deploy the same way sessions do: the app mounts AgentThreadRequestHandler (which is what enables the feature; the self-hosted REST API is the only surface that records threads — queue-mode runners and the other deployment adapters do not) and declares thread.type in config.yaml, and Terraform provisions the backend, injecting only the connection detail. Terraform never sets AK_THREAD__TYPE itself.

• AWS (serverless + containerized): create_dynamodb_thread_table = true provisions a DynamoDB table (partition session_id, sort sk, TTL on expiry_time) and injects AK_THREAD__DYNAMODB__TABLE_NAME.
• GCP (serverless + containerized): create_firestore_thread_collection = true requires create_firestore_database = true, reuses that database, and injects AK_THREAD__FIRESTORE__COLLECTION_NAME, __PROJECT_ID, and __DATABASE_ID. No new resource is provisioned — the collection is created on first write.
• Redis / Valkey: no dedicated Terraform flag — reuse whatever create_redis_cluster / create_valkey_cluster already provisions and declare thread: {type: redis} (or valkey) with the cluster URL in config.yaml.

Setting the Terraform flag without declaring thread.type in config.yaml silently leaves threads on the non-durable in-memory backend (any AK_THREAD__* var materialises AKConfig.thread, but type still defaults to in_memory) — always pair the flag with the matching thread.type.

Deploying Scheduled Tasks

Scheduled tasks (ak-add-capabilities) deploy on the same two-step split: the app declares schedule.provider.type and schedule.store.type in config.yaml, and Terraform provisions the backends, injecting only the coordinates. Terraform never sets AK_SCHEDULE__PROVIDER__TYPE or AK_SCHEDULE__STORE__TYPE.

Scheduling requires queue mode — occurrences are delivered into the input queue, so queue_mode = true is mandatory on AWS. enable_scheduling without it is rejected.

• AWS (serverless + containerized): enable_scheduling = true provisions an EventBridge Scheduler schedule group and the execution role Scheduler assumes to deliver triggers to the input queue, grants both the request/REST role and the agent-runner role scheduler:CreateSchedule|UpdateSchedule|DeleteSchedule|GetSchedule plus iam:PassRole on that role, and injects AK_SCHEDULE__PROVIDER__EVENTBRIDGE__GROUP_NAME, __ROLE_ARN and __QUEUE_ARN. It also flips the input queue to content-based deduplication, because Scheduler cannot set a MessageDeduplicationId (application senders are unaffected — their explicit id takes precedence).
• AWS (serverless + containerized): create_dynamodb_schedule_table = true provisions a DynamoDB table (partition task_id, no sort key, no GSI, TTL on expiry_time) and injects AK_SCHEDULE__STORE__DYNAMODB__TABLE_NAME.
• Redis / Valkey task store: no dedicated Terraform flag — reuse whatever create_redis_cluster / create_valkey_cluster already provisions and declare schedule: {store: {type: redis}} (or valkey) with the cluster URL in config.yaml.
• Azure / GCP: no scheduling provider ships for these clouds yet. The local provider only works in a single always-on process with the in_memory transport and store, which no cloud deployment mode gives you.

queue_mode                     = true
enable_scheduling              = true
create_dynamodb_schedule_table = true

schedule:
  provider:
    type: eventbridge
  store:
    type: dynamodb

Setting the Terraform flags without declaring both types in config.yaml silently leaves scheduling on the in-process local provider and in_memory store (any AK_SCHEDULE__* var materialises AKConfig.schedule, but the types still default), while the provisioned group and table sit unused — always pair the flags with the matching types. The reverse is safe: declaring the types without the flags fails at startup with an AKConfigError on the missing group_name / role_arn / queue_arn.

The blocks above enable deferring and the agent tools. The /api/v1/schedules management routes are not mounted from config — on ECS queue mode, pass the handler to the IO container's entrypoint, where it is served alongside the queue-producing chat route:

# app_rest_service.py
from agentkernel.aws import ECSIOHandler
from agentkernel.schedule import ScheduleRESTRequestHandler

if __name__ == "__main__":
    # add authoriser=... to scope listings to the caller; without one the routes are open
    ECSIOHandler.run(handlers=[ScheduleRESTRequestHandler()])

Remember to list the schedule paths in gateway_endpoints — the gateway only proxies paths it is told about.

New outputs on both AWS stacks (null unless the matching flag is set): schedule_group_name, schedule_group_arn, scheduler_execution_role_arn, schedule_table_name, schedule_table_arn.

Deploying Secret Resolution (AWS SSM)

Secret resolution (ak-add-capabilities) follows the same split: the app declares secret.provider.type: aws_ssm in config.yaml and resolves keys with SecretManager.current().get("OPENAI_API_KEY"); Terraform grants read access and injects only the deployment scope. Terraform never sets AK_SECRET__PROVIDER__TYPE.

• AWS (serverless + containerized): ssm_enabled = true grants only the tier that runs the agent (the agent runner when queue_mode = true; otherwise the serverless request handler / containerized REST service) ssm:GetParameter on arn:aws:ssm:<region>:<account>:parameter/ak/<prefix>/* and injects AK_SECRET__PREFIX = <prefix> (the module's own prefix). The response handler and WebSocket connection handler never get SSM access, so resolve secrets in agent-runner code only.
• Terraform does not create the parameters — create each one yourself as a SecureString (AWS-managed alias/aws/ssm key) at /ak/<prefix>/<key lowercased>, e.g. aws ssm put-parameter --name /ak/<prefix>/openai_api_key --type SecureString --value sk-....
• Drop the key from environment_variables: a set, non-empty environment variable always wins over SSM, so leaving OPENAI_API_KEY there bypasses the store.
• The app needs the aws extra (agentkernel[aws]) for the aws_ssm provider.

ssm_enabled = true

secret:
  provider:
    type: aws_ssm
  # prefix is injected by Terraform as AK_SECRET__PREFIX

from agentkernel.secret import SecretManager
from agents import set_default_openai_key

set_default_openai_key(SecretManager.current().get("OPENAI_API_KEY"))

Declaring aws_ssm without ssm_enabled (and without setting secret.prefix yourself) fails at startup with an AKConfigError on the missing prefix; with a prefix but no IAM grant, lookups raise SecretError (AccessDenied). Setting ssm_enabled without declaring the provider leaves resolution on the env provider. See examples/aws-serverless/openai and examples/aws-containerized/openai-dynamodb-scalable.

AWS Serverless (Lambda + API Gateway)

Agent Code Pattern

Use Lambda handler entrypoint:

from agentkernel.aws import Lambda
from agentkernel.openai import OpenAIModule

# Register agents with your framework module
OpenAIModule([...])

handler = Lambda.handler

A) Basic Synchronous REST (rest_sync)

Use this for straightforward request/response APIs.

This is the single-Lambda pattern: use request_handler plus any gateway_endpoints. Do not add agent_runner or response_handler unless you are using queue mode.

module "serverless_agents" {
  source  = "yaalalabs/ak-serverless/aws"
  version = "0.9.3"

  prefix               = var.prefix
  product_display_name = "AK Serverless"
  region               = var.region

  execution_mode = "rest_sync"

  request_handler = {
    function_name        = "chat-handler"
    function_description = "AK request handler"
    handler_path         = "lambda.handler"
    package_type         = "Image"
    package_path         = "../dist"
    timeout              = 45
    memory_size          = 256
    environment_variables = {
      OPENAI_API_KEY = var.openai_api_key
    }
  }

  gateway_endpoints = [
    { path = "app", method = "GET" },
    { path = "app_info", method = "POST" }
  ]
}

B) Scalable Queue Mode (rest_sync or rest_async)

Use this for high throughput and long-running requests.

This is the multi-artifact pattern used by the scalable example: a request handler zip, an agent-runner image, and a response-handler zip.

Each Lambda can use one of three package_type values:

package_path and lambda_package_s3/ecr_image_uri are mutually exclusive — set only one.

Development / local build (build artifacts locally before running terraform apply):

module "serverless_agents" {
  source  = "yaalalabs/ak-serverless/aws"
  version = "0.9.3"

  prefix             = var.prefix
  region             = var.region
  product_display_name = "AK Scalable REST"

  queue_mode     = true
  execution_mode = "rest_sync" # can also be "rest_async"

  create_dynamodb_memory_table   = true
  create_dynamodb_response_store = true

  # Uncomment with `schedule: {provider: {type: eventbridge}, store: {type: dynamodb}}` in config.yaml
  # enable_scheduling              = true
  # create_dynamodb_schedule_table = true

  request_handler = {
    function_name        = "request-handler"
    function_description = "Receives REST requests"
    handler_path         = "lambda_request_handler.handler"
    package_type         = "LocalZip"
    package_path         = "../dist_request_handler.zip"
    timeout              = 45
    memory_size          = 256
    environment_variables = {
    OPENAI_API_KEY = var.openai_api_key
    }
  }

  agent_runner = {
    function_name        = "agent-runner"
    function_description = "Processes queued requests"
    handler_path         = "lambda_agent_runner.handler"
    package_type         = "Image"
    package_path         = "../dist_agent_runner"
    timeout              = 45
    memory_size          = 512
    environment_variables = {
      OPENAI_API_KEY = var.openai_api_key
    }
  }

  response_handler = {
    function_name        = "response-handler"
    function_description = "Handles async response completion"
    handler_path         = "lambda_response_handler.handler"
    package_type         = "LocalZip"
    package_path         = "../dist_response_handler.zip"
    timeout              = 45
    memory_size          = 256
  }

  queue_config = {
    input_queue_visibility_timeout         = 60
    input_queue_max_receive_count          = 3
    input_queue_create_dlq                 = false
    input_queue_message_retention_seconds  = 300
    output_queue_visibility_timeout        = 60
    output_queue_max_receive_count         = 3
    output_queue_create_dlq                = false
    output_queue_message_retention_seconds = 300
    batch_size                             = 10
    maximum_batching_window_in_seconds     = 0
  }
}

Production: external artifact sources (S3 / ECR)

Build and push artifacts in CI/CD, then point Terraform at them so terraform apply contains no local build step:

  request_handler = {
    function_name    = "request-handler"
    handler_path     = "lambda_request_handler.handler"
    package_type     = "S3Zip"
    lambda_package_s3 = {
      bucket     = "my-lambda-packages-bucket"
      key        = "dist_request_handler.zip"
      version_id = "<object-version-id>" # from your versioned bucket → redeploys work (#548)
    }
    timeout     = 45
    memory_size = 256
    environment_variables = { OPENAI_API_KEY = var.openai_api_key }
  }

  agent_runner = {
    function_name = "agent-runner"
    handler_path  = "lambda_agent_runner.handler"
    package_type  = "Image"
    ecr_image_uri = "123456789012.dkr.ecr.us-west-2.amazonaws.com/agent-runner:latest"
    timeout       = 45
    memory_size   = 512
    environment_variables = { OPENAI_API_KEY = var.openai_api_key }
  }

  response_handler = {
    function_name    = "response-handler"
    handler_path     = "lambda_response_handler.handler"
    package_type     = "S3Zip"
    lambda_package_s3 = {
      bucket     = "my-lambda-packages-bucket"
      key        = "dist_response_handler.zip"
      version_id = "<object-version-id>" # from your versioned bucket → redeploys work (#548)
    }
    timeout     = 45
    memory_size = 256
  }

See examples/aws-serverless/scalable-openai for a complete working example of this pattern.

Terraform only replaces a Lambda's code when s3_bucket, s3_key, s3_object_version, or source_code_hash changes. If you re-upload to the same S3 key in an unversioned bucket, none of those change and the function keeps running the old code. To make S3Zip redeploys reliable when pointing at your own artifact (lambda_package_s3): enable versioning on the bucket holding the ZIPs, then pass the uploaded object's version through lambda_package_s3.version_id.

Queue mode config.yaml (bundled into every Lambda package — execution.mode, queue URLs, table names, and max_receive_count are all injected automatically by Terraform as environment variables; only set values that are NOT injected):

# For rest_sync or rest_async
execution:
  response_store:
    type: dynamodb  # or redis / valkey — not injected, must be set here
    retry_count: 5
    delay: 5
session:
  type: dynamodb  # or redis / valkey — not injected, must be set here

• rest_sync: request handler sends to queue, polls the response store until the result is available, then returns it synchronously.
• rest_async: request handler returns {status: "ACCEPTED", request_id} immediately; the client polls GET /api/{version}/{endpoint} with {"request_id": "<id>"} in the JSON body.
• SQS visibility timeout for each queue must be >= the corresponding Lambda timeout.

Required pyproject.toml extras for queue mode:

dependencies = [
  "agentkernel[openai,api,aws]>=0.9.3"  # include 'redis' if using Redis, or 'valkey' if using Valkey session/response store
]

C) WebSocket Async (async)

Use this for realtime bidirectional interactions.

This follows the current websocket example shape: the request handler stays on the module's top-level Lambda inputs, then queue workers and WebSocket handlers are added via nested blocks.

module "serverless_agents" {
  source  = "yaalalabs/ak-serverless/aws"
  version = "0.9.3"

  prefix               = var.prefix
  region               = var.region
  product_display_name = "AK WebSocket Serverless Example"

  queue_mode     = true
  execution_mode = "async"

  create_redis_cluster           = true
  create_dynamodb_response_store = true

  request_handler = {
    function_name        = "request-handler"
    function_description = "Receives WebSocket requests"
    handler_path         = "lambda_request_handler.handler"
    package_type         = "LocalZip"
    package_path         = "../dist_request_handler.zip"
    timeout              = 45
    memory_size          = 256
    environment_variables = {
      OPENAI_API_KEY = var.openai_api_key
    }
  }

  agent_runner = {
    function_name        = "agent-runner"
    function_description = "Processes chat tasks"
    handler_path         = "lambda_agent_runner.handler"
    package_type         = "Image"
    package_path         = "../dist_agent_runner"
    timeout              = 45
    memory_size          = 512
    environment_variables = {
      OPENAI_API_KEY = var.openai_api_key
    }
  }

  response_handler = {
    function_name        = "response-handler"
    function_description = "Sends responses to connections"
    handler_path         = "lambda_response_handler.handler"
    package_type         = "LocalZip"
    package_path         = "../dist_response_handler.zip"
    timeout              = 45
    memory_size          = 256
  }

  ws_connection_handler = {
    function_name        = "ws-connection-handler"
    function_description = "Handles $connect/$disconnect"
    handler_path         = "lambda_ws_connection_handler.handler"
    package_path         = "../dist_ws_connection_handler.zip"
    timeout              = 45
    memory_size          = 256
  }

  ws_routes = [
    { route = "app" },
    { route = "app_info" }
  ]
}

WebSocket lambda_ws_connection_handler.py — implement AuthValidator to validate the bearer token on $connect:

import jwt
import os
from agentkernel.auth import AuthValidator, ValidationResult
from agentkernel.aws import WebsocketConnectionHandler


class MyAuthValidator(AuthValidator):
    def validate(self, token: str) -> ValidationResult:
        try:
            payload = jwt.decode(
                token,
                os.environ["JWT_SECRET"],
                algorithms=["HS256"],
                issuer=os.environ["JWT_ISSUER"],
                audience=os.environ["JWT_AUDIENCE"],
            )
            user_id = payload.get("userId", "")
            if user_id:
                return ValidationResult(is_valid=True, claims={"userId": user_id})
            return ValidationResult(is_valid=False, error_msg="userId claim missing")
        except Exception as e:
            return ValidationResult(is_valid=False, error_msg=str(e))


handler = WebsocketConnectionHandler.set_auth_validator(MyAuthValidator()).handler

WebSocket lambda_request_handler.py — routes are registered by route name (no leading slash, no HTTP method) because the WebSocket API uses $request.body.route for dispatch:

import json
from agentkernel.aws import Lambda


@Lambda.register("app")               # WebSocket route name, no slash/method
def app_handler(event, context):
    return {"response": "Hello from app route"}


@Lambda.register("app_info")
def app_info_handler(event, context):
    payload = json.loads(event.get("body") or "{}")
    return {"response": "Hello from app_info route"}


handler = Lambda.handler

Contrast with REST mode where routes use a leading slash and HTTP method:

@Lambda.register("/app", method="GET")    # REST mode: path + method

WebSocket config.yaml (bundled into every Lambda package — execution.mode, websocket_api.chat_route, queue URLs, and connection table name are all injected automatically by Terraform; only set values that are NOT injected):

session:
  type: redis                   # or valkey / dynamodb — not injected, must be set here
  redis:
    prefix: "ak:myapp:"

• Authentication is mandatory for WebSocket mode. AuthValidator.validate() must return claims["userId"]; connections without a valid token are rejected at $connect.
• The authorizer Terraform block is not supported in WebSocket mode — leave it unset.
• Only LocalZip is supported for ws_connection_handler.package_path.
• ws_routes custom routes must have a matching @Lambda.register("route_name") entry. ws_chat_route is the built-in AI chat route — it is handled internally by the framework and does not need a @Lambda.register entry.

Required pyproject.toml extras for WebSocket mode:

dependencies = [
  "agentkernel[openai,api,aws,redis,auth]>=0.9.3"
]

D) WebSocket Token Streaming (stream)

Use this when the client should receive each stream event as soon as it is produced, instead of waiting for the full response.

Same Terraform shape as WebSocket Async (request_handler, agent_runner, response_handler, ws_connection_handler, ws_routes) — only execution_mode changes:

module "serverless_agents" {
  source  = "yaalalabs/ak-serverless/aws"
  version = "0.9.3"

  prefix               = var.prefix
  region               = var.region
  product_display_name = "AK Streaming WebSocket Example"

  queue_mode     = true
  execution_mode = "stream"

  create_redis_cluster           = true
  # create_redis_response_store, create_valkey_response_store, and create_dynamodb_response_store must stay false/unset:
  # WebSocket modes (async/stream) push responses over the connection and Terraform
  # validation fails if a response store is enabled for them.

  request_handler       = { ... } # same shape as async mode
  agent_runner          = { ... } # runs ServerlessStreamAgentRunner, streams chunks to output queue
  response_handler      = { ... } # broadcasts each chunk as a STREAM_CHUNK message
  ws_connection_handler = { ... }
  ws_routes             = [ { route = "app" }, { route = "app_info" } ]
}

config.yaml — the only required setting beyond WebSocket async mode is the execution mode itself:

execution:
  mode: stream

This makes ServerlessAgentRunner.handle() dispatch to ServerlessStreamAgentRunner (queue mode) — no code change needed in the agent runner Lambda beyond the standard Lambda.handler entrypoint.

Message format — clients receive a sequence of STREAM_CHUNK messages instead of one CHAT_RESPONSE:

{"type": "STREAM_CHUNK", "delta": "Hello", "done": false, "session_id": "user-1"}
{"type": "STREAM_CHUNK", "delta": " world", "done": false, "session_id": "user-1"}
{"type": "STREAM_CHUNK", "delta": "!", "done": true, "session_id": "user-1"}

On an unrecoverable error, the final chunk carries error instead of delta, with done: true.

• Queue disabled (queue_mode = false): the request handler Lambda streams tokens directly to the WebSocket client without SQS.
• create_redis_response_store / create_valkey_response_store / create_dynamodb_response_store must be false for stream (same constraint as async) — Terraform validation enforces this.
• See examples/aws-serverless/streaming-openai for a complete working example.

Containerized / direct streaming (no Terraform WebSocket setup): wherever the built-in FastAPI REST server runs — local/self-hosted, AWS ECS single-container REST, Azure Container Apps, GCP Cloud Run — SSE event streaming can be enabled by setting execution.mode: stream in config.yaml. POST /api/v1/chat and /api/v1/chat-multipart then return text/event-stream responses instead of JSON — no queue or WebSocket infrastructure is required for this mode. Not available on AWS Lambda or Azure Functions (serverless), which use WebSocket for streaming instead.