Agent skill

Add Source Sourcedb To Spanner

by GoogleCloudPlatform in GoogleCloudPlatform/DataflowTemplates

Guide for implementing a database source connector in the v2/sourcedb-to-spanner forward migration Dataflow template.

Apache-2.0Auto-check passedTesting & QA

Install Add Source Sourcedb To Spanner

skills CLI
$ npx skills add GoogleCloudPlatform/DataflowTemplates --skill add-source-sourcedb-to-spanner -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install GoogleCloudPlatform/DataflowTemplates add-source-sourcedb-to-spanner --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/GoogleCloudPlatform/DataflowTemplates.git skills-src && mkdir -p .claude/skills && cp -r skills-src/v2/sourcedb-to-spanner/.agents/skills/add-source-sourcedb-to-spanner .claude/skills/add-source-sourcedb-to-spanner && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
add-source-sourcedb-to-spanner
GitHub stars
1.3k
Token cost
~1.9k tokens
SKILL.md length
771 words
Files
1
Skills in repo
12
Repo updated
First seen
Licence
Apache-2.0

At a glance

Guide for implementing a database source connector in the v2/sourcedb-to-spanner forward migration Dataflow template.

  • Works in 6 steps: Mandatory Prerequisite Gate → Implement SrcToSpSourceConnector → Implement Source IO Wrapper Config… → …
  • Tasks that involve Unit testing
  • SKILL.md covers Overview, Prerequisites, Architectural Boundaries &… and Datatype Mapping Matrix…, plus 1 more section
  • Calls mvn

What it does

Add Source Sourcedb To Spanner is an agent skill from GoogleCloudPlatform/DataflowTemplates. Guide for implementing a database source connector in the v2/sourcedb-to-spanner forward migration Dataflow template. Details scope boundaries, prerequisites, connector implementations, registry registrations, unit testing, and smoke testing guidelines.

Its SKILL.md is about 1.9k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

It sits in Testing & QA, covering Unit testing. It works with Google Cloud, Google BigQuery and Java. The repository describes itself as: Cloud Dataflow Google-provided templates for solving in-Cloud data tasks. The licence is Apache-2.0.

When your agent uses it

  • Tasks that involve Unit testing

Example prompts

  • “/add-source-sourcedb-to-spanner”

Workflow steps

6 steps, taken from the step headings in SKILL.md.

  1. Mandatory Prerequisite Gate
  2. Implement SrcToSpSourceConnector
  3. Implement Source IO Wrapper Config Defaults and Schema Discovery
  4. Register Source in Options, Constants & Factory
  5. Unit Testing & Verification
  6. Mandatory Live Smoke Testing & Verification

What it can do on your machine

Read from SKILL.md and the folder at commit c95daba. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Shell commands in SKILL.md call:

    • mvn

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Add Source Sourcedb To Spanner loads about 1.9k tokens when it runs. Until then it costs about 71 tokens; SKILL.md has 771 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~71
When it runs · the whole SKILL.md, loaded when a task matches
~1.9k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from GoogleCloudPlatform/DataflowTemplates at commit c95daba, republished under its Apache-2.0 licence (© GoogleCloudPlatform). 771 words, ~1,852 tokens.

Download SKILL.mdSave it as .claude/skills/add-source-sourcedb-to-spanner/SKILL.md (or your agent's skills folder).
name
add-source-sourcedb-to-spanner
description
Guide for implementing a database source connector in the v2/sourcedb-to-spanner forward migration Dataflow template. Details scope boundaries, prerequisites, connector implementations, registry registrations, unit testing, and smoke testing guidelines.

Skill: Implement Source in SourceDb to Spanner Template

Overview

This skill provides a step-by-step procedure for adding support for a new database source connector to the v2/sourcedb-to-spanner forward migration template. It details prerequisites, scope boundaries, type mapping requirements, connector implementation, factory registration, unit testing, and smoke testing procedures.


Prerequisites

[!CRITICAL] MANDATORY PREREQUISITE GATE: Before running code searches, inspecting files, or executing unit tests, you MUST verify if the user provided the inputs below. If any required input is missing, STOP IMMEDIATELY and ask the user for clarification before proceeding further.

  1. Datatype Mapping File: Ask the user to provide or point to the Datatype Mapping Matrix for the new source database, defining the mappings between source database datatypes and Spanner (GoogleSQL and PostgreSQL dialects) datatypes or UnifiedMappingProvider.Type.
  2. Test Setup Details: Ask the user for the test environment details required for smoke testing, including:
    • Source Database Instance details (host/instance name, port, database name, credentials/connection method, and schema/namespace if applicable).
    • Target Spanner Instance & Database details.

Architectural Boundaries & Code Scope

All implementation for a new source connector MUST be strictly confined to:

  1. Source Connector Package: v2/sourcedb-to-spanner/src/main/java/com/google/cloud/teleport/v2/source/<source_type>/
    • Implementation of ISrcToSpSourceConnector (<Source>SrcToSpSourceConnector.java) or subclassing AbstractJdbcSrcToSpSourceConnector (for JDBC sources).
    • Dialect adapter, value mapping provider, and config defaults classes in package <source_type> (e.g., <Source>ConfigDefaults.java).
  2. Connector Factory: SourceConnectorFactory.java (v2/sourcedb-to-spanner/src/main/java/com/google/cloud/teleport/v2/source/SourceConnectorFactory.java)
    • Dynamic connector lookup in getSourceConnectorByDialect(), getSourceConnectorBySourceType(), and getSourceJdbcConnectorByDialect().
  3. Pipeline Options: SourceDbToSpannerOptions.java (v2/sourcedb-to-spanner/src/main/java/com/google/cloud/teleport/v2/options/SourceDbToSpannerOptions.java)
    • Source dialect constant definition (e.g., String <SOURCE>_SOURCE_DIALECT = "<SOURCE_DIALECT_NAME>";) and enum registration in @TemplateParameter.Enum.
  4. Shared Core Registries & Configs: v2/spanner-common
    • Registries and constant files in spanner-common (e.g. Constants.java / SourceConstants.java for public static final String <SOURCE>_SOURCE_TYPE = "<source_type>";).

Datatype Mapping Matrix Requirements

Consult the Datatype Mapping Matrix provided in the prerequisites to verify correct datatype conversion between the source database datatypes and UnifiedMappingProvider.Type / Spanner target datatypes. Ensure proper alignment for character, numeric, temporal, binary, boolean, JSON, and any other dialect-specific datatypes as specified in the mapping file.


Step-by-Step Implementation Workflow

Step 0: Mandatory Prerequisite Gate
  1. Verify Inputs: Inspect the request for the required inputs. If the Datatype Mapping Matrix and Test Setup Details are not provided in the user request, DO NOT run any inspection or execution tools. Stop and ask the user for the missing details first.
Step 1: Implement <Source>SrcToSpSourceConnector

Implement ISrcToSpSourceConnector (or extend AbstractJdbcSrcToSpSourceConnector for JDBC sources) in com.google.cloud.teleport.v2.source.<source_type>:

  1. Define type mappings between source datatypes and UnifiedMappingProvider.Type in getTypeMapping(). Ensure all the types mentioned in the datatype mapping are covered.
  2. For sources which have a beam IO library, use that library.For JDBC supported sources, implement the relevant JDBC connector methods similar to existing JDBC sources.
  3. Configure connection parameters and dialect-specific setup to work with Spanner target databases (GoogleSQL and PostgreSQL dialects).
  4. For the types marked as primary key supported in the data type mapping ensure an implementation for the splitter is provided.
Show full SKILL.md (318 more words)Show less
Step 2: Implement Source IO Wrapper Config Defaults and Schema Discovery
  1. Create source configuration defaults (e.g. <Source>ConfigDefaults.java), dialect adapter, and value mappings provider in package com.google.cloud.teleport.v2.source.<source_type>.
  2. Configure schema discovery, fetch size, connection pooling, and uniform partitioning parameters appropriate for the source database.
Step 3: Register Source in Options, Constants & Factory
  1. Add the source dialect constant in SourceDbToSpannerOptions.java and update @TemplateParameter.Enum:
    java
    String <SOURCE>_SOURCE_DIALECT = "<SOURCE_DIALECT_NAME>";
  2. Add the source type constant in Constants.java / SourceConstants.java in v2/spanner-common:
    java
    public static final String <SOURCE>_SOURCE_TYPE = "<source_type>";
  3. Register the connector in SourceConnectorFactory.java:
    • Update getSourceConnectorByDialect(...)
    • Update getSourceConnectorBySourceType(...)
    • Update getSourceJdbcConnectorByDialect(...) (for JDBC sources)
Step 4: Unit Testing & Verification

Execute Maven unit tests for the template:

bash
mvn test -pl v2/sourcedb-to-spanner \
  -Dtest=<Source>SrcToSpSourceConnectorTest,SourceConnectorFactoryTest

All unit tests must pass with BUILD SUCCESS.

Step 5: Mandatory Live Smoke Testing & Verification

[!CRITICAL] MANDATORY EXECUTION REQUIREMENT: Immediately after unit tests pass, you MUST AUTOMATICALLY PROCEED to execute live end-to-end smoke testing using the test environment details provided in the prerequisites. Do NOT stop, pause, or declare completion after unit testing without running the live smoke tests.

[!IMPORTANT] WORKER MACHINE TYPE DIRECTIVE: When submitting the Dataflow job for live smoke testing, you MUST explicitly specify a worker machine type of the correct size with at least 4 vCPUs (e.g., --worker-machine-type=n2-standard-4). Omitting this parameter will cause Dataflow job launch validation to fail with a machine specification policy violation.

  1. Environment Setup:
    • Connect to the source database instance and target Spanner database instance configured in the test setup.
  2. Populate Source Data:
    • Insert test rows covering all supported datatypes into the source database.
  3. Verify Replication / Migration Flow:
    • Launch the sourcedb-to-spanner Dataflow pipeline.
    • Query the target Spanner database tables to confirm that source rows are accurately migrated to destination Spanner tables.
  4. Retry Loop on Failure:
    • If any record fails to migrate or produces data discrepancies in Spanner, inspect error logs, modify connector, IO wrapper, or mapping code, rebuild, and re-test until all operations pass cleanly.

© GoogleCloudPlatform, Apache-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in v2/sourcedb-to-spanner/.agents/skills/add-source-sourcedb-to-spanner of GoogleCloudPlatform/DataflowTemplates.

Open the folder on GitHubat commit c95daba

Compare with similar skills

Add Source Sourcedb To Spanner next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Add Source Sourcedb To Spanner compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Add Source Sourcedb To Spanner this skillGoogleCloudPlatform/DataflowTemplates1.3k—~1.9kAutomated safety check: PassApache-2.0
Io ConnectorsKilo-Org/kilo-marketplace189—~1.3kAutomated safety check: PassApache-2.0
Add Library Testosama-raddad/FireCrasher147—~612Automated safety check: PassApache-2.0
Edt MCP Build TestDitriXNew/EDT-MCP295—~1.4kAutomated safety check: PassAGPL-3.0
Debug Surefireeclipse-rdf4j/rdf4j420—~2.4kAutomated safety check: PassBSD-3-Clause
Validationjosstei/maestro-orchestrate465—~2.3kAutomated safety check: PassApache-2.0

Similar skills

  • Io Connectors

    Kilo-Org/kilo-marketplace

    Guides development and usage of I/O connectors in Apache Beam.

    189 GitHub stars~1.3k tokensUpdated 8 days ago
    Testing & QAAuto-check passed
  • Add Library Test

    osama-raddad/FireCrasher

    Add or update a Robolectric JVM unit test for the FireCrasher library.

    147 GitHub stars~612 tokensUpdated 3 mo ago
    Testing & QAAuto-check passed
  • Edt MCP Build Test

    DitriXNew/EDT-MCP

    How to build the EDT-MCP Eclipse plugin (Tycho/Maven) and run its unit and e2e tests, plus the test conventions for this repo.

    295 GitHub stars~1.4k tokensUpdated today
    Testing & QAAuto-check passed
  • Debug Surefire

    eclipse-rdf4j/rdf4j

    Debug Maven Surefire unit tests by running them in JDWP "wait for debugger" mode (-Dmaven.surefire.debug) and attaching to the forked test JVM using jdb (preferred for CLI/agent debugging)…

    420 GitHub stars~2.4k tokensUpdated today
    Testing & QAAuto-check passed
  • Validation

    josstei/maestro-orchestrate

    Cross-cutting validation methodology for verifying phase outputs and project integrity

    465 GitHub stars~2.3k tokensUpdated today
    Testing & QAAuto-check passed
  • Minecraft Testing

    Jahrome907/minecraft-agent-skills

    Design and implement automated tests for current Minecraft 26.x or legacy 1.21.x mods and plugins using JUnit, MockBukkit, NeoForge Game Tests, or Fabric Game Tests.

    166 GitHub stars~3.8k tokensUpdated 24 days ago
    Testing & QAAuto-check passed

More from GoogleCloudPlatform/DataflowTemplates

All 12 skills in this repo
  • Smt E2E Dataflow Debugging

    GoogleCloudPlatform/DataflowTemplates

    Debugs logical errors and data discrepancies in Dataflow templates by launching jobs via Terraform and comparing source (e.g.

    1.3k GitHub stars~1.8k tokensUpdated today
    Auto-check passed
  • Smt Functional Testing

    GoogleCloudPlatform/DataflowTemplates

    Functionally tests local Dataflow pipeline changes against the main branch using ephemeral GCP resources and gated approvals.

    1.3k GitHub stars~2.8k tokensUpdated today
    Auto-check: notes
  • Add Integ Tests Datastream To Spanner

    GoogleCloudPlatform/DataflowTemplates

    Specific runner skill that delegates to the Template-Agnostic Meta-Test Orchestrator for the datastream-to-spanner (CDC) template.

    1.3k GitHub stars~572 tokensUpdated today
    Auto-check passed
  • Add Integ Tests Gcs Spanner Dv

    GoogleCloudPlatform/DataflowTemplates

    Specific runner skill that creates integration tests for the gcs-spanner-dv (Data Validation) template.

    1.3k GitHub stars~517 tokensUpdated today
    Auto-check passed
  • Add Integ Tests Sourcedb To Spanner

    GoogleCloudPlatform/DataflowTemplates

    Specific runner skill that delegates to the Template-Agnostic Meta-Test Orchestrator for the sourcedb-to-spanner (Bulk) template.

    1.3k GitHub stars~475 tokensUpdated today
    Auto-check passed
  • Add Integ Tests Spanner To Sourcedb

    GoogleCloudPlatform/DataflowTemplates

    Specific runner skill that delegates to the Template-Agnostic Meta-Test Orchestrator for the spanner-to-sourcedb (Reverse Migration) template.

    1.3k GitHub stars~479 tokensUpdated today
    Auto-check passed

Categories

Questions about Add Source Sourcedb To Spanner

What does Add Source Sourcedb To Spanner do?

Guide for implementing a database source connector in the v2/sourcedb-to-spanner forward migration Dataflow template. Add Source Sourcedb To Spanner is an agent skill from GoogleCloudPlatform/DataflowTemplates. Guide for implementing a database source connector in the v2/sourcedb-to-spanner forward migration Dataflow template.

When should I use Add Source Sourcedb To Spanner?

Add Source Sourcedb To Spanner fits situations like: tasks that involve Unit testing.

How do I install Add Source Sourcedb To Spanner in Claude Code?

Run `npx skills add GoogleCloudPlatform/DataflowTemplates --skill add-source-sourcedb-to-spanner -a claude-code`. Or copy the skill folder (v2/sourcedb-to-spanner/.agents/skills/add-source-sourcedb-to-spanner in GoogleCloudPlatform/DataflowTemplates) into .claude/skills/add-source-sourcedb-to-spanner in your project. Claude Code loads it when a task matches its description.

How do I install Add Source Sourcedb To Spanner in Codex?

Run `npx skills add GoogleCloudPlatform/DataflowTemplates --skill add-source-sourcedb-to-spanner -a codex`. Or copy the skill folder (v2/sourcedb-to-spanner/.agents/skills/add-source-sourcedb-to-spanner in GoogleCloudPlatform/DataflowTemplates) into .agents/skills/add-source-sourcedb-to-spanner in your project. Codex loads it when a task matches its description.

Can I use Add Source Sourcedb To Spanner in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add GoogleCloudPlatform/DataflowTemplates --skill add-source-sourcedb-to-spanner -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/add-source-sourcedb-to-spanner, .gemini/skills/add-source-sourcedb-to-spanner, .github/skills/add-source-sourcedb-to-spanner and .opencode/skills/add-source-sourcedb-to-spanner in your project.

What does Add Source Sourcedb To Spanner need to run?

Going by SKILL.md and its folder, Add Source Sourcedb To Spanner needs the command-line tools its instructions call (mvn).

Does Add Source Sourcedb To Spanner access the network?

SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.

Is Add Source Sourcedb To Spanner safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Add Source Sourcedb To Spanner use?

Add Source Sourcedb To Spanner is published under the Apache-2.0 licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Add Source Sourcedb To Spanner use?

About 1.9k tokens (SKILL.md is roughly 7.4k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Add Source Sourcedb To Spanner?

Skills that share tags, products or a category with Add Source Sourcedb To Spanner: Io Connectors (Kilo-Org/kilo-marketplace, 189 stars), Add Library Test (osama-raddad/FireCrasher, 147 stars), Edt MCP Build Test (DitriXNew/EDT-MCP, 295 stars) and Debug Surefire (eclipse-rdf4j/rdf4j, 420 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Add Source Sourcedb To Spanner?

GoogleCloudPlatform (a GitHub organization) maintains it in GoogleCloudPlatform/DataflowTemplates, which has 1,315 GitHub stars. The repository holds 12 skills in this directory. The repository was last updated on October 7, 2026.

Source: GoogleCloudPlatform/DataflowTemplates on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.