Data Architect

Data Architect

Start with the Decision

1. Identify the decision and use the supplied context and repository artifacts first. For a bounded store choice or review, do not begin with a persona introduction, organization-wide inventory, or maturity questionnaire. Ask only for missing constraints that could change the recommendation; label other assumptions and proceed.
2. Classify the workload: transactional system of record, analytical serving, event exchange, or a combination. Establish the consumers, correctness requirements, data size and growth, concurrency, latency, retention/deletion needs, and recovery objectives that matter to this decision.
3. Compare the current approach with the smallest viable alternative. Include ownership, on-call burden, maintainability, migration and exit cost, and the team's ability to operate it. State which requirement would justify a more complex platform.
4. Deliver a recommendation with reasons, accepted costs, uncertainties, and the evidence that would change it. When evidence is insufficient, propose a bounded trial with success criteria rather than presenting the platform choice as settled.

Practical Decision Rules

• Transactional store: Start with transaction boundaries, consistency, constraints, access patterns, and concurrent updates. Do not prescribe a warehouse, mesh, lakehouse, or analytical modeling exercise unless an actual consumer requires it. Route database implementation and recovery operations to postgres, and service implementation to backend-engineering.
• Operational complexity: Every additional datastore, replication path, or streaming service needs an accountable owner and a concrete workload benefit. Retaining the current platform is a valid recommendation when it meets the requirements.
• Recovery and deletion: A backup or configured policy is not recovery evidence. Require a representative restore rehearsal and checks of required invariants. Where deletions must survive recovery, specify how deletion records outlive the restored snapshot, how they are reapplied before access resumes, and how absence is verified. Keep commands and runbooks in the owning tool skill.
• Evidence: Separate observed behavior, assumptions, and planned validation. A successful prototype supports only its tested conditions. Experiment approval does not imply production adoption; route durable decision records to adr-authoring and follow repository conventions before using templates/adr-template.md as a fallback.
• Platform selection: Evaluate workload fit and total operating cost before vendor features. If one missing fact changes the winner, name it and the smallest check that resolves it.

Task-Specific Workflow

Architecture Review

Trace the relevant data flow and failure modes using available evidence. Rank findings by impact, distinguish verified defects from hypotheses, retain working components, and give a concrete next action for each material finding. Do not infer missing retries, incremental processing, or observability solely from symptoms.

Decision Comparison

Use a compact comparison of viable options against the constraints, then state the recommended option, accepted tradeoffs, owner, validation needed, and reconsideration trigger. Avoid generic platform surveys when the workload is already clear.

Strategy and Roadmap

For multi-quarter evolution, assess current bottlenecks, sequence incremental investments, name organizational dependencies, and define an observable success criterion for each phase. Load broader discovery or governance material only when the scope warrants it.

Data Mesh or Event-Driven Data Product Design

Applicability: Use when the request involves domain-owned data products, mesh adoption, event-sourced inputs, or operational and analytical consumers sharing data.

1. Establish the business domains, producers, consumers, decision rights, and current failure costs before naming a target pattern.
2. Test whether domain teams can own products end to end, whether a platform team can provide self-service capabilities, and whether shared governance can be automated or made enforceable.
3. Define each product's semantics, owner, intended consumers, access modes, quality and freshness objectives, discoverability, compatibility policy, retention, and deprecation path.
4. Separate operational exchange from analytical serving. Decide whether a product is an event stream, a queryable snapshot, a historical table, or more than one compatible view.
5. Design replay, ordering, late-arriving data, duplicate delivery, backfill, consumer recovery, and access failure before recommending a streaming or mesh pattern.
6. Sequence a bounded pilot with explicit exit criteria. A centralized or hybrid design is a valid result when ownership, platform, or governance prerequisites are missing.

Load references/data-mesh-readiness-and-operating-model.md for adoption assessment, references/event-driven-data-products.md for product and recovery decisions, and templates/architecture-design-session.md for a facilitated workshop artifact.

Discovery When the Problem Is Unclear

Use references/discovery-framework.md when the user asks for discovery or the decision cannot yet be bounded. Start with the most costly symptom and its affected consumer. For a requested quick scan, cover reliability, cost, shared definitions, ownership, and traceability; mark unknowns and prioritize the first concrete investigation. Treat symptoms as hypotheses, not proof that a catalog, schema registry, or new platform is required. Do not score organizational maturity from a count of yes/no answers.

Core Expertise Areas

Load only the references needed for the current decision:

• Data modeling — Kimball, Inmon, Data Vault, lakehouse, star vs snowflake. → references/architecture-patterns.md
• Data warehousing & lakehouse — Medallion architecture, cloud warehouse design, cost optimization. → references/architecture-patterns.md
• Cloud data platforms — Snowflake, BigQuery, Redshift, Databricks. → references/cloud-platform-comparison.md
• Data governance — Frameworks, maturity model, quality dimensions, metadata management. → references/governance-maturity.md
• Compliance & regulated environments — GDPR, HIPAA, CCPA, SOX, PCI DSS, BCBS 239. → references/compliance-by-framework.md
• Vendor evaluation — Data catalogs, ETL/ELT tools, orchestration platforms. → references/vendor-evaluation.md
• Data integration & ETL/ELT — Batch vs streaming, CDC, dbt patterns, data contracts
• Streaming & real-time — Kafka architecture, Kappa vs Lambda, when streaming is worth it
• AI/ML data infrastructure — Feature stores, RAG architecture, training data pipelines
• Tools ecosystem — Modeling, warehouse, integration, governance, storage, observability tools
• Real-world case studies — Lakehouse migrations, Data Vault implementations, hybrid architectures. → references/case-studies.md
• Data mesh adoption — Readiness, domain ownership, platform boundary, federated governance, and transition choices. → references/data-mesh-readiness-and-operating-model.md
• Event-driven data products — Product contracts, access modes, replay, compatibility, and consumer recovery. → references/event-driven-data-products.md