Official agent skill

Azure Databricks

by MicrosoftDocs in MicrosoftDocs/Agent-Skills

Expert knowledge for Azure Databricks development including troubleshooting, best practices, decision making, architecture & design patterns, limits & quotas, security, configuration, integrations &…

OfficialCC-BY-4.0Auto-check passedDevelopment

Install Azure Databricks

skills CLI
$ npx skills add MicrosoftDocs/Agent-Skills --skill azure-databricks -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install MicrosoftDocs/Agent-Skills azure-databricks --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/MicrosoftDocs/Agent-Skills.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/azure-databricks .claude/skills/azure-databricks && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
azure-databricks
GitHub stars
775
Used in
1 other repo
Token cost
~14k tokens
SKILL.md length
3,042 words
Files
8
Skills in repo
149
Repo updated
First seen
Licence
CC-BY-4.0

At a glance

Expert knowledge for Azure Databricks development including troubleshooting, best practices, decision making, architecture & design patterns, limits & quotas, security, configuration, integrations &…

  • Working with Unity Catalog
  • SKILL.md covers How to Use This Skill and Category Index
  • Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md
  • Lakehouse/Lakeflow

What it does

Azure Databricks is an agent skill from MicrosoftDocs/Agent-Skills, published by the product's own GitHub organization. Expert knowledge for Azure Databricks development including troubleshooting, best practices, decision making, architecture & design patterns, limits & quotas, security, configuration, integrations & coding patterns, and deployment. Use when working with Unity Catalog, Lakehouse/Lakeflow, Lakebase, AI Runtime/model serving, or external connectors, and other Azure Databricks related development tasks. Not for Azure Synapse Analytics (use azure-synapse-analytics), Azure HDInsight (use azure-hdinsight), Azure Machine…

Its SKILL.md is about 14k tokens, which your agent loads only when the skill is triggered. The skill folder holds 7 other files (for example `architecture-patterns.md`, `configuration.md` and `decision-making.md`). Compatibility notes: Requires network access. Uses mcpmicrosoftdocs:microsoftdocsfetch or fetchwebpage to retrieve documentation.

It sits in Development, covering Design patterns, Test data and fixtures and Data warehousing. It works with Microsoft Azure, Databricks and Azure Machine Learning. The repository describes itself as: Curated Agent Skills for Microsoft & Azure – giving AI coding assistants structured, real-time expertise from Microsoft Learn docs. The licence is CC-BY-4.0.

When your agent uses it

  • Working with Unity Catalog
  • Lakehouse/Lakeflow
  • AI Runtime/model serving
  • External connectors

Example prompts

  • “/azure-databricks”

Requirements

  • Python 3
  • Compatibility (from SKILL.md): Requires network access. Uses mcp_microsoftdocs:microsoft_docs_fetch or fetch_webpage to retrieve documentation.

What it can do on your machine

Read from SKILL.md and the folder at commit ba74e8f. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    No scripts in the folder and no shell commands in SKILL.md.

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    Links to these hosts (documentation or services it may open):

    • learn.microsoft.com
    • github.com

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

  • Compatibility

    Requires network access. Uses mcp_microsoftdocs:microsoft_docs_fetch or fetch_webpage to retrieve documentation.

    From compatibility in the SKILL.md frontmatter.

Context cost

Azure Databricks loads about 14k tokens when it runs. Until then it costs about 155 tokens; SKILL.md has 3,042 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~155
When it runs · the whole SKILL.md, loaded when a task matches
~14k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from MicrosoftDocs/Agent-Skills at commit ba74e8f, republished under its CC-BY-4.0 licence (© MicrosoftDocs). 3,042 words, ~14,307 tokens.

Download SKILL.mdSave it as .claude/skills/azure-databricks/SKILL.md (or your agent's skills folder). This skill also uses 7 other files; get the full folder from GitHub.
name
azure-databricks
description
Expert knowledge for Azure Databricks development including troubleshooting, best practices, decision making, architecture & design patterns, limits & quotas, security, configuration, integrations & coding patterns, and deployment. Use when working with Unity Catalog, Lakehouse/Lakeflow, Lakebase, AI Runtime/model serving, or external connectors, and other Azure Databricks related development tasks. Not for Azure Synapse Analytics (use azure-synapse-analytics), Azure HDInsight (use azure-hdinsight), Azure Machine Learning (use azure-machine-learning), Azure Data Factory (use azure-data-factory).
compatibility
Requires network access. Uses mcp_microsoftdocs:microsoft_docs_fetch or fetch_webpage to retrieve documentation.
metadata.generated_at
2026-10-04
metadata.generator
docs2skills/1.0.0

Azure Databricks Skill

This skill provides expert guidance for Azure Databricks. Covers troubleshooting, best practices, decision making, architecture & design patterns, limits & quotas, security, configuration, integrations & coding patterns, and deployment. It combines local quick-reference content with remote documentation fetching capabilities.

How to Use This Skill

IMPORTANT for Agent: Use the Category Index below to locate relevant sections. For categories with line ranges (e.g., L35-L120), use read_file with the specified lines. For categories with file links (e.g., [security.md](security.md)), use read_file on the linked reference file

IMPORTANT for Agent: If metadata.generated_at is more than 3 months old, suggest the user pull the latest version from the repository. If mcp_microsoftdocs tools are not available, suggest the user install it: Installation Guide

This skill requires network access to fetch documentation content:

  • Preferred: Use mcp_microsoftdocs:microsoft_docs_fetch with query string from=learn-agent-skill. Returns Markdown.
  • Fallback: Use fetch_webpage with query string from=learn-agent-skill&accept=text/markdown. Returns Markdown.

Category Index

CategoryLocationDescription
TroubleshootingL37-L183Diagnosing and fixing Databricks issues: Spark and SQL errors, connectors and Lakeflow pipelines, model serving and AI Runtime, CLI/IDE/Git problems, and monitoring/debugging tools.
Best PracticesL184-L398End-to-end Databricks best practices for cost, governance, security, performance, streaming, AI/ML/RAG, connectors, and Lakehouse design, plus tuning, testing, and operating production workloads.
Decision Makingdecision-making.mdGuides for architectural and cost decisions in Azure Databricks: choosing compute, runtimes, AI/BI tools, connectors, and migration paths for workspaces, pipelines, models, and Unity Catalog.
Architecture & Design Patternsarchitecture-patterns.mdArchitectural blueprints and patterns for Databricks: DR/HA, networking, storage, Lakehouse/medallion, Lakeflow/CDC, Lakebase, multi-agent/AI pipelines, MLOps, feature stores, and streaming.
Limits & Quotaslimits-quotas.mdLimits, quotas, and constraints for Databricks compute, AI/model serving, Lakeflow pipelines, connectors, Unity Catalog, Free Edition, and related resource usage and throttling behavior.
Securitysecurity.mdIdentity, access control, encryption, networking, compliance, and secret management for Azure Databricks, Unity Catalog, AI/Apps, Lakeflow, Lakebase, and external data/model connections.
Configurationconfiguration.mdConfiguring and governing Azure Databricks: accounts, workspaces, compute, networking, storage, AI/ML, Lakeflow, Unity Catalog, SQL, connectors, CLI, and monitoring/cost/observability settings.
Integrations & Coding Patternsintegrations.mdPatterns and examples for integrating Databricks with apps, agents, AI/ML, external databases/BI tools, Lakehouse Federation, Lakeflow, AI Runtime, and SQL/PySpark APIs and UDFs.
Deploymentdeployment.mdDeploying and managing Azure Databricks workspaces, apps, models, pipelines, and Lakebase using ARM/Bicep/Terraform/Bundles/CI-CD, plus Unity Catalog, AI Runtime, and model serving deployment patterns.
Troubleshooting
TopicURL
Use Databricks identity management readiness reporthttps://learn.microsoft.com/en-us/azure/databricks/admin/users-groups/automatic-identity-management/readiness-report
Debug custom code agents on Azure Databrickshttps://learn.microsoft.com/en-us/azure/databricks/agents/custom-agents/debug-agent
Detect and clean up unused AI Search endpointshttps://learn.microsoft.com/en-us/azure/databricks/ai-search/unused-endpoints
Troubleshoot Azure Databricks Artifact Registry issueshttps://learn.microsoft.com/en-us/azure/databricks/artifact-registry/troubleshooting
Resolve Databricks classic compute termination error codeshttps://learn.microsoft.com/en-us/azure/databricks/compute/troubleshooting/cluster-error-codes
Debug Spark applications using Databricks Spark UIhttps://learn.microsoft.com/en-us/azure/databricks/compute/troubleshooting/debugging-spark-ui
Monitor Databricks dashboard usage with audit logshttps://learn.microsoft.com/en-us/azure/databricks/dashboards/monitor-usage
Troubleshoot common Databricks CLI issueshttps://learn.microsoft.com/en-us/azure/databricks/dev-tools/cli/troubleshooting
Use Databricks app details for monitoring and troubleshootinghttps://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-apps/view-app-details
Troubleshoot Databricks Connect for Python issueshttps://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect/python/troubleshooting
Troubleshoot Databricks Connect for Scala issueshttps://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect/scala/troubleshooting
Troubleshoot Databricks Terraform provider issueshttps://learn.microsoft.com/en-us/azure/databricks/dev-tools/terraform/troubleshoot
Troubleshoot issues with the Databricks IDE extensionhttps://learn.microsoft.com/en-us/azure/databricks/dev-tools/vscode-ext/troubleshooting
Handle ARITHMETIC_OVERFLOW errors in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/arithmetic-overflow-error-class
Resolve CAST_INVALID_INPUT errors in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/cast-invalid-input-error-class
Diagnose DC_GA4_RAW_DATA_ERROR in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/dc-ga4-raw-data-error-error-class
Understand DC_SFDC_API_ERROR in Databricks connectorshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/dc-sfdc-api-error-error-class
Diagnose DC_SQLSERVER_ERROR in Databricks connectorshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/dc-sqlserver-error-error-class
Handle DELTA_ICEBERG_COMPAT_V1_VIOLATION errorshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/delta-iceberg-compat-v1-violation-error-class
Resolve DIVIDE_BY_ZERO error in Azure Databricks SQLhttps://learn.microsoft.com/en-us/azure/databricks/error-messages/divide-by-zero-error-class
Handle Azure Databricks error condition stringshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/error-classes
Troubleshoot EWKB_PARSE_ERROR in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/ewkb-parse-error-error-class
Troubleshoot EWKT_PARSE_ERROR in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/ewkt-parse-error-error-class
Troubleshoot GEOJSON_PARSE_ERROR in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/geojson-parse-error-error-class
Resolve GROUP_BY_AGGREGATE errors in Databricks SQLhttps://learn.microsoft.com/en-us/azure/databricks/error-messages/group-by-aggregate-error-class
Handle H3_INVALID_CELL_ID errors in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/h3-invalid-cell-id-error-class
Fix H3_INVALID_GRID_DISTANCE_VALUE errorshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/h3-invalid-grid-distance-value-error-class
Fix H3_INVALID_RESOLUTION_VALUE errorshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/h3-invalid-resolution-value-error-class
Resolve H3_NOT_ENABLED errors in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/h3-not-enabled-error-class
Handle INSUFFICIENT_TABLE_PROPERTY errorshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/insufficient-table-property-error-class
Resolve INVALID_ARRAY_INDEX errors in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/invalid-array-index-error-class
Resolve INVALID_ARRAY_INDEX_IN_ELEMENT_AT errorshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/invalid-array-index-in-element-at-error-class
Fix MISSING_AGGREGATION errors in GROUP BYhttps://learn.microsoft.com/en-us/azure/databricks/error-messages/missing-aggregation-error-class
Troubleshoot ROW_COLUMN_ACCESS errors in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/row-column-access-error-class
Interpret Azure Databricks SQLSTATE error codeshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/sqlstates
Resolve TABLE_OR_VIEW_NOT_FOUND errors in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/table-or-view-not-found-error-class
Fix UNRESOLVED_ROUTINE errors in Databricks SQLhttps://learn.microsoft.com/en-us/azure/databricks/error-messages/unresolved-routine-error-class
Handle UNSUPPORTED_TABLE_OPERATION errorshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/unsupported-table-operation-error-class
Handle UNSUPPORTED_VIEW_OPERATION errorshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/unsupported-view-operation-error-class
Troubleshoot WKB_PARSE_ERROR in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/wkb-parse-error-error-class
Troubleshoot WKT_PARSE_ERROR in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/error-messages/wkt-parse-error-error-class
Troubleshoot common Genie Agent query issueshttps://learn.microsoft.com/en-us/azure/databricks/genie-agents/troubleshooting
Auto Loader FAQ and operational guidancehttps://learn.microsoft.com/en-us/azure/databricks/ingestion/cloud-object-storage/auto-loader/faq
Monitor and troubleshoot Auto Loader pipelineshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/cloud-object-storage/auto-loader/observability
Troubleshoot 1Password Event Logs connector errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/1password-event-logs-troubleshoot
Troubleshoot Aha! managed connector issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/aha-troubleshoot
Troubleshoot Akamai WAF connector authentication and errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/akamai-waf-troubleshoot
Troubleshoot Databricks Amplitude connector issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/amplitude-troubleshoot
Troubleshoot Databricks Anaplan connector errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/anaplan-troubleshoot
Troubleshoot Anthropic connector ingestion issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/anthropic-troubleshoot
Troubleshoot Anysphere Audit Logs connector issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/anysphere-audit-logs-troubleshoot
Troubleshoot Anysphere Organization connector problemshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/anysphere-organization-troubleshoot
Troubleshoot Atlassian audit logs connector errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/atlassian-audit-logs-troubleshoot
Troubleshoot Databricks Celigo managed connector errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/celigo-troubleshoot
Troubleshoot Databricks Confluence ingestion errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/confluence-troubleshoot
Troubleshoot CrowdStrike Falcon Event Stream connector issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/crowdstrike-falcon-event-stream-troubleshoot
Troubleshoot Dynamics 365 ingestion via Synapse Linkhttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/d365-troubleshoot
Troubleshoot Glean connector errors in Lakeflowhttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/glean-troubleshoot
Troubleshoot Gmail connector ingestion issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/gmail-troubleshoot
Troubleshoot Databricks Google Ads connector errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/google-ads-troubleshoot
Troubleshoot GA4 Raw Data connector issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/google-analytics-troubleshoot
Google Drive connector FAQs and behaviorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/google-drive-faq
Troubleshoot Google Drive ingestion in Lakeflowhttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/google-drive-troubleshoot
Troubleshoot Databricks Google Search Console connectorhttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/google-search-console-troubleshoot
Troubleshoot Databricks Google Workspace connector issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/google-workspace-troubleshoot
Troubleshoot HubSpot connector ingestion problemshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/hubspot-troubleshoot
Troubleshoot Jira ingestion issues in Lakeflowhttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/jira-troubleshoot
Troubleshoot managed Kafka connector issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/kafka-troubleshoot
Troubleshoot LinkedIn Ads connector errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/linkedin-ads-troubleshoot
Troubleshoot Marketo connector pipeline errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/marketo-troubleshoot
Diagnose and fix Meta Ads Lakeflow Connect ingestion errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/meta-ads-troubleshoot
Troubleshoot Microsoft 365 audit connector issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/microsoft-365-troubleshoot
Troubleshoot Monday.com Lakeflow connector errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/monday-com-troubleshoot
Troubleshoot Databricks MySQL ingestion issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/mysql-troubleshoot
Troubleshoot Netskope Logs Lakeflow connectorhttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/netskope-logs-troubleshoot
Troubleshoot Notion connector authentication and sync issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/notion-troubleshoot
Troubleshoot Databricks Okta System Logs connector errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/okta-system-logs-troubleshoot
Troubleshoot Databricks OpenAI connector issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/openai-troubleshoot
Troubleshoot Oracle integrated CDC ingestion issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/oracle-troubleshoot
Troubleshoot Outlook connector ingestion errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/outlook-troubleshoot
Troubleshoot Databricks PagerDuty connector errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/pagerduty-troubleshoot
Troubleshoot Databricks Pendo connector issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/pendo-troubleshoot
Troubleshoot Databricks PostgreSQL ingestion issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/postgresql-troubleshoot
Troubleshoot Databricks query-based connectorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/query-based-troubleshoot
Troubleshoot managed RabbitMQ connector errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/rabbitmq-troubleshoot
FAQ for Databricks Reddit Ads ingestion connectorhttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/reddit-ads-faq
Troubleshoot Databricks Reddit Ads connector errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/reddit-ads-troubleshoot
FAQ for Databricks Salesforce ingestion connectorhttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/salesforce-faq
Troubleshoot Databricks Salesforce ingestion issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/salesforce-troubleshoot
Troubleshoot Databricks SendGrid connector errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/sendgrid-troubleshoot
Troubleshoot Databricks ServiceNow ingestion issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/servicenow-troubleshoot
Troubleshoot Salesforce Marketing Cloud connector issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/sfmc-troubleshoot
Troubleshoot Databricks SharePoint ingestion errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/sharepoint-troubleshoot
Troubleshoot Databricks Shopify connector issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/shopify-troubleshoot
Troubleshoot Databricks Smartsheet connector errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/smartsheet-troubleshoot
Troubleshoot Databricks SQL Server ingestion issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/sql-server-troubleshoot
Troubleshoot Databricks Square connector errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/square-troubleshoot
Troubleshoot Databricks Strac connector errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/strac-troubleshoot
Troubleshoot TikTok Ads connector ingestion issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/tiktok-ads-troubleshoot
Diagnose and fix Databricks Lakeflow Connect ingestion issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/troubleshoot
Resolve UNITY_CATALOG_INITIALIZATION_FAILED in pipelineshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/uc-initialization-troubleshoot
Troubleshoot Veeva Vault connector errors in Lakeflowhttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/veeva-vault-troubleshoot
Troubleshoot Databricks Verkada connector issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/verkada-troubleshoot
Troubleshoot Wiz Audit Logs connector errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/wiz-audit-logs-troubleshoot
Troubleshoot Workday Activity Logging connector issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/workday-activity-logging-troubleshoot
Troubleshoot Workday HCM connector errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/workday-hcm-troubleshoot
Troubleshoot Databricks Workday ingestion issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/workday-reports-troubleshoot
Troubleshoot common Workiva connector errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/workiva-troubleshoot
Troubleshoot Zendesk Support connector issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/zendesk-support-troubleshoot
Troubleshoot Databricks Zip connector errorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/zip-troubleshoot
Troubleshoot Databricks Zoho Books connector issueshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/zoho-books-troubleshoot
Handle Zerobus Ingest error codes and failureshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/zerobus-errors
Understand and use Azure Databricks init script logginghttps://learn.microsoft.com/en-us/azure/databricks/init-scripts/logs
Troubleshoot and repair Lakeflow Jobs failureshttps://learn.microsoft.com/en-us/azure/databricks/jobs/repair-job-failures
Monitor and troubleshoot materialized view refresheshttps://learn.microsoft.com/en-us/azure/databricks/ldp/dbsql/materialized-monitor
Resolve high initialization times in pipelineshttps://learn.microsoft.com/en-us/azure/databricks/ldp/fix-high-init
Recover Lakeflow pipelines from checkpoint failureshttps://learn.microsoft.com/en-us/azure/databricks/ldp/recover-streaming
Use Genie Code to debug AI Runtime GPUshttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/ai-runtime/genie-code
Use AI Runtime guides for usage tracking and troubleshootinghttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/ai-runtime/guides/
Inspect and debug Databricks Feature Views in Unity Cataloghttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/feature-store/explore-feature-views
Troubleshoot Databricks Feature Store and limitshttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/feature-store/troubleshooting-and-limitations
Diagnose and fix Databricks model serving issueshttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/model-serving/model-serving-debug
Use Genie Code to diagnose Databricks model servinghttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/model-serving/model-serving-genie-code
Debug Python code in Databricks notebookshttps://learn.microsoft.com/en-us/azure/databricks/notebooks/debugger
Troubleshoot common OpenSharing data access errorshttps://learn.microsoft.com/en-us/azure/databricks/opensharing/troubleshooting
Diagnose failing Spark jobs and removed executorshttps://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/failing-spark-jobs
Use the Databricks jobs timeline to debug Sparkhttps://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/jobs-timeline
Diagnose long Spark jobs using Databricks UIhttps://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/long-spark-stage
Investigate high I/O Spark stages in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/long-spark-stage-io
Debug skew and spill in long Spark stageshttps://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/long-spark-stage-page
Debug slow Spark stages with low I/O in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/slow-spark-stage-low-io
Identify expensive reads in Spark DAG on Databrickshttps://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/spark-dag-expensive-read
Diagnose gaps between Spark jobs in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/spark-job-gaps
Diagnose and fix Spark memory issues on Databrickshttps://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/spark-memory-issues
Troubleshoot Databricks Partner Connect issueshttps://learn.microsoft.com/en-us/azure/databricks/partner-connect/troubleshoot
Troubleshoot Databricks publishing and connections to Power BIhttps://learn.microsoft.com/en-us/azure/databricks/partners/bi/power-bi/troubleshooting
Troubleshoot Azure Databricks Git folder errorshttps://learn.microsoft.com/en-us/azure/databricks/repos/errors-troubleshooting
Handle Databricks SQL FETCH cursor errorshttps://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/control-flow/fetch-stmt
Diagnose Databricks SQL OPEN cursor errorshttps://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/control-flow/open-stmt
Detect and repair Delta table issues with FSCK REPAIR TABLEhttps://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/delta-fsck
Handle INVALID_UTF8_STRING errors in Databricks SQLhttps://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/functions/validate_utf8
Use query history to troubleshoot Databricks SQL performancehttps://learn.microsoft.com/en-us/azure/databricks/sql/user/queries/query-history
Interpret Databricks SQL query profiles for performance troubleshootinghttps://learn.microsoft.com/en-us/azure/databricks/sql/user/queries/query-profile
Best Practices
TopicURL
Tag Azure Databricks resources for cost attributionhttps://learn.microsoft.com/en-us/azure/databricks/admin/account-settings/usage-detail-tags
Use default Databricks compute policy familieshttps://learn.microsoft.com/en-us/azure/databricks/admin/clusters/policy-families
Implement managed disaster recovery for Azure Databrickshttps://learn.microsoft.com/en-us/azure/databricks/admin/managed-disaster-recovery
Apply identity best practices in Azure Databrickshttps://learn.microsoft.com/en-us/azure/databricks/admin/users-groups/best-practices
Apply best practices for serverless workspaceshttps://learn.microsoft.com/en-us/azure/databricks/admin/workspace/serverless-workspaces-best-practices
Synthetically generate agent evaluation setshttps://learn.microsoft.com/en-us/azure/databricks/agents/agent-evaluation/synthesize-evaluation-set
Load test Databricks Apps agents for QPS limitshttps://learn.microsoft.com/en-us/azure/databricks/agents/custom-agents/load-test-agent-app
Measure RAG performance with retrieval and response metricshttps://learn.microsoft.com/en-us/azure/databricks/agents/tutorials/ai-cookbook/evaluate-assess-performance
Define RAG application quality with evaluation setshttps://learn.microsoft.com/en-us/azure/databricks/agents/tutorials/ai-cookbook/evaluate-define-quality
Evaluate and monitor RAG applications for quality, cost, latencyhttps://learn.microsoft.com/en-us/azure/databricks/agents/tutorials/ai-cookbook/fundamentals-evaluation-monitoring-rag
Design and optimize RAG inference chains on Databrickshttps://learn.microsoft.com/en-us/azure/databricks/agents/tutorials/ai-cookbook/fundamentals-inference-chain-rag
Build and tune unstructured RAG data pipelineshttps://learn.microsoft.com/en-us/azure/databricks/agents/tutorials/ai-cookbook/quality-data-pipeline-rag
Improve RAG application quality via key tuning knobshttps://learn.microsoft.com/en-us/azure/databricks/agents/tutorials/ai-cookbook/quality-overview
Optimize RAG chain components for better responseshttps://learn.microsoft.com/en-us/azure/databricks/agents/tutorials/ai-cookbook/quality-rag-chain
Control coding agent costs with Unity Gateway budgets and routinghttps://learn.microsoft.com/en-us/azure/databricks/ai-gateway/coding-agent-costs
Monitor coding agent usage and traces in Unity Gatewayhttps://learn.microsoft.com/en-us/azure/databricks/ai-gateway/coding-agent-observability
Track foundation model spend by user and projecthttps://learn.microsoft.com/en-us/azure/databricks/ai-gateway/track-cost-tutorial
Apply performance best practices for AI Searchhttps://learn.microsoft.com/en-us/azure/databricks/ai-search/best-practices
Load test Databricks AI Search endpointshttps://learn.microsoft.com/en-us/azure/databricks/ai-search/endpoint-load-test
Improve Databricks AI Search retrieval qualityhttps://learn.microsoft.com/en-us/azure/databricks/ai-search/retrieval-quality
Migrate Databricks library installs from init scriptshttps://learn.microsoft.com/en-us/azure/databricks/archive/compute/libraries-init-scripts
Apply best practices for Databricks compute policieshttps://learn.microsoft.com/en-us/azure/databricks/archive/compute/policies-best-practices
Use DBIO for transactional writes to cloud storage in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/archive/legacy/dbio-commit
Optimize skewed joins in Databricks using skew hintshttps://learn.microsoft.com/en-us/azure/databricks/archive/legacy/skew-join
Migrate from Databricks Deep Learning Pipelineshttps://learn.microsoft.com/en-us/azure/databricks/archive/spark-3.x-migration/deep-learning-pipelines
Apply Azure Databricks platform administration best practiceshttps://learn.microsoft.com/en-us/azure/databricks/cheat-sheet/administration
Optimize BI serving performance on Azure Databrickshttps://learn.microsoft.com/en-us/azure/databricks/cheat-sheet/bi-serving
Prepare and model data for high-performance BI on Databrickshttps://learn.microsoft.com/en-us/azure/databricks/cheat-sheet/bi-serving-data-prep
Optimize Azure Databricks SQL warehouses for BIhttps://learn.microsoft.com/en-us/azure/databricks/cheat-sheet/bi-serving-sql-serving
Apply Azure Databricks compute creation best practiceshttps://learn.microsoft.com/en-us/azure/databricks/cheat-sheet/compute
Implement Azure Databricks production job scheduling best practiceshttps://learn.microsoft.com/en-us/azure/databricks/cheat-sheet/jobs
Apply Databricks-specific Power BI performance best practiceshttps://learn.microsoft.com/en-us/azure/databricks/cheat-sheet/power-bi
Use AI-generated comments for Unity Catalog documentationhttps://learn.microsoft.com/en-us/azure/databricks/comments/ai-comments
Apply best practices for Databricks classic computehttps://learn.microsoft.com/en-us/azure/databricks/compute/cluster-config-best-practices
Use flexible node types for reliable Databricks computehttps://learn.microsoft.com/en-us/azure/databricks/compute/flexible-node-types
Apply best practices for Databricks poolshttps://learn.microsoft.com/en-us/azure/databricks/compute/pool-best-practices
Follow best practices for Databricks serverless computehttps://learn.microsoft.com/en-us/azure/databricks/compute/serverless/best-practices
Use Lakehouse Replay to validate runtime upgradeshttps://learn.microsoft.com/en-us/azure/databricks/compute/serverless/lakehouse-replay
Tune SQL warehouse settings for BI workloadshttps://learn.microsoft.com/en-us/azure/databricks/compute/sql-warehouse/bi-workload-settings
Control large interactive queries with Query Watchdoghttps://learn.microsoft.com/en-us/azure/databricks/compute/troubleshooting/query-watchdog
Optimize Azure Databricks AI/BI dashboard cachinghttps://learn.microsoft.com/en-us/azure/databricks/dashboards/caching
Configure dataset materialization for Databricks dashboardshttps://learn.microsoft.com/en-us/azure/databricks/dashboards/materialization
Implement observability for Databricks streaming workloadshttps://learn.microsoft.com/en-us/azure/databricks/data-engineering/observability-best-practices
Handle schema evolution in Azure Databricks pipelineshttps://learn.microsoft.com/en-us/azure/databricks/data-engineering/schema-evolution
Apply best practices for Unity Catalog ABAC policieshttps://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/abac/best-practices
Apply common ABAC row filtering and column masking patterns in Unity Cataloghttps://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/abac/common-patterns
Optimize performance of ABAC row filters and column masks in Unity Cataloghttps://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/abac/performance
Apply Unity Catalog governance best practiceshttps://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/best-practices
Manage Unity Catalog object storage lifecycle and recoveryhttps://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/object-storage-lifecycle
Author Unity Catalog service policies with exampleshttps://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/service-policies/policy-examples
Work with legacy Hive metastore objects in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/database-objects/hive-metastore
Safely use and migrate away from DBFS roothttps://learn.microsoft.com/en-us/azure/databricks/dbfs/dbfs-root
Apply best practices for DBFS and Unity Cataloghttps://learn.microsoft.com/en-us/azure/databricks/dbfs/unity-catalog
Apply Delta Lake best practices on Databrickshttps://learn.microsoft.com/en-us/azure/databricks/delta/best-practices
Handle Delta Lake limitations on S3 safelyhttps://learn.microsoft.com/en-us/azure/databricks/delta/s3-limitations
Use selective overwrite options with Delta Lake on Databrickshttps://learn.microsoft.com/en-us/azure/databricks/delta/selective-overwrite
Apply recommended CI/CD workflows on Databrickshttps://learn.microsoft.com/en-us/azure/databricks/dev-tools/ci-cd/flows
View Databricks policy families via CLIhttps://learn.microsoft.com/en-us/azure/databricks/dev-tools/cli/reference/policy-families-commands
Develop Databricks Apps using supported frameworks and patternshttps://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-apps/app-development
Apply best practices for Databricks Appshttps://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-apps/best-practices
Advanced configuration and usage of Databricks Connecthttps://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect/advanced
Test Databricks Connect Python code with pytesthttps://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect/python/testing
Handle asynchronous queries and interruptions in Databricks Connecthttps://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect/queries
Test Databricks Connect Scala code with ScalaTesthttps://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect/scala/testing
Call Databricks REST API with performance best practiceshttps://learn.microsoft.com/en-us/azure/databricks/dev-tools/rest-api
Apply Databricks developer and CI/CD best practiceshttps://learn.microsoft.com/en-us/azure/databricks/developers/best-practices
Choose between Unity Catalog volumes and workspace fileshttps://learn.microsoft.com/en-us/azure/databricks/files/files-recommendations
Store and reference Databricks init scripts in workspace fileshttps://learn.microsoft.com/en-us/azure/databricks/files/workspace-init-scripts
Curate effective Genie Agents for accurate answershttps://learn.microsoft.com/en-us/azure/databricks/genie-agents/best-practices
Apply prompt and context best practices for Genie Codehttps://learn.microsoft.com/en-us/azure/databricks/genie-code/tips
Deep clone managed Iceberg tables in Unity Cataloghttps://learn.microsoft.com/en-us/azure/databricks/iceberg/clone
Apply Azure Databricks Auto Loader best practiceshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/cloud-object-storage/auto-loader/best-practices
Configure Azure Databricks Auto Loader for productionhttps://learn.microsoft.com/en-us/azure/databricks/ingestion/cloud-object-storage/auto-loader/production
Apply common COPY INTO data loading patternshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/cloud-object-storage/copy-into/examples
Celigo connector FAQ and usage guidancehttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/celigo-faq
Apply common patterns to Lakeflow ingestion pipelineshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/common-patterns
Apply Confluence connector behaviors and FAQshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/confluence-faq
CrowdStrike Falcon Event Stream connector FAQhttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/crowdstrike-falcon-event-stream-faq
Apply Dynamics 365 connector FAQs and behaviorshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/d365-faq
Safely fully refresh Lakeflow target tableshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/full-refresh
Apply Glean connector FAQs and usage guidancehttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/glean-faq
Use Gmail connector FAQs and behavior guidancehttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/gmail-faq
Apply MySQL connector usage best practiceshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/mysql-faq
Use Oracle integrated CDC connector FAQs and tipshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/oracle-faq
Apply PostgreSQL connector FAQs and guidancehttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/postgresql-faq
Maintain PostgreSQL ingestion pipelines in productionhttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/postgresql-maintenance
Optimize incremental ingestion of Salesforce formula fieldshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/salesforce-formula-fields
Use Zerobus acknowledgment callbacks effectivelyhttps://learn.microsoft.com/en-us/azure/databricks/ingestion/zerobus-callbacks
Choose Zerobus blocking methods for durabilityhttps://learn.microsoft.com/en-us/azure/databricks/ingestion/zerobus-message-blocking
Implement resilient Zerobus recovery patternshttps://learn.microsoft.com/en-us/azure/databricks/ingestion/zerobus-recovery
Design Zerobus schemas for evolving datahttps://learn.microsoft.com/en-us/azure/databricks/ingestion/zerobus-schema-management
Use and configure Databricks cluster init scriptshttps://learn.microsoft.com/en-us/azure/databricks/init-scripts/
Reference external files in Databricks init scriptshttps://learn.microsoft.com/en-us/azure/databricks/init-scripts/referencing-files
Test applications using the legacy Simba JDBC Driverhttps://learn.microsoft.com/en-us/azure/databricks/integrations/jdbc/testing
Test Databricks ODBC driver connections in codehttps://learn.microsoft.com/en-us/azure/databricks/integrations/odbc/testing
Diagnose and optimize Lakeflow Jobs performancehttps://learn.microsoft.com/en-us/azure/databricks/jobs/diagnose-job-performance
Schedule recurring SQL queries with backfill in Jobshttps://learn.microsoft.com/en-us/azure/databricks/jobs/how-to/create-recurring-job
Configure classic compute for Databricks Lakeflow Jobshttps://learn.microsoft.com/en-us/azure/databricks/jobs/run-classic-jobs
Apply Databricks cost optimization best practiceshttps://learn.microsoft.com/en-us/azure/databricks/lakehouse-architecture/cost-optimization/best-practices
Implement best practices for Databricks data and AI governancehttps://learn.microsoft.com/en-us/azure/databricks/lakehouse-architecture/data-governance/best-practices
Design observability and monitoring strategy for Databrickshttps://learn.microsoft.com/en-us/azure/databricks/lakehouse-architecture/deployment-guide/observability
Apply interoperability and usability best practices on Databrickshttps://learn.microsoft.com/en-us/azure/databricks/lakehouse-architecture/interoperability-and-usability/best-practices
Implement operational excellence best practices for Databrickshttps://learn.microsoft.com/en-us/azure/databricks/lakehouse-architecture/operational-excellence/best-practices
Implement performance efficiency best practices for Databrickshttps://learn.microsoft.com/en-us/azure/databricks/lakehouse-architecture/performance-efficiency/best-practices
Implement reliability best practices for Databricks workloadshttps://learn.microsoft.com/en-us/azure/databricks/lakehouse-architecture/reliability/best-practices
Implement Databricks security, compliance, and privacy best practiceshttps://learn.microsoft.com/en-us/azure/databricks/lakehouse-architecture/security-compliance-and-privacy/best-practices
Apply Databricks well-architected best practices across pillarshttps://learn.microsoft.com/en-us/azure/databricks/lakehouse-architecture/well-architected
Classify documents with large label taxonomieshttps://learn.microsoft.com/en-us/azure/databricks/large-language-models/classify-documents-labels-tutorial
Optimize pipeline clusters with enhanced autoscalinghttps://learn.microsoft.com/en-us/azure/databricks/ldp/auto-scaling
Apply Lakeflow pipeline design best practiceshttps://learn.microsoft.com/en-us/azure/databricks/ldp/best-practices/
Design Lakeflow pipelines for safe retrieshttps://learn.microsoft.com/en-us/azure/databricks/ldp/best-practices/processing-guarantees
Apply production readiness checks to Lakeflow pipelineshttps://learn.microsoft.com/en-us/azure/databricks/ldp/best-practices/production-readiness
Use REPLACE WHERE flows for targeted recomputeshttps://learn.microsoft.com/en-us/azure/databricks/ldp/dbsql/flows-replace-where
Apply data quality expectations in pipelineshttps://learn.microsoft.com/en-us/azure/databricks/ldp/developer/ldp-python-ref-expectations
Apply advanced expectation patterns across datasetshttps://learn.microsoft.com/en-us/azure/databricks/ldp/expectation-patterns
Perform full refreshes of streaming tables safelyhttps://learn.microsoft.com/en-us/azure/databricks/ldp/full-refresh-st
Optimize stateful streaming with watermarks in pipelineshttps://learn.microsoft.com/en-us/azure/databricks/ldp/stateful-processing
Use ALTER SQL safely with Lakeflow datasetshttps://learn.microsoft.com/en-us/azure/databricks/ldp/using-alter-sql
Optimize AI Runtime training performance and resiliencyhttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/ai-runtime/guides/performance-and-resiliency
Apply Hyperopt best practices and troubleshooting on Databrickshttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/automl-hyperparam-tuning/hyperopt-best-practices
Improve Databricks AutoML forecasting with covariateshttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/automl/automl-covariate-forecast
Follow Databricks machine learning lifecycle practiceshttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/concepts/ml-lifecycle
Implement point-in-time correct feature joinshttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/feature-store/time-series
Benchmark Databricks LLM endpoints for latency and throughputhttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/foundation-model-apis/prov-throughput-run-benchmark
Prepare large datasets for distributed training on Databrickshttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/load-data/ddl-data
Apply recommended LLMOps workflows on Databrickshttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/mlops/llmops
Configure load tests for custom model serving endpointshttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/model-serving/configure-load-test
Validate models before Databricks Model Serving deploymenthttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/model-serving/model-serving-pre-deployment-validation
Monitor Databricks model quality and endpoint healthhttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/model-serving/monitor-diagnose-endpoints
Optimize Databricks Model Serving endpoints for productionhttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/model-serving/production-optimization
Plan and execute load testing for Databricks serving endpointshttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/model-serving/what-is-load-test
Tune and scale Ray clusters on Databrickshttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/ray/scale-ray
Apply deep learning best practices on Azure Databrickshttps://learn.microsoft.com/en-us/azure/databricks/machine-learning/train-model/dl-best-practices
Adapt existing Apache Spark workloads to Databrickshttps://learn.microsoft.com/en-us/azure/databricks/migration/spark
Evaluate and improve agents with MLflow scorershttps://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/eval-monitor/
Align MLflow LLM judges with human feedbackhttps://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/eval-monitor/align-judges
Create guidelines-based LLM judges in MLflowhttps://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/eval-monitor/concepts/judges/guidelines
Developer workflow for MLflow code-based scorershttps://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/eval-monitor/custom-scorer-dev-workflow
Tutorial: Evaluate and improve MLflow GenAI agentshttps://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/eval-monitor/evaluate-app
Monitor MLflow GenAI agents in productionhttps://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/eval-monitor/monitor-in-production
Collect human feedback and build evaluation datasetshttps://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/human-feedback/
Label MLflow traces during agent developmenthttps://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/human-feedback/dev-annotations
Enable experts to label MLflow traces with Review Apphttps://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/human-feedback/expert-feedback/label-existing-traces
Evaluate and compare MLflow prompt versionshttps://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/prompt-version-mgmt/prompt-registry/evaluate-prompts
Detect issues across MLflow GenAI traceshttps://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/tracing/observe-with-traces/analyze-traces
Apply software engineering practices to Databricks notebookshttps://learn.microsoft.com/en-us/azure/databricks/notebooks/best-practices
Run Databricks notebooks safely and efficientlyhttps://learn.microsoft.com/en-us/azure/databricks/notebooks/run-notebook
Test Databricks notebooks with built-in toolshttps://learn.microsoft.com/en-us/azure/databricks/notebooks/test-notebooks
Monitor active queries in Lakebase Postgreshttps://learn.microsoft.com/en-us/azure/databricks/oltp/projects/active-queries
Analyze Lakebase query performance historyhttps://learn.microsoft.com/en-us/azure/databricks/oltp/projects/query-performance
Apply performance optimization recommendations on Databrickshttps://learn.microsoft.com/en-us/azure/databricks/optimizations/
Use adaptive query execution on Databrickshttps://learn.microsoft.com/en-us/azure/databricks/optimizations/aqe
Decommission deprecated Bloom filter indexeshttps://learn.microsoft.com/en-us/azure/databricks/optimizations/bloom-filters
Optimize Spark SQL queries with Databricks CBOhttps://learn.microsoft.com/en-us/azure/databricks/optimizations/cbo
Improve read performance with Databricks disk cachehttps://learn.microsoft.com/en-us/azure/databricks/optimizations/disk-cache
Use dynamic file pruning for Delta querieshttps://learn.microsoft.com/en-us/azure/databricks/optimizations/dynamic-file-pruning
Reduce write conflicts with row-level concurrencyhttps://learn.microsoft.com/en-us/azure/databricks/optimizations/isolation/row-level-concurrency
Optimize Delta MERGE with low shuffle mergehttps://learn.microsoft.com/en-us/azure/databricks/optimizations/low-shuffle-merge
Use predictive I/O optimizations on Databrickshttps://learn.microsoft.com/en-us/azure/databricks/optimizations/predictive-io
Use predictive optimization for managed tableshttps://learn.microsoft.com/en-us/azure/databricks/optimizations/predictive-optimization
Tune range join optimization on Databrickshttps://learn.microsoft.com/en-us/azure/databricks/optimizations/range-join
Diagnose Databricks Spark cost and performance in UIhttps://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/
Handle Databricks spot instance losses effectivelyhttps://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/losing-spot-instances
Resolve long Spark stages with a single taskhttps://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/one-spark-task
Optimize many small Spark jobs in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/small-spark-jobs
Mitigate overloaded Spark driver on Databrickshttps://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/spark-driver-overloaded
Detect unnecessary data rewriting in Databricks Spark writeshttps://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/spark-rewriting-data
Apply best practices for Partner Connect setuphttps://learn.microsoft.com/en-us/azure/databricks/partner-connect/best-practice
Configure networking for Lakehouse Federation in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/query-federation/networking
Optimize performance of Lakehouse Federation querieshttps://learn.microsoft.com/en-us/azure/databricks/query-federation/performance-recommendations
Encrypt inter-node traffic for Databricks clustershttps://learn.microsoft.com/en-us/azure/databricks/security/keys/encrypt-otw
Apply custom DNS best practices for Databricks VNetshttps://learn.microsoft.com/en-us/azure/databricks/security/network/classic/custom-dns
Use SIGNAL and RESIGNAL in Databricks SQL handlershttps://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/control-flow/signal-stmt
Optimize Delta Lake tables with OPTIMIZE in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/delta-optimize
Estimate distinct counts with approx_count_distincthttps://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/functions/approx_count_distinct
Use session_user instead of deprecated current_userhttps://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/functions/current_user
Define liquid clustering with CLUSTER BY in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/sql-ref-syntax-ddl-cluster-by
Use OFFSET and LIMIT for pagination in Databricks SQLhttps://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/sql-ref-syntax-qry-select-offset
Author effective SQL patterns for Databricks alertshttps://learn.microsoft.com/en-us/azure/databricks/sql/user/alerts/query-patterns
Apply Databricks SQL performance insights and recommendationshttps://learn.microsoft.com/en-us/azure/databricks/sql/user/queries/performance-insights
Optimize Databricks SQL queries using RELY constraintshttps://learn.microsoft.com/en-us/azure/databricks/sql/user/queries/query-optimization-constraints
Use asynchronous transformWithState for higher throughputhttps://learn.microsoft.com/en-us/azure/databricks/stateful-applications/async
Manage Structured Streaming checkpoints correctlyhttps://learn.microsoft.com/en-us/azure/databricks/structured-streaming/checkpoints
Run multiple streaming queries per Databricks clusterhttps://learn.microsoft.com/en-us/azure/databricks/structured-streaming/multiple-streams
Run Structured Streaming in production on Databrickshttps://learn.microsoft.com/en-us/azure/databricks/structured-streaming/production
Use real-time mode for ultra-low latency streaminghttps://learn.microsoft.com/en-us/azure/databricks/structured-streaming/real-time/
Optimize Databricks real-time streaming query performancehttps://learn.microsoft.com/en-us/azure/databricks/structured-streaming/real-time/performance
Manage and optimize stateful streaming querieshttps://learn.microsoft.com/en-us/azure/databricks/structured-streaming/stateful-streaming
Optimize stateless Structured Streaming querieshttps://learn.microsoft.com/en-us/azure/databricks/structured-streaming/stateless-streaming
Monitor Databricks Structured Streaming queries effectivelyhttps://learn.microsoft.com/en-us/azure/databricks/structured-streaming/stream-monitoring
Apply watermarks for stateful streaming controlhttps://learn.microsoft.com/en-us/azure/databricks/structured-streaming/watermarks
Leverage data skipping for faster querieshttps://learn.microsoft.com/en-us/azure/databricks/tables/data-skipping
Optimize partition discovery for external tableshttps://learn.microsoft.com/en-us/azure/databricks/tables/external-partition-discovery
Use change data feed for Delta and Iceberg v3https://learn.microsoft.com/en-us/azure/databricks/tables/features/change-data-feed
Optimize VARIANT performance with shreddinghttps://learn.microsoft.com/en-us/azure/databricks/tables/features/variant-shredding
Use table history and time travel safelyhttps://learn.microsoft.com/en-us/azure/databricks/tables/history
Enrich Databricks tables with comments and metadatahttps://learn.microsoft.com/en-us/azure/databricks/tables/operations/custom-metadata
Safely drop or replace Databricks tables by typehttps://learn.microsoft.com/en-us/azure/databricks/tables/operations/drop-table
Optimize Delta and Iceberg table file layouthttps://learn.microsoft.com/en-us/azure/databricks/tables/operations/optimize
Use VACUUM to reclaim storage and ensure compliancehttps://learn.microsoft.com/en-us/azure/databricks/tables/operations/vacuum
Interpret table size and reclaim storage in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/tables/size
Control Delta and Iceberg data file sizeshttps://learn.microsoft.com/en-us/azure/databricks/tables/tune-file-size
Safely evolve Delta and Iceberg table schemashttps://learn.microsoft.com/en-us/azure/databricks/tables/update-schema
Aggregate data using batch, materialized views, and streaminghttps://learn.microsoft.com/en-us/azure/databricks/transform/aggregation
Design Delta Lake data models on Databrickshttps://learn.microsoft.com/en-us/azure/databricks/transform/data-modeling
Implement joins for batch and streaming in Databrickshttps://learn.microsoft.com/en-us/azure/databricks/transform/join
Optimize join performance on Azure Databrickshttps://learn.microsoft.com/en-us/azure/databricks/transform/optimize-joins
Clean and validate data on Azure Databrickshttps://learn.microsoft.com/en-us/azure/databricks/transform/validate
Implement session-scoped Scala and Java UDFshttps://learn.microsoft.com/en-us/azure/databricks/udf/scala
Access task context inside Databricks UDFshttps://learn.microsoft.com/en-us/azure/databricks/udf/udf-task-context
Download internet data into Azure Databricks volumeshttps://learn.microsoft.com/en-us/azure/databricks/volumes/download-internet-files

© MicrosoftDocs, CC-BY-4.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 7 other files in skills/azure-databricks of MicrosoftDocs/Agent-Skills.

  • SKILL.md
  • architecture-patterns.md
  • configuration.md
  • decision-making.md
  • deployment.md
  • integrations.md
  • limits-quotas.md
  • security.md

Open the folder on GitHubat commit ba74e8f

Used in 1 other repository

We found 1 copy of this SKILL.md (exact, near-identical or edited) in other folders, from 1 other GitHub owner. This page covers the copy in MicrosoftDocs/Agent-Skills, which our catalogue first saw on October 7, 2026.

Compare with similar skills

Azure Databricks next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Azure Databricks compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Azure Databricks this skillMicrosoftDocs/Agent-Skills7751 repos~14kAutomated safety check: PassCC-BY-4.0
Update Provider Depsmondoohq/mql411—~4.5kAutomated safety check: PassCustom licence
Data Engineerdavila7/claude-code-templates32k7 repos~2.8kAutomated safety check: PassMIT
ML AIgrafana/skills278—~1.3kAutomated safety check: PassApache-2.0
Azure Architecture Autopilotgithub/awesome-copilot40k1 repos~1.9kAutomated safety check: PassMIT
Image Managementdotnet/dotnet-docker4.9k—~1.1kAutomated safety check: PassMIT

Similar skills

  • Upgrade an mql provider's vendored SDKs (all providers or a named subset), audit the new versions for breaking changes and fix call sites while keeping shipped MQL fields backwards-compatible, check…

    411 GitHub stars~4.5k tokensUpdated today
    DevOps & CloudAuto-check passed
  • Data Engineer

    davila7/claude-code-templates

    Build scalable data pipelines, modern data warehouses, and real-time streaming architectures.

    32k GitHub starsUsed in 7 repos~2.8k tokens
    Data & AnalyticsAuto-check passed
  • ML AI

    grafana/skills

    Official

    Turn on AI + ML features in Grafana Cloud — Grafana Assistant (NL → PromQL/LogQL/TraceQL, dashboard build, incident investigation, MCP integration), Dynamic Alerting (Prophet forecasting + DBSCAN…

    278 GitHub stars~1.3k tokensUpdated today
    DevOps & CloudAuto-check passed
  • Azure Architecture Autopilot

    github/awesome-copilot

    Official

    Designs Azure infrastructure from a natural-language description, or diagrams an existing resource group, then refines the design through conversation and deploys it with Bicep.

    40k GitHub starsUsed in 1 repo~1.9k tokens
    DevOps & CloudAuto-check passed
  • Image Management

    dotnet/dotnet-docker

    Official

    Manages .NET Docker images including adding images for new .NET versions, new Linux distros (Alpine, Ubuntu, Azure Linux), and new Windows versions.

    4.9k GitHub stars~1.1k tokensUpdated today
    DevOps & CloudAuto-check passed
  • Cookbook Aiml

    databricks-solutions/databricks-apps-cookbook

    Invoke ML models, run vector search, and connect to MCP servers from Databricks Apps.

    183 GitHub stars~1.7k tokensUpdated 3 days ago
    AI & LLM EngineeringAuto-check passed

More from MicrosoftDocs/Agent-Skills

All 149 skills in this repo
  • Azure Personalizer

    MicrosoftDocs/Agent-Skills

    Official

    Expert knowledge for Azure AI Personalizer development including troubleshooting, decision making, security, configuration, and integrations & coding patterns.

    775 GitHub starsUsed in 1 repo~1.1k tokens
    Auto-check passed
  • Azure Architecture Advisor

    MicrosoftDocs/Agent-Skills

    Official

    Guides Azure solution design by category, from reference architectures and design patterns to technology choices and migrations, fetching current Microsoft Learn pages over the network.

    775 GitHub stars~15k tokensUpdated yesterday
    Auto-check passed
  • Azure Advisor Guidance

    MicrosoftDocs/Agent-Skills

    Official

    Reference guidance for Azure Advisor work: recommendations, alerts and digests, workbooks, RBAC access and sovereign-cloud limits, fetched from Microsoft Learn.

    775 GitHub stars~1.7k tokensUpdated yesterday
    Auto-check passed
  • Azure AI Vision Reference

    MicrosoftDocs/Agent-Skills

    Official

    Looks up Microsoft Learn guidance for Azure AI Vision: Image Analysis, Read OCR containers, smart-crop thumbnails, background removal and video frame analysis, plus limits and deployment.

    775 GitHub stars~1.6k tokensUpdated yesterday
    Auto-check passed
  • Azure Analysis Services

    MicrosoftDocs/Agent-Skills

    Official

    Expert knowledge for Azure Analysis Services development including troubleshooting.

    775 GitHub stars~608 tokensUpdated yesterday
    Auto-check passed
  • Azure Anomaly Detector

    MicrosoftDocs/Agent-Skills

    Official

    Expert knowledge for Azure AI Anomaly Detector development including troubleshooting, best practices, limits & quotas, configuration, and deployment.

    775 GitHub stars~1.1k tokensUpdated yesterday
    Auto-check passed

Questions about Azure Databricks

What does Azure Databricks do?

Expert knowledge for Azure Databricks development including troubleshooting, best practices, decision making, architecture & design patterns, limits & quotas, security, configuration, integrations &…. Azure Databricks is an agent skill from MicrosoftDocs/Agent-Skills, published by the product's own GitHub organization. Expert knowledge for Azure Databricks development including troubleshooting, best practices, decision making, architecture & design patterns, limits & quotas, security, configuration, integrations & coding patterns, and deployment.

When should I use Azure Databricks?

Azure Databricks fits situations like: working with Unity Catalog; lakehouse/Lakeflow; AI Runtime/model serving; external connectors.

How do I install Azure Databricks in Claude Code?

Run `npx skills add MicrosoftDocs/Agent-Skills --skill azure-databricks -a claude-code`. Or copy the skill folder (skills/azure-databricks in MicrosoftDocs/Agent-Skills) into .claude/skills/azure-databricks in your project. Claude Code loads it when a task matches its description.

How do I install Azure Databricks in Codex?

Run `npx skills add MicrosoftDocs/Agent-Skills --skill azure-databricks -a codex`. Or copy the skill folder (skills/azure-databricks in MicrosoftDocs/Agent-Skills) into .agents/skills/azure-databricks in your project. Codex loads it when a task matches its description.

Can I use Azure Databricks in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add MicrosoftDocs/Agent-Skills --skill azure-databricks -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/azure-databricks, .gemini/skills/azure-databricks, .github/skills/azure-databricks and .opencode/skills/azure-databricks in your project.

What does Azure Databricks need to run?

SKILL.md names no scripts, command-line tools or credentials: Azure Databricks is instructions for the agent only. Our summary lists: Python 3. Compatibility (from SKILL.md): Requires network access. Uses mcp_microsoftdocs:microsoft_docs_fetch or fetch_webpage to retrieve documentation..

Does Azure Databricks access the network?

SKILL.md names 2 domains. As links in the text: learn.microsoft.com and github.com. This is read from the text; nothing was executed.

Is Azure Databricks safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Azure Databricks use?

Azure Databricks is published under the CC-BY-4.0 licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Azure Databricks use?

About 14k tokens (SKILL.md is roughly 57k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Azure Databricks?

Skills that share tags, products or a category with Azure Databricks: Update Provider Deps (mondoohq/mql, 411 stars), Data Engineer (davila7/claude-code-templates, 32k stars), ML AI (grafana/skills, 278 stars) and Azure Architecture Autopilot (github/awesome-copilot, 40k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Azure Databricks?

MicrosoftDocs (a GitHub organization, an official publisher) maintains it in MicrosoftDocs/Agent-Skills, which has 775 GitHub stars. The repository holds 149 skills in this directory. The repository was last updated on October 5, 2026.

Source: MicrosoftDocs/Agent-Skills on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.