待翻译:Run, debug, and scale Databricks workloads from your local IDE
AI 服务暂时不可用,以下为来源摘要,待恢复后补全翻译:The Databricks workspace is purposefully built for data analysis and data engineering. However...
AI 服务暂时不可用,以下为来源正文,待恢复后补全翻译。
Run, debug, and scale Databricks workloads from your local IDE | Databricks Blog Skip to main content Interactively run and debug Databricks workloads from your IDE: connect to Serverless, AI Runtime, and dedicated clusters. Minimize switches to the workspace: browse Unity Catalog and edit your workspace files directly from the IDE. Manage a single environment between the IDE and workspace: your files and project dependencies are always in sync. The Databricks workspace is purposefully built for data analysis and data engineering. However, you might prefer using local IDEs and the CLI to take advantage of your own tooling and coding agents like Cursor, Copilot, and Claude Code. This is especially true when developing complex, large-scale pipelines or machine learning models. Until now, the Databricks extension for Visual Studio and Cursor and Databricks Connect enabled local Spark development using Databricks compute. But running non-Spark workloads remotely and keeping dependencies in sync with Databricks Runtime remained common pain points. We’re now closing these gaps. With our latest updates to the IDE experience, you can now connect VS Code, Cursor, or your terminal directly to Databricks compute. Run, debug, and scale Python and SQL workloads on real cluster infrastructure while maintaining all of your IDE ergonomics. Remote execution without compromise Using our new SSH tunnel (see docs), you can connect your local editor or CLI to Serverless, AI Runtime, and dedicated clusters: Interactively run and debug workspace files and notebooks from VS Code, Cursor, or the CLI. Expand Use the same environment across the IDE and workspace, your dependencies and files are always in sync with Databricks Runtime and the workspace. Expand Leverage coding agents in the SSH tunnel so they have full workspace context and work with Databricks more effectively. Cursor and Copilot work out of the box, while other agents like Claude Code can be installed when the SSH tunnel is running. Expand Getting started is simple. You can connect to the SSH tunnel with a single command using the Databricks CLI: databricks ssh connect to connect to serverless. databricks ssh connect --accelerator to connect to AI Runtime where GPU type can be (GPU_1xA10 or GPU_8xH100). databricks ssh connect --cluster to connect to a dedicated cluster. You can also start the SSH tunnel in an IDE by including --ide vscode or --ide cursor as an additional flag. Alternatively, you can start the SSH tunnel directly from the most recent version of the IDE extension. We’ve also included other features that make it easier to use the CLI and IDE as your main place of work: Manage project dependencies (docs): Specify a workspace base environment with the --base-environment flag to start your SSH tunnel with Python dependencies pre-installed. see Monitor usage and costs (docs): Attach a serverless usage policy with the --usage-policy-id flag to track SSH tunnel costs by user, team, or project. Explore your data assets using Unity Catalog from the IDE (docs): Browse Catalogs, Schemas, and all your data assets without having to switch to the workspace in the midst of your development flow. Expand For full details on connecting to the SSH tunnel, check out the docs here. What’s next Unity AI Gateway will be automatically configured for SSH tunnel users, so you can govern access and spend on every agent, tool, model, and MCP. The existing IDE Extension will be integrated into the SSH tunnel, so you can deploy and manage Declarative Automation Bundles from a user interface in the IDE. Non-Python dependencies and custom Docker images will be configurable at SSH tunnel startup, so you can take full control of your environment. Conclusion With these features you can develop from whatever environment you prefer while operating at the frontier of data and ML engineering. Point your IDE and agents at Databricks, run and debug against real compute, and maintain a fast dev loop. Learn more + Next Steps To get started with the development tools showcased in the blog, check out the following documentation: ’SSH tunnel’ (AWS | Azure | GCP) Connect to Databricks compute to interactively run Python and SQL workloads from the IDE or CLI while keeping all code and data secure within your Databricks workspace. ‘IDE extension’ (AWS | Azure | GCP) Work with local files and define, deploy, and run Declarative Automation Bundles using a user interface in the IDE. ‘Databricks Connect’ (AWS | Azure | GCP) Connect your local development environment to Databricks compute to remotely run Spark workloads. ‘Unity AI Gateway’ (AWS | Azure | GCP) Control which AI services teams can use, route and manage AI traffic, set guardrails, and monitor usage from one control plane. To understand which tools best fit your needs, see Connect from your IDE. Get the latest posts in your inbox Subscribe to our blog and get the latest posts delivered to your inbox. Sign up View all blogs