待翻译：Show HN: OSS, K8s-native AI platform for distributed multi-model inference

AI 服务暂时不可用，以下为来源正文，待恢复后补全翻译。

Uh oh! There was an error while loading. Please reload this page. Notifications You must be signed in to change notification settings Fork 0 Star 12 BranchesTags Open more actions menu Latest commit History 23 Commits 23 Commits Folders and files NameName Last commit message Last commit date .github .github app_mcp app_mcp app_serving app_serving app_shaide app_shaide docs docs harbor harbor infra infra installer installer monitoring monitoring pkg pkg .gitignore .gitignore CONTRIBUTING.md CONTRIBUTING.md LICENSE LICENSE README.md README.md RELEASE.md RELEASE.md docs-requirements.txt docs-requirements.txt hooks.py hooks.py mkdocs.yml mkdocs.yml serve-docs.sh serve-docs.sh Repository files navigation Distributed, multi-model LLM inference on Kubernetes you own - installed by a single command, all the way down to air-gapped clusters. Quickstart · Architecture · Documentation · Contributing · Join the Community Important shaide is in early access. The platform runs production workloads today, but this repository is young: interfaces, stack configuration and module layout may still change between releases. Pin a release rather than tracking main, and please open an issue if something breaks. What is shaide? shaide is a self-hosted AI platform that serves AI models at scale on your own Kubernetes clusters. The goal is to run many models side by side, each with multiple replicas, and route traffic across them. Standing up enterprise AI infrastructure today means assembling a long list of moving parts - an inference engine, a serving orchestrator, a gateway, a model registry, storage, observability, and that is only the beginning. Each one has to be chosen, configured and glued to the next, component by component, then again for every environment. shaide ships that whole stack as one installable platform. The entire platform is managed as infrastructure as code. Every layer - the internal registry, the gateway, model serving and the application layer - is defined as a Pulumi project in this repository, so your AI infrastructure is versioned and reproducible. And it stays inside your perimeter. shaide is built for organisations whose data cannot leave their infrastructure: regulated industries, defence, public sector, or anyone who simply will not send prompts to a third-party API. There is nothing phoning home and no dependency on a vendor's cloud - including fully air-gapped installations with no internet access at all. Why shaide Sovereign by design. Everything runs in your infrastructure. An internal OCI registry mirrors every container image and model weight, so a cluster can operate with no egress whatsoever. Installed by one command. Point the interactive terminal installer at a prepared cluster and it installs the entire platform through guided prompts. No Helm chart archaeology, no twelve READMEs to follow in order. Distributed multi-model serving. vLLM as the inference engine, llm-d for multi-instance orchestration. Serve generative and embedding models concurrently, each scaled independently across GPU nodes. Built for agent fleets. Multi-model routing, KV-cache-aware scheduling and inference-pool load balancing keep many concurrent agents served from one platform. Runs anywhere Kubernetes runs. AWS EKS, GCP GKE, Azure AKS and on-prem RKE2 - the same platform and the same installer across all of them. OpenAI-compatible API. Point any OpenAI-compatible SDK, agent framework or tool at your endpoint and change nothing but the base URL. Infrastructure as Code. Every layer is a Pulumi Go program, so deployments are reviewable, diffable and reproducible. Quickstart 1. Prepare a cluster shaide installs onto an existing Kubernetes cluster. Check it against the prerequisites: → Prerequisites 2. Choose your models The installer ships with everything it deploys. The one input you supply is a model manifest listing the models to publish into the internal registry. Important Supplying models.yaml by hand is a temporary step. Model selection moves into the installer in the next release, and this file will no longer be required. mkdir -p /tmp/manifests cat > /tmp/manifests/models.yaml PULUMI_CONFIG_PASSPHRASE= HF_TOKEN= mkdir -p "${STORAGE_PATH}" docker run --rm -it \ --network host \ -e PULUMI_CONFIG_PASSPHRASE \ -e HF_TOKEN \ -e MODEL_MANIFEST_PATH=/manifests/models.yaml \ -v "$HOME/.kube/config:/.kube/config:ro" \ -v /tmp/manifests/models.yaml:/manifests/models.yaml:ro \ --mount "type=bind,src=${STORAGE_PATH},dst=/var/shaide-installer" \ ghcr.io/axem-solutions/shaide/installer:oss → Installer guide 4. Verify Once the installer completes, check that the platform is serving: curl https:///v1/models Using the API shaide exposes an OpenAI-compatible API, so existing clients work unchanged: from openai import OpenAI client = OpenAI( base_url="https:///v1", api_key="", ) response = client.chat.completions.create( model="", messages=[{"role": "user", "content": "Hello!"}], ) print(response.choices[0].message.content) Architecture shaide is built in layers, each deployed as an independent Pulumi program: Layer What it does Platform services The internal OCI registry that holds every image and model weight, plus the shared Istio Gateway and Gateway API layer. Serving Per-model inference stacks: vLLM engine pods orchestrated by llm-d, fronted by an inference gateway that routes and load-balances across replicas. Application The shaide server - the universal API surface, authentication and user management - together with the control panel UI. Packages The interactive installer, the observability stack, and shared libraries used across every layer. Deployment proceeds in a fixed order: 1. Internal OCI registry harbor 2. Gateway + Istio infra/gateway-provider 3. Model serving app_serving 4. Application layer app_shaide 5. Optional add-ons app_mcp, monitoring The shaide components shaide is developed across three open-source repositories: Repository Role shaide This repository - the core: infrastructure, model serving and the installer. shaide_server The universal interface to every service the platform provides, plus authentication and user management. shaide_control_panel The web UI for operating the platform. Repository layout ├── app_serving/ Per-model LLM serving stacks (vLLM + llm-d) ├── app_shaide/ Application layer: shaide server and control panel ├── app_mcp/ MCP server datasources deployed into the shared gateway ├── harbor/ Internal OCI registry for images and model weights ├── infra/ In-cluster platform services: the shared gateway ├── installer/ Containerized interactive installer ├── monitoring/ Observability stack └── pkg/ Shared Go module with the Pulumi deployment logic Full technical documentation lives at docs. Contributing Contributions are welcome. See CONTRIBUTING.md for development setup, coding standards and the review process. Security Please do not report security vulnerabilities through public issues. See our security policy for how to disclose them responsibly. License Licensed under the Apache License 2.0. See LICENSE for details. shaide Copyright 2026 axem solutions Kft. This product includes software developed by third parties. See their respective licensing for additional copyright and licensing information. Topics Resources Readme Apache-2.0 license Code of conduct Code of conduct Contributing Contributing Security policy Security policy Activity Custom properties Stars 12 stars Watchers 0 watching Forks 0 forks Report repository