待翻译：LLMPanel Deploy vLLM to RunPod or Vast.ai Without Kubernetes

AI 服务暂时不可用，以下为来源正文，待恢复后补全翻译。

Open-source LLM deployment platform Deploy LLMs on any GPU, anywhere Pick a model, pick a GPU, hit deploy. LLMPanel provisions the container, exposes an OpenAI-compatible endpoint, and streams every GPU metric back to one dashboard — on your own hardware or any cloud. Start for free Self-host in 5 minutes MIT licensed · no GPU lock-in · works with vLLM and Ollama LLMPanel acme-ml 3 running · 3 GPUs Deployments4 total · 3 running NameGPUtok/sStatus Click a row to inspect · start / stop from the panel llama-3.3-70b Modelmeta-llama/Llama-3.3-70B-Instruct ProviderRunPod GPU2× H100 80GB Cost$4.38/hr Endpointllama-70b.gw.llmpanel.io/v1 VRAM81% Throughput1420 tok/s Your own GPUs, or burst to any cloud RunPod Vast.ai Lambda CoreWeave Nebius Crusoe Hyperstack FluidStack Paperspace TensorDock RunPod Vast.ai Lambda CoreWeave Nebius Crusoe Hyperstack FluidStack Paperspace TensorDock The problem Running your own LLMs shouldn't need a platform team Self-hosting inference today means stitching together clouds, containers, keys and dashboards. Every piece works — none of it works together. Before Scattered tooling A different dashboard for every GPU cloud Hand-written docker run commands nobody remembers One API key shared by every app and teammate nvidia-smi in a tmux pane is the monitoring No idea what a model actually costs per hour With LLMPanel One control plane for inference Docker hosts, RunPod and Vast.ai in one fleet view Pick a model, pick a GPU, deploy — vLLM or Ollama Scoped keys with rpm / tpm / daily caps and revocation Per-GPU utilisation, VRAM and temperature, live Cost per hour and uptime on every deployment One-click deployments Pick a model from the catalog, choose a GPU, hit deploy. The container, weights and endpoint are handled in minutes. OpenAI-compatible gateway Every deployment sits behind one API with managed keys, rate limits and routing. Swap models without touching your app. Real-time GPU monitoring Utilisation, VRAM, temperature and tokens per second streamed live to your dashboard for every card. Multi-provider fleet Run on your own Docker hosts or burst to RunPod and Vast.ai with your API keys — the same workflow either way. Gateway One endpoint, scoped keys, no surprises Every deployment sits behind one OpenAI-compatible base URL. Hand each app or environment its own key with rpm, tpm and daily caps — and revoke it in one click when it leaks. Learn more Gateway · API keys NameKeyrpmUsage today web-app · prodactivelp_live_8f2a…60071% mobile · prodactivelp_live_c91d…30042% evals · stagingactivelp_test_17b0…608% limits · rpm · tpm · daily cap · expiry — per key Metrics See what every GPU is doing, right now Utilisation, VRAM, temperature and power draw for every card across Docker hosts and cloud providers, plus container CPU and RAM — live and historical, no exporters to wire up. Learn more Metrics · fleet GPU 0 · H100 80GBrunpod-eu-1 util92% vram81% 67°C540 W GPU 1 · H100 80GBrunpod-eu-1 util88% vram79% 65°C520 W GPU 0 · A100 80GBdocker · lab-01 util74% vram64% 58°C290 W GPU 0 · RTX 4090vast · 7713 util61% vram88% 71°C380 W Logs Drill into any request Status, latency, prompt and completion token split, and which key made the call. Filter by status, model, key or time range and open the full payload inline. Learn more Logs · requests idmodel · keystatusmstokens Agents & RAG Ground a model in your own docs Attach an agent to any deployment with its own system prompt and knowledge base. Upload files, get chunking and embeddings for free, and test it in the built-in chat before wiring it to Telegram or Discord. Learn more Agents · docs-assistantllama-3.3-70b · rag on you Which GPU do I need for Llama 3.3 70B? assistant Llama 3.3 70B in FP8 needs roughly 70 GB of VRAM for weights, so a single H100 80GB works for short contexts. For 32k context, deploy on 2× H100 with tensor parallelism — the catalog preset does this for you. source · docs/sizing-guide.md Core operations Everything between a checkpoint and production LLMPanel covers the full life of a model in production — so your team stops stitching tools together. Deploy Ship any Hugging Face model on vLLM or Ollama to a Docker host or a cloud GPU in one flow. Route One OpenAI-compatible base URL in front of every deployment. Point your SDK at it and go. Monitor Per-GPU metrics, request logs and usage analytics — live and historical, no exporters to wire up. Scale Add a second host or burst to RunPod when demand spikes; the gateway keeps the same URL. Secure Scoped API keys with rpm, tpm and daily caps, expiry, per-key usage and one-click revocation. Back up Snapshot panel data and config on a schedule, and restore from any point when you need to. How it works Live in an afternoon No Kubernetes, no YAML. Three steps from a bare server to your first OpenAI-compatible endpoint. 01Step 01 Install the panel One command on any Linux box with Docker. It brings up the API, the dashboard and the gateway, and prints the login URL. bash curl -fsSL https://raw.githubusercontent.com/zf0x00/openpanel-oss/master/install.sh | bash 02Step 02 Connect your GPUs Add a local Docker host, or paste a RunPod / Vast.ai API key. Every card shows up in the fleet view with live metrics. Local Docker hostconnected RunPod API keyconnected Vast.ai API keyconnected 03Step 03 Deploy and call it Pick a model, pick a GPU, deploy. Point any OpenAI SDK at the gateway URL with a scoped key — nothing else changes. python from openai import OpenAI client = OpenAI( base_url="https://gw.llmpanel.io/v1", api_key="lp_live_…", ) client.chat.completions.create( model="llama-3.3-70b", messages=[{"role": "user", "content": "hi"}], ) Integrations Works with your stack Bring the GPUs, engines and SDKs you already use. LLMPanel sits in the middle without asking you to migrate anything. DockerLocal hosts RunPodCloud GPUs Vast.aiCloud GPUs LambdaCloud GPUs HyperstackCloud GPUs vLLMInference engine OllamaInference engine Hugging FaceModel catalog OpenAI SDKAny language TelegramChannel DiscordChannel WebhooksEvents & alerts Pricing Self-host free, or let us run it LLMPanel is open source and free to self-host, forever. Pick a cloud plan when you'd rather not babysit the panel itself. Self-Hosted $0forever Free forever. Run the full platform on your own hardware or cloud. Open source, no strings attached Deploy yourself Full platform, every feature included Unlimited deployments & models Bring your own GPUs or cloud keys OpenAI-compatible gateway & API keys Community support on GitHub Founder pricingSolo $19/mo For one developer running a handful of models. We host the panel. First 50 customers: $9/mo, forever Start with Solo 4 deployments included $8/mo per extra deployment Single seat Hosted dashboard — no panel to babysit OpenAI-compatible gateway & API keys Most popularTeam $49/mo For teams sharing a GPU fleet across one workspace. Extra deployments just $5/mo each Start with Team Everything in Solo 10 deployments included $5/mo per extra deployment Unlimited team members Shared workspace with roles & invites Every plan includes the OpenAI-compatible gateway and per-GPU metrics. Put your models on your own GPUs today Install in five minutes, deploy in one click, and call it from any OpenAI SDK. Free to self-host, forever. Start for freeStar on GitHub