跳到主要内容
AI News HubLIVE
更多
来源内容 · 翻译待补全1 分钟阅读

待翻译:Serverless routing made sticky

文章摘要

AI 服务暂时不可用,以下为来源摘要,待恢复后补全翻译:Voice agents, multiplayer games, and other workloads can now build on Sticky Sessions. Achieve strong affinity without losing autoscaling. Here’s how we built this on Modal Servers.

待翻译:Serverless routing made sticky
报告错误

纠错通道尚未开通,可先复制下方文章信息留存。

查看更正说明
直接读正文

AI 服务暂时不可用,以下为来源正文,待恢复后补全翻译。

At Modal, our serverless compute powers real-time inference, training, and batch jobs, scaling automatically with demand. Sandboxes offer a different model, containers whose lifecycles you explicitly control. A growing number of workloads require something in-between; autoscaling serverless compute that maintains continuity between requests. Consider a speech-to-text model with decoder state for an audio stream, a voice agent that holds a conversation, or a multiplayer game. For these workloads, users must come back to the same container on every reconnect for the entirety of their session. We recently launched Sticky Sessions, built on top of our low-latency Modal Servers. Every session’s requests land on the same container. To use Sticky Sessions, just create a Modal Server with the @modal.sessioned() decorator and start the sessions with the SDK or the HTTP API: @app.server() @modal.sessioned() class Server: @modal.enter() def start(self): subprocess.Popen(["python", "-m", "http.server", "8000"]) @app.local_entrypoint() def main(): session = Server.sessions.start(idle_timeout=600) headers = {"Modal-Authorization": f"Bearer {session.token}"} print(requests.get(Server.get_url(), headers=headers).text)

展开要点与分析

文章情报

工程师中级

要点

  • AI 服务暂时不可用,系统已先保留来源内容与降级元数据。
  • Voice agents, multiplayer games, and other workloads can now build on Sticky Sessions. Achieve strong affinity without losing autoscaling. Here’s how we built this on Modal Server…

要点与分析由自动化流程生成,可能有误,请结合原始来源核实。