2026-07-30 11:55 UTCIn-site rewrite3 min readUpdated: 2026-07-30 12:01 UTC

Go LLM SDK for streaming, tool-calling AI backends (plus frontend React lib)

Grafana's open-source Go SDK for LLMs, compatible with Vercel's AI SDK, supports streaming, tool calling, structured output, multiple providers (Anthropic, Bedrock, OpenAI, etc.), and seamless integration with React frontends.

SourceHacker News AIAuthor: matryer

Uh oh!

There was an error while loading. Please reload this page.

Notifications You must be signed in to change notification settings

Fork 0

Star 18

BranchesTags

Open more actions menu

Folders and files

NameName

Last commit message

Last commit date

Latest commit

History

4 Commits

.agents/skills

.claude/skills

.github

docs

examples

fallback

gateway

internal

middleware

openspec

output

provider

providers

registry

schema

scripts

test

.gitignore

.golangci.yml

.markdownlint-cli2.jsonc

AGENTS.md

CODE_OF_CONDUCT.md

CONTRIBUTING.md

LICENSE

NOTICE

README.md

agent.go

agent_test.go

chunk.go

chunk_test.go

convert.go

convert_test.go

doc.go

extract_content.go

extract_content_test.go

generatetext.go

go.mod

go.sum

go.work

go.work.sum

http.go

http_test.go

id.go

id_test.go

integration_test.go

message.go

message_json.go

message_json_test.go

mise.toml

missing_tool_results_error.go

options.go

options_test.go

output.go

renovate.json

retry.go

retry_error.go

retry_integration_test.go

retry_test.go

stream.go

stream_test.go

streamtext.go

streamtext_output_test.go

streamtext_test.go

text.go

textstream.go

to_response_messages.go

to_response_messages_test.go

tool.go

tool_approval_errors.go

tool_approval_signature.go

tool_approval_signature_test.go

tool_fingerprint.go

tool_fingerprint_test.go

typed_tool.go

typed_tool_test.go

types.go

ui_message_reader.go

ui_message_reader_test.go

Repository files navigation

Why

The SDK gives Go applications one API for model calls, streaming, tools, structured output, and multi-step agents across supported providers. It follows the design of Vercel's AI SDK and stays wire-compatible with its TypeScript frontend hooks. A Go endpoint can stream Server-Sent Events (SSE) directly to hooks such as useChat.

Go backend React frontend ────────── ────────────── aisdk.StreamText(...) ── SSE ──▶ useChat({ transport }) aisdk.WriteUIMessageStream(w, …) // same protocol

See How a request runs for the generation, tool, and streaming flow. Reuse an existing AI SDK React frontend or replace a TypeScript backend with Go without adding a protocol adapter.

Features

StreamText / GenerateText — stream a response or wait for the complete result, with retries and multi-step tool execution

React compatibility — serve useChat, useCompletion, and useObject

Composable tools — call plain Go functions from a model and require approval for consequential actions

Structured output — generate schema-validated objects, arrays, and choices

Multiple providers — call Anthropic, Amazon Bedrock, OpenAI, OpenAI-compatible APIs, and Grafana's hosted endpoint from internal services

Production controls — configure timeouts, fallback, logging, Prometheus metrics, and Agent Observability

Install

Create a Go project and install the core module and one provider:

mkdir ai-sdk-quickstart cd ai-sdk-quickstart go mod init example.com/ai-sdk-quickstart go get github.com/grafana/ai-sdk go get github.com/grafana/ai-sdk/providers/anthropic

See Choose a provider for Amazon Bedrock, OpenAI, OpenAI-compatible APIs, and the internally provisioned Grafana hosted endpoint.

Quick start

Save this complete program as main.go. It makes one model call and prints the response:

package main

import ( "context" "fmt" "log" "os"

aisdk "github.com/grafana/ai-sdk" "github.com/grafana/ai-sdk/provider" "github.com/grafana/ai-sdk/providers/anthropic" )

func main() { apiKey := os.Getenv("ANTHROPIC_API_KEY") if apiKey == "" { log.Fatal("ANTHROPIC_API_KEY is required") }

model := anthropic.New(apiKey, "claude-sonnet-5") result, err := aisdk.GenerateText(context.Background(), model, aisdk.WithModelMessages(provider.UserText("Explain goroutines in one sentence.")), ) if err != nil { log.Fatal(err) }

fmt.Println(result.Text) }

Run it with an Anthropic API key:

ANTHROPIC_API_KEY=sk-... go run .

For project initialization and credential guidance, follow Installation. To stream this response to a React client, continue with Build a full-stack chat.

Where to go next

Goal Start here

Make model calls from Go Generate text from Go

Build a React chat Full-stack chat

Return typed data Structured output

Let a model call Go code Tools

Build a reusable agent Agent loops

Choose a model provider Provider overview

Add logging or observability Middleware overview

Prepare for production Production checklist

Full index: Documentation · Runnable code: Examples · Exact APIs: pkg.go.dev

Contributing

Contributions are welcome. CONTRIBUTING.md covers the development setup, the two conventions that make this repository unusual — upstream parity with the Vercel AI SDK, and spec-driven development with OpenSpec — and the pull request checklist. All participants follow our Code of Conduct.

License

Apache License 2.0. This SDK follows the design of Vercel's AI SDK, also Apache-2.0 licensed; attribution is recorded in NOTICE.

Apache-2.0 license

Code of conduct

Contributing

Security policy

Activity

Custom properties

Stars

18 stars

Watchers

0 watching

Forks

0 forks

Report repository