2026-07-13 14:00 UTCIn-site rewrite4 min readUpdated: 2026-07-13 14:10 UTC

Structured Language Model Generation with Outlines

Outlines is an open-source library that introduces deterministic certainty into LLMs' output generation process for better, more reliable generation of structured outputs.

SourceKDnuggetsAuthor: Iván Palomares Carrascosa

--> Structured Language Model Generation with Outlines - KDnuggets

-->

Join Newsletter

Introduction

Usually, when asking an LLM — abbreviation for "Large Language Model" — for a neat, structured output like JSON objects, for instance, a mix of careful prompt crafting with a "pinch" of luck is required. Otherwise, it might be tricky to get the model to obtain the perfectly structured output you are expecting. Or so it was, until a novel open-source library came onto the scene: outlines.

This library is designed to prevent typical issues experienced by LLMs in these specific output-oriented use cases, such as hallucinations. More precisely, it introduces a degree of deterministic certainty into the output generation process.

Let's uncover what outlines allows us to do through this illustrative article, in which we will show some practical examples in Python!

Use Case 1: Multiple-Choice Classification for Sentiment Analysis

Before fully diving into the first use case, you might be wondering. How does outlines work and how does it guarantee correctness in structured model outputs? At the inference level, it masks out "syntactically illegal" tokens during generation instead of attempting to fix poor text once generated. This makes it virtually impossible to break the rules underlying the specific output format sought.

Let's see a first example in which we are building an analysis pipeline for customer support tickets, and we want exactly one option from a limited, approved list of possible options. This is pretty much like a classification problem, and generate.choice() is the function that helps us mimic it, by forcing the model at hand to choose one of the predefined literals or classes.

But first, let's install it alongside the transformers to load pre-trained LLMs:

pip install outlines[transformers]

The following code uses outlines.from_transformers() to load a pre-trained model aided by Hugging Face's auto classes for a model and its associated tokenizer. But the icing on the cake is: they are both wrapped in an outlines object that will later help tell the model what exactly to obtain. At the inference stage, we pass not only the user prompt asking to classify a review, but also a Literal object that contains the output constraints the model should limit to:

import outlines from transformers import AutoTokenizer, AutoModelForCausalLM from typing import Literal

1. Loading the backend using standard Transformer-based models

model_name = "microsoft/Phi-3-mini-4k-instruct"

We use outlines to load the model with its from_transformers() function

model = outlines.from_transformers( AutoModelForCausalLM.from_pretrained(model_name), AutoTokenizer.from_pretrained(model_name) )

2. Calling the model directly, passing our approved strings as type constraints

sentiment = model( "Classify the sentiment of this customer review: 'I've been waiting two weeks for my delivery and it's still missing.'", Literal["Positive", "Negative", "Neutral"] )

print(sentiment)

Output:

Negative

A word of warning here: although the literal we defined is part of Python's built-in typing module, rather than outlines, our out-of-the-box library still assumes model control here: both the model and tokenizer are wrapped into an object that enforces standard Python types, building a finite state machine under the hood that limits the output to the options provided only.

Use Case 2: JSON Object Generation

This example first defines a Pydantic object that defines the desired structure for a JSON object describing a fictional character with a name, description, and age. It then uses our previously wrapped outlines model, passing the character object to ensure the output generated strictly follows this structure for the JSON object requested:

from pydantic import BaseModel

1. Define a Pydantic model for the desired JSON structure

class Character(BaseModel): name: str description: str age: int

2. Using the outlines-wrapped model to generate a JSON output conforming to the Pydantic model

json_output = model( "Generate a JSON object describing a fictional character named 'Anya'.", Character, max_new_tokens=200 )

print(json_output)

Output:

{ "name": "Anya", "description": "Anya is a young, adventurous woman with a passion for exploring new places and meeting new people. She has long, curly hair and bright green eyes that sparkle with curiosity. Anya is always eager to learn and loves to share her knowledge with others. She is kind-hearted and always willing to lend a helping hand to those in need. Anya's favorite hobbies include hiking, reading, and playing the guitar. She is a free spirit who values freedom and independence above all else." ,"age": 25 }

Use Case 3: Pure JSON Generation for REST APIs

This third example, also JSON-related, is similar to the previous one but in a slightly different context. Imagine you are building an API backed requiring a well-defined JSON payload for updating a database. Asking a standard LLM to get this output will more often than not yield to annoying, trailing characters like commas that are likely to crash a JSON parser.

With outlines, we define our JSON payload schema once again with a Pydantic-based custom class object.

from pydantic import BaseModel from typing import Literal import json

class ServerHealth(BaseModel): service_name: str uptime_seconds: int status: Literal["OK", "DEGRADED", "DOWN"]

1. Outlines should produce a raw string guaranteed to be valid JSON

raw_json_string = model( "Report the current status of the main Auth database.", ServerHealth, max_new_tokens=50 )

print(type(raw_json_string)) # This will just print:

2. Pretty-printing

parsed_json = json.loads(raw_json_string) print(json.dumps(parsed_json, indent=2))

Output:

{ "service_name": "auth_db_status", "uptime_seconds": 1623456789, "status": "OK" }

Closing Remarks

Since LLMs are trained to be chat-lovers capable of breaking syntax or hallucinating to "sound like humans" in their conversations with us, getting them to produce reliable, structured outputs like clean JSON objects can feel like a bit of a pain. Outlines a new, open-source library that introduces deterministic certainty into LLMs' output generation process for better, more reliable generation of structured outputs. This article showed three simple yet useful use cases for beginners with this interesting tool.

Iván Palomares Carrascosa is a leader, writer, speaker, and adviser in AI, machine learning, deep learning & LLMs. He trains and guides others in harnessing AI in the real world.

Our Top 5 Free Course Recommendations

-->