2026-07-09 15:38 UTCIn-site rewrite6 min readUpdated: 2026-07-09 22:55 UTC

LLM Orchestration Frameworks Compared: LangChain vs. LlamaIndex vs. Raw API Calls

A comparison of LangChain, LlamaIndex, and raw API calls for LLM applications, covering their strengths, trade-offs, and a decision framework for choosing the right abstraction level.

SourceMachine Learning MasteryAuthor: Shittu Olumide

LLM Orchestration Frameworks Compared: LangChain vs. LlamaIndex vs. Raw API Calls - MachineLearningMastery.com

In this article, you will learn how LangChain, LlamaIndex, and raw API calls each solve a different layer of the LLM application stack, and how to choose among them based on what your project actually requires.

Topics we will cover include:

What each option is designed to do, stated plainly without marketing spin.

How the three approaches compare on performance, token overhead, debugging clarity, and code volume.

A practical decision framework for picking the right level of abstraction before you build — and before that choice becomes expensive to undo.

Let’s not waste any more time.

Introduction

You have a working prompt. The model is giving good answers. Then the next requirement lands. Maybe it is memory; the model needs to remember what was said three messages ago. Maybe it is retrieval — the model needs to answer questions about documents it was not trained on. Maybe it is tool use; the model needs to check a database, run a calculation, or call an external API before it can respond. Suddenly, a single client.chat.completions.create() call is not enough, and you are standing at the first real architectural decision in your LLM project.

Three paths exist from that moment: reach for LangChain, reach for LlamaIndex, or build a thin layer on top of the raw SDK yourself. Getting this choice wrong does not break the prototype. It breaks the production system six months later, when you are debugging stack traces 40 frames deep, paying 2.7x what you should be on token costs, or spending a sprint migrating away from breaking API changes.

LLM API spend doubled from \$3.5 billion to \$8.4 billion between late 2024 and mid-2025. These are real production budgets. The framework layer — the code that sits between your application and the model — directly determines how much of that spend is doing useful work versus paying for abstraction you did not need.

This article gives you an honest comparison: what each option actually is, where it genuinely wins, where it costs you, and a decision framework you can use tomorrow.

The Landscape in Plain English

Before comparing trade-offs, it helps to understand what each option actually is — not what its marketing says, but what problem it was built to solve.

LangChain started in October 2022 as a general-purpose framework for chaining LLM operations together. Its core idea was that building real applications required composing multiple steps — prompt templates, model calls, output parsers, memory, tools — and there should be a standard way to do that. It has grown into the largest LLM framework by adoption: 119K GitHub stars, 500+ integrations, and a sprawling ecosystem. The LangChain team now builds LangGraph, a separate package for stateful, graph-based agent workflows, as the recommended way to build production agents within the ecosystem.

LlamaIndex (launched as GPT Index in November 2022) was built to solve a different problem: getting LLMs to reason over your own data. Its design is organized around data ingestion, chunking, embedding, indexing, and retrieval. Where LangChain is about orchestrating what happens between steps, LlamaIndex is about making the retrieval step itself as accurate and efficient as possible. It sits at 44K GitHub stars with 300+ data connectors through LlamaHub, covering sources like Notion, Google Drive, Slack, PDFs, and databases.

Raw API calls means using the OpenAI Python SDK, the Anthropic SDK, or any model provider’s client directly — no orchestration layer, no abstractions beyond what the provider ships. You write the prompt, call the model, and handle the response yourself. This is not the primitive fallback it is sometimes presented as; it is the approach production teams are increasingly migrating back to for workloads where the framework’s complexity stopped paying for itself.

The critical thing to understand before reading any comparison is that these three options are not competing on the same dimension. LangChain is an orchestration toolkit. LlamaIndex is a retrieval toolkit. Raw API calls are a stance on how much abstraction you need. Many production systems use two of them together. The question is always: given what I am actually building, which layer of abstraction earns its cost?

LangChain: The Orchestration Layer

LangChain’s strength is assembling complexity. If your application involves multiple steps, multiple tools, conditional routing, memory across turns, or agents that reason before acting, LangChain provides the building blocks for all of it, with connectors to 500+ services and a community large enough that someone has already solved most of the edge cases you will encounter.

LangGraph, built by the same team and stable at v1.0 since October 2025, is where the serious agent work lives now. It models agent workflows as directed graphs, where nodes are Python functions, edges are state transitions, and a central typed state object flows through the entire execution. It has built-in persistence via checkpointers to SQLite, PostgreSQL, or Redis, which means agents can pause mid-workflow, persist their state, and resume hours later. That is genuinely hard to build yourself and is one of LangChain’s clearest justifications in a production context.

The honest trade-offs are worth naming directly. LangChain adds ~10ms framework overhead per step, and LangGraph adds ~14ms. For most human-facing applications that make LLM calls taking 1–3 seconds each, this is irrelevant. For high-throughput pipelines processing thousands of requests per minute, it compounds. Stack traces from LangChain production errors routinely span 15 to 40 frames of internal framework code; finding the actual source of a bug is slower than in a system you wrote yourself. And for simple use cases, one documented comparison found LangChain incurring 2.7x higher costs than a native implementation for a basic RAG pipeline — the abstraction overhead consumed tokens that did not need to be consumed.

LangChain v1.0 (October 2025) committed to API stability after a turbulent v0.1 through v0.3 period that forced multiple breaking migrations. That history is worth knowing. For new projects, the stability concern is largely resolved. For teams running v0.x code in production, the migration cost to v1.0 is real.

Here is a working LangChain LCEL chain — the modern way to compose LangChain operations.

Prerequisites:

pip install langchain langchain-openai python-dotenv

How to run: Save as langchain_chain.py, add OPENAI_API_KEY to your .env, run python langchain_chain.py

langchain_chain.py

A LangChain LCEL chain: prompt template → model → output parser

Prerequisites: pip install langchain langchain-openai python-dotenv

How to run: python langchain_chain.py

import os

from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate

from langchain_core.output_parsers import StrOutputParser

from langchain_openai import ChatOpenAI

load_dotenv()

── MODEL ─────────────────────────────────────────────────────────────────────

ChatOpenAI wraps OpenAI's chat models. Swap the model string to switch

to gpt-4o-mini (cheaper) or claude-3-5-sonnet (via langchain-anthropic) --

the chain code below stays identical either way. This model portability

is one of LangChain's genuine advantages over raw API calls.

llm = ChatOpenAI(

model="gpt-4o",

temperature=0.2,

api_key=os.getenv("OPENAI_API_KEY")

)

── PROMPT TEMPLATE ───────────────────────────────────────────────────────────

ChatPromptTemplate defines the message structure with named variables.

{topic} gets filled in at runtime -- templates are reusable and versionable.

prompt = ChatPromptTemplate.from_messages([

("system", "You are a concise technical explainer. Keep answers under 100 words."),

("human", "Explain {topic} in simple terms.")

])

── OUTPUT PARSER ─────────────────────────────────────────────────────────────

StrOutputParser extracts the text content from the model's AIMessage response.

Without it you get back an AIMessage object rather than a plain string.

parser = StrOutputParser()

── CHAIN (LCEL) ──────────────────────────────────────────────────────────────

The pipe operator (|) builds a sequential chain: prompt → llm → parser.

LCEL (LangChain Expression Language) makes the composition readable and

supports streaming, batching, and async execution with the same interface.

chain = prompt | llm | parser

if name == "main":

invoke() runs the full chain synchronously

result = chain.invoke({"topic": "vector embeddings"})

print(result)

stream() yields tokens as they arrive -- no code changes needed for streaming

print("\n--- Streaming response ---")

for chunk in chain.stream({"topic": "RAG pipelines"}):

print(chunk, end="", flush=True)

print()

What this does: Three objects — prompt, llm, parser — are connected with the | operator. LangChain’s LCEL executes them in order: the template fills in {topic}, passes a formatted message to the model, and the parser extracts a plain string from the response. The same chain supports .invoke(), .stream(), .batch(), and .ainvoke() without any changes to the chain definition itself. That interface consistency is the clearest argument for LangChain on projects that need multiple execution patterns.

Here is the same foundation extended to a tool-using agent with LangGraph.

Prerequisites:

pip install langchain langchain-openai langgraph langchain-community python-dotenv

How to run: Save as langchain_agent.py and run python langchain_agent.py

langchain_agent.py

A LangGraph ReAct agent with two tools: web search and a calculator

Prerequisites: pip install langchain langchain-openai langgraph langchain-community python-dotenv

How to run: python langchain_agent.py

import os

from dotenv import load_dotenv

from langchain_openai import ChatOpenAI

from langchain.tools import tool

from langchain_community.tools import DuckDuckGoSearchRun

from langchain_core.messages import HumanMessage

from langgraph.prebuilt import create_react_agent

load_dotenv()

llm = ChatOpenAI(model="gpt-4o", temperature=0, api_key=os.getenv("OPENAI_API_KEY"))

Web search -- no API key required

search = DuckDuckGoSearchRun()

@tool

def calculate(expression: str) -> str:

"""

Evaluate a safe mathematical expression. Use for arithmetic or percentage calculations.

Input: a Python math expression string (e.g., '1500 * 0.08').

"""

try:

result = eval(expression, {"builtins": {}}, {})

return f"Result: {result}"

except Exception as e:

return f"Error: {str(e)}"

tools = [search, calculate]

create_react_agent wires together the LLM, tools, and a built-in ReAct loop.

The agent thinks, calls a tool, reads the result, and continues until done.

agent = create_react_agent(llm, tools)

if name == "main":

result = agent.invoke({

"messages": [HumanMessage(content="What is 15% of 2400?")]

})

print(result["messages"][-1].content)

What this does: create_react_agent abstracts the full reasoning loop. The model decides whether to use a tool, LangGraph executes the selected tool, feeds the result back into the message history, and repeats until the model has a final answer. What would take 50+ lines in a raw implementation is four lines here. That abstraction is appropriate when y

[truncated for AI cost control]