How Harmonic Rebuilt Scout On Deep Agents And 4xd Retention With Langsmith

Case Studies

How Harmonic Rebuilt Scout on Deep Agents and 4x'd Retention with LangSmith

June 3, 2026

min

Go back to blog

Create agents

If you need to find or research startups, Harmonic is the place to go. Founded as a sourcing tool for venture capital firms, the platform now supports a diverse range of investors tracking what’s happening in private markets.

‍Scout is Harmonic’s AI. A conversational interface where users can query Harmonic’s data for questions from understanding the background of a founding team, to analyzing funding trends in a niche technology. Rebuilding Scout on Deep Agents and LangSmith accelerated product iteration from months to days, allowing the team to unlock new value akin to an "investment advisor” for users and expand their TAM to new markets.

The Cost of Maintaining A Hand-Built Pipeline

The first version of Scout simplified how users onboarded and interacted with Harmonic’s vast amounts of public and private data. Built on composable subgraphs with LangSmith evals at every node, Scout V1 was already a capable LangGraph-powered agent. Investors could search in natural language, and Scout would translate complex queries (for example, “show me AI companies in SF or NY that have raised funding in the last year from top investors and have a connection to someone on my team”) into precise filter configurations across Harmonic's data index.

Users no longer had to configure, create and save detailed filter parameters manually, but it was architecturally rigid. Any intent outside its workflows would fail, maintaining it required hundreds of evals, and each new capability meant a new subgraph.

When users started asking for things like personalized outreach drafts or abstract investment thesis searches, Scout couldn't keep up. Harmonic needed an architecture that could handle an open-ended range of tasks and improve as the underlying models evolved, without constant re-tuning.

A New Agent Harness: Rebuilding Scout on Deep Agents

Austin Berke, lead engineer on Scout, describes Scout V2 as the near-opposite of their previous approach:

"Before, we spent a lot of time building rigid structures, writing evals, and tediously iterating on every node. With this version, we said: let's use the best and smartest models, give it access to thoughtful tools that interface with our data, and see how Deep Agents does. And immediately we saw incredible improvements."

The new architecture is deliberately simple. A single frontier model in a Deep Agents harness with access to two categories of tools. One set queries Harmonic's global data layer (40M companies, 200M people, 230K investors). The other accesses firm-specific context including pipeline lists, CRM notes, prior email and LinkedIn connections.

The Deep Agents harness manages long-horizon task execution and context window management out of the box. Compared to the multi-graph pipeline from V1, it's a fraction of the complexity.

"The model has the agency it needs to not run into dead ends. We're always checking ourselves to keep the graph as simple as possible," emphasized Seth Wieder, Product Manager at Harmonic.

The team previously maintained an internal OKR targeting an 80% good Scout outcome rate. That “Scoutcome” metric is no longer actively discussed; consistent quality is now the default.

UX: From Search Tool to Trusted Advisor

Scout V1 worked the way most data tools work. Users typed a search, reviewed a list, refined, and repeated. Scout V2 functions less like a search box and more like a trusted advisor. You can hand it an investment thesis and ask it to pick five companies aligned with it. You can say "I'm meeting with this founder in 10 minutes; tell me everything I should know," and Scout will pull from your CRM, prior emails, LinkedIn connections, and the company's public profile, then synthesize across your entire system of record.

Scout V2 resolved a key agent UX tension. As Austin puts it: "Agents love things that feel like code. Users hate things that feel like code." A frontier model's natural output for visual artifacts is structured and technical (e.g. SVG, HTML, JSON). Yet users expect a clean chat window with rich, intuitive elements. Bridging the two means deciding, for every artifact Scout produces, where it lives, who can see it, and how the model stays aware of what the user is looking at.

The team's design rule became: anything the user sees, the model must be able to discover. Otherwise the agent loses the thread the moment the user asks a follow-up question.

Take, for example, visualization and company search.

Visualizations

Early on in testing, a user asked Scout for a visual market map, and the team expected it to reject the request. Instead, Scout returned SVG code that the frontend didn't have the ability to directly render. The team leaned in to figure out how to turn this into an intentional feature rather than a failure mode.

Visualizations are now a first-class primitive in a Scout conversation. The model authors the visualization inline as part of its message, and the frontend renders it in place. Both the model and the user see the same artifact in the same conversation. The team didn't have to design a chart feature, a timeline feature, or a market map feature. The pattern (meeting prep, research reports, team-build-out timelines) repeats because the agent and UX share one source of truth.

Company search

Users want Scout to search across thousands of companies at a time—far more than can stream back as a chat message. The design where Scout fired a search, rendered results in a side panel, and moved on, breaks down the moment the user asks "why are those three at the top?" The model never saw the result set, so it can't answer. Scout solves this by writing results to a shared filesystem the agent can read from on-demand, paired with tools that let the model page through the same data the user is seeing. The frontend renders rows as they arrive. The model can revisit any of them when the conversation calls for it.

Using the shared filesystem design, Scout doesn't need a custom workflow for every use case. "Not having to build a unique workflow experience for every individual use case or customers is a huge unlock," noted Austin.

When it comes to the product development implications, Seth added:

"Before Scout, we'd build for how we expected users to interact with our product. We would get feature requests, prioritize them, and make tradeoffs. Now, in this agentic model, we give users the ability to interact with the data how they want to and we observe how they use it. We discover emergent use cases every day."

Running Scout in Production on LangSmith Deployment

The team behind Scout is lean and product-focused. LangSmith Deployment manages the platform layer for durable thread execution, reliable deployments that can scale infinitely as usage increases, and observability. Austin describes it as "set and forget," freeing the team to focus on building the interface for what makes Harmonic's product distinct (global company and person data combined with your firm’s context and connections) versus the infrastructure underneath.

For iteration, the team built an internal dashboard that renders full conversation traces so engineers see exactly what users saw, including any generated artifacts. They also use the LangSmith MCP alongside their AI coding tools:

"We can pull down LangSmith traces, see how they interact with the code being executed, and make changes informed by what's actually happening. There's a loose self-improving loop: make changes, execute a graph, see the trace, pull the trace back in."

As early adopters of LangSmith Engine, the team also benefits from a more explicit self-improving loop as Engine identifies and clusters failure modes, and suggests code improvements and evals on the spot:

"Our Deep Agents traces can contain dozens or hundreds of turns, which makes reviewing and identifying patterns tedious. LangSmith Engine saves our team hours of digging by not only identifying emerging failure modes, but also proactively suggesting evals and code changes to resolve them quickly,” said Austin.

4x Retention By Switching to Deep Agents

The response from users has been extraordinary. Investors describe Scout as "incredibly helpful,” "like my heaven, and "game-changing." Even more telling is what one early-stage partner observed when comparing notes with other firms—that "everyone's secret weapon is becoming Scout," with investors recognizing Scout is becoming industry-standard when discussing must-have tools with peers.

That word-of-mouth feedback is reflected in usage metrics: week-one to week-four retention of Scout users jumped to 4x what it was on V1. And the average session duration of Scout users has increased 10x as users now conduct longer, multi-step conversations versus one-off company searches.

What's Next: A Bigger TAM

The value Scout now provides allows Harmonic to expand to emerging markets. Scout can answer any question about startups thanks to its unique startup database, which unlocks use cases beyond investing. Innovation and Corporate Development teams, talent teams, recruiters, and GTM teams selling to startups are using Scout to stay in the know. And the Harmonic team remains as excited as ever about its mission to deliver opportunities to startups.

Want to build production agents on your own proprietary data layer? Learn about Deep Agents or read the docs.

See what your agent is really doing

LangSmith, our agent engineering platform, helps developers debug every agent decision, eval changes, and deploy in one click.

Try LangSmith

Get a demo