Legal AI, not a coding agent with scaffolding

Blog/Research/Real legal AI

Blog/Research/Real legal AI

Legal AI, not a coding agent with scaffolding.

Alan Yahya·July 15, 2026·12 min read

The majority of legal AI use is transacted through agents built for coding. Let’s focus on what a system built for a legal purpose might tangibly achieve.

Generally, legal AI should find the strongest supportable arguments, identify the correct evidence, and make that evidence easy to verify.

An agent workspace is the correct interface. The user sees an audit trail including the tools and sources used by the model. They are able to control the context by creating side chats, and define specific sources and skills to manipulate model behaviour.

Agent grounding must happen at the level of individual claims and edits. It is simply not good enough to retrieve an external document and cite it as a whole. This level of granularity depends on the argument- which may call for an individual sentence, a paragraph, or a set of pages. The hierarchy, status, and intended purpose of each source should be explicit. That information can be stored via an ontology or similar graph network. For example, legal authority vs client facts, current document language vs previous drafting language.

When the AI suggests an edit, the lawyer should be able to see why it made that change. That includes the original instruction, the clauses it reviewed, the facts it considered, the legal sources it relied on, and any warnings or uncertainties that can be calculated deterministically and packaged neatly for the user.

AI should not make changes automatically. It should show the proposed edit as a redline (or tracked change!) that can be processed individually. This workflow goes some way to deterrings users from blindly accepting changes. Needless to say, changes are logged and revocable at any time using an underlying version control system. Letting a probabilistic process edit a document without a version control system would simply be reckless.

Skills, memory and compaction should preserve access to evidence. They should not replace evidence with summaries. That is not good enough in a legal use-case. Rather, it should leave behind a lookup artifact that allows the exact original content to be viewed for factual information.

AI, even evidence grounded, must never subvert legal judgment. Sources may be outdated, narrow, incomplete, or limited to a particular jurisdiction. The system should make these limitations more visible, not less, and enable a lawyer to make fully informed decisions with up-to-date information.

Coding agents only enforce mechanical correctness, with interpretation as a guideline. Lawyers are being sanctioned because evidence grounding is being treated as an optional guidance layer, rather than a core product or workflow requirement. And unfortunately, that is the current story for legal AI tools.

Deleting a clause: did the patch match, and were its dependencies checked?

Take the instruction: delete Section 12.7 because its obligation appears to be duplicated elsewhere. There are two different ways this can go wrong. The text may have changed since the edit was prepared, or another clause may still refer to Section 12.7.

Codex's native patch path addresses the first failure directly. It looks for the old lines in the current file:

codex/codex-rs/apply-patch/src/lib.rs:765

Code example+−

let mut pattern: &[String] = &chunk.old_lines;

let mut found = seek_sequence::seek_sequence(original_lines, pattern, line_index, chunk.is_end_of_file);

If the expected text is no longer present, the patch fails rather than applying against a stale location:

codex/codex-rs/apply-patch/src/lib.rs:787

Code example+−

if let Some(start_idx) = found { replacements.push((start_idx, pattern.len(), new_slice.to_vec())); line_index = start_idx + pattern.len(); } else { return Err(ApplyPatchError::ComputeReplacements(format!( "Failed to find expected lines in {}:\n{}", path, chunk.old_lines.join("\n"), ))); }

That is a strong textual invariant, but it says nothing about legal-document dependencies. Codex can search for cross-references, but the native patch application does not require that search to have happened.

Lexifina records whether a structured cross-reference lookup for the section completed successfully:

Code example+−

def _completed_cross_reference_terms(ctx: ToolUseContext) -> set[str]: terms: set[str] = set() for row in getattr(ctx.document_state, "tool_call_log", None) or []: if str(row.get("tool_name") or "") != "get_cross_references": continue if str(row.get("status") or "").lower() != "ok" or row.get("is_error"): continue raw_input = row.get("raw_input") if not isinstance(raw_input, dict): raw_input = row.get("input_preview") if not isinstance(raw_input, dict): continue term = _normalized_reference_term(raw_input.get("term")) if term: terms.add(term) return terms

For a whole-section deletion, absence of that recorded review becomes an edit-level warning:

Code example+−

def _cross_reference_advisories( *, ctx: ToolUseContext, original_text: str, new_text: str, ) -> List[Dict[str, Any]]: if not original_text.strip() or new_text.strip(): return []

label = _section_reference_label(original_text) if not label or _normalized_reference_term(label) in _completed_cross_reference_terms(ctx): return []

return [ { "code": "cross_reference_review_not_recorded", "severity": "warning", "message": ( f"No completed structured cross-reference review for {label} " "was recorded before this deletion. The deletion remains staged " "for your decision." ), "source": "deterministic_legal_review", "details": {"section_label": label}, } ]

Each system protects against different failures. Codex prevents a stale hunk from being applied to text it no longer matches. Lexifina carries the absence of a structural-dependency check into the review object.

Compaction: can the model recover the exact tool output?

Suppose a research tool returned a long opinion extract, the model used two paragraphs from it, and the conversation was compacted before drafting finished. A narrative summary that says “the case supports the argument” is not the evidence. The relevant question is whether the active agent can get the exact tool output back.

In Codex's local compaction path, the replacement history is built from retained user messages and a generated summary:

codex/codex-rs/core/src/compact.rs:323

Code example+−

let history_snapshot = sess.clone_history().await; let history_items = history_snapshot.raw_items(); let summary_suffix = get_last_assistant_message_from_turn(history_items).unwrap_or_default(); let summary_text = format!("{SUMMARY_PREFIX}\n{summary_suffix}"); let user_messages = collect_user_messages(history_items);

let mut new_history = build_compacted_history(Vec::new(), &user_messages, &summary_text);

That is the model's active replacement history. It does not add a per-tool-result lookup to the summary. This does not mean Codex deletes its durable rollout, and an agent may be able to reread a source file or rerun a tool.

Lexifina's compaction marker leaves the model an executable address for a cleared result:

Code example+−

Marker template — note the {tool_use_id} placeholder is filled per

tool result so the model can call read_tool_result with the right

argument without searching for the id. Format string, not f-string,

so callers that import this module can introspect it.

TOOL_RESULT_CLEARED_MARKER_TEMPLATE = ( "[Tool result cleared during compaction — full content available " 'via read_tool_result(tool_use_id="{tool_use_id}") if needed.]' )

read_tool_result resolves that id to the persisted full output and returns the requested range without asking another model to reconstruct it:

Code example+−

path = Path(RESULT_PERSIST_DIR) / process_id / f"{input_data.tool_use_id}.txt" if not path.exists(): return ToolResult( is_error=True, assistant_text=( f"No persisted result found for tool_use_id=" f"{input_data.tool_use_id!r}. Either the id is wrong " "or the result was small enough to ship inline (no " "truncation notice = no persisted file). Check the " "tool result you copied the id from." ), summary="not found", )

Code example+−

bounded_len = min(int(input_data.length), MAX_FETCH_CHARS) try: with path.open("r", encoding="utf-8") as f: f.seek(int(input_data.offset)) chunk = f.read(bounded_len)

This recovery is limited to the active run: the model-facing file is temporary and deleted when the run ends:

Code example+−

Result-size cap: oversized rendered text is persisted to

disk and the model-facing block becomes a head/tail preview

with the tool_use_id. Skipped when the tool itself opts out

via max_result_size_chars=math.inf (read_tool_result, where

persisting a fetched chunk would create a circular loop) or

when the result is an error (errors are usually small and

already structured for the model — persisting them just adds

a layer of indirection). The persisted file is cleaned up

at end-of-run by routers/agentic.py.

In short, compaction can replace bulk text without forcing the next model to trust a paraphrase; the tool_use_id remains a route back to the exact persisted tool text. Codex's local replacement history retains a summary but does not itself expose an equivalent call-id lookup.

A redline shows what changed. Can it prove which drafting decision changed it?

Suppose version 3 capped indemnity at the fees paid under the agreement. The team then reverted to an earlier draft and created version 6, where the cap is gone. The lawyer's question is not merely “what text differs?” It is: did the current drafting branch remove the cap, which accepted edit did it, what instruction produced that edit, and is the answer based on the saved change record or a reconstruction made later?

Codex maintains an exact net diff for file changes made by committed patches during the current request:

codex/codex-rs/core/src/turn_diff_tracker.rs:48

Code example+−

/// Tracks the net text diff for the current turn from committed apply_patch /// mutations, without rereading the workspace filesystem. pub struct TurnDiffTracker { valid: bool, display_roots_by_environment: HashMap, baseline_by_path: HashMap, current_by_path: HashMap, origin_by_current_path: HashMap, next_revision: u64, rendered_diffs: HashMap>, unified_diff: Option,

Only exact patch deltas are admitted to that tracker. An inexact delta invalidates the result rather than allowing a potentially misleading diff to survive:

codex/codex-rs/core/src/turn_diff_tracker.rs:93

Code example+−

pub fn track_delta(&mut self, environment_id: &str, delta: &AppliedPatchDelta) { if !self.valid { return; }

if !delta.is_exact() { self.invalidate(); return; }

for change in delta.changes() { self.apply_change(environment_id, change); } self.refresh_unified_diff(); }

The event emitted to the client contains that unified diff:

codex/codex-rs/core/src/session/turn.rs:2495

Code example+−

if should_emit_turn_diff { let unified_diff = { let tracker = turn_diff_tracker.lock().await; tracker.get_unified_diff() }; if let Some(unified_diff) = unified_diff { let msg = EventMsg::TurnDiff(TurnDiffEvent { unified_diff }); sess.clone().send_event(&turn_context, msg).await; } }

The event's entire payload is one textual field:

codex/codex-rs/protocol/src/protocol.rs:3669

Code example+−

#[derive(Debug, Clone, Deserialize, Serialize, JsonSchema, TS)] pub struct TurnDiffEvent { pub unified_diff: String, }

That is a strong answer to “what did this Codex request change?” It does not make the diff event itself a saved-document lineage record: the hunk has no parent version, accepted edit id, triggering instruction, reviewer, application time, or confidence field. Codex can retain surrounding rollout events, and a version-control system can add commit history. The narrower point is that those relationships are not invariants of this native diff object.

Lexifi

[truncated for AI cost control]