AI 服務暫時不可用，以下為來源正文，待恢復後補全翻譯。
← Blog When vectors became an implementation detail How Pinecone built integrated embedding into the Documents API Silas Smith Oct 22, 2026 Engineering Share: Nobody searches for a vector. People describe what they're looking for and expect to find it. Yet for most of Pinecone's history, you had to turn both your data and every search into vectors yourself, with your own embedding pipeline, before you could search. We first moved embedding inside Pinecone with the Records API in 2024, and now we’ve added it to the Documents API alongside full-text search. You just send Pinecone text, and it runs the embedding pipeline for you on every write and every search. This is the story of how we built it. The Documents API: native vectors, full-text search, and now integrated embedding, with more on the way. When I joined Pinecone in early 2024, what stood out was how much work our users had to do outside of Pinecone before they could use us. Good search depends on preparing your data ahead of time and orchestrating the queries your application receives. Over the last 2.5 years I’ve kept coming back to the same problem: how to get Pinecone to take on more of that work, so our users can spend more time building their application and less time building retrieval pipelines. Vectors are a big part of that preparation, and they always seemed to me like an implementation detail. They're essential to how the database finds what you're looking for, but not something you should have to build yourself. At its heart, this is an API story, so it makes sense to start at the beginning. For us, that’s the Vectors API. Bring your own vectors (circa 2021) Pinecone started as a pure vector database, and while we may now call the original API the Vectors API, we used to just call it “the API”. Not surprisingly, it speaks in terms of vectors: it either takes in vectors and stores them, or takes a query vector to search the vector space with. That means wiring a third-party embedding provider into your data pipeline, with all of the usual extra work that comes with it: authentication and key rotation, rate limits and retry with backoff, billing, and monitoring. Most applications are also split in two parts, with one path for loading data and another for queries, so you end up having to build the integration twice, including balancing the need for low-latency queries with high-throughput data loading. The Vectors API also asks you to conform your data to its request shape: an ID, a vector, and some metadata in a prescribed format. And you're faced with a decision about where to keep the source text behind each vector: either you need to keep it in a separate data store, or you put it in a metadata field in Pinecone, a tradeoff between complexity in your application and cost and performance in your index. A separate data store means that every search result requires a reverse lookup to find its corresponding text. Putting the text in a metadata field is simpler, but metadata is meant for small values you filter on, like categories. Pinecone indexes it into roaring bitmaps, so every block of text you store there adds to your storage and write costs and uses up part of the metadata limit per record. And it doesn’t make the words in it any more searchable. With the Vectors API, you’re responsible for the embedding integration and often have to build it twice (once for writes and again for queries) and need to decide where to store the related text, with trade-offs involved. Laying the groundwork (2024–2025) It quickly became clear that every one of our users was building the same embedding pipeline (often twice). But before we could take on any of that work, two fundamental components had to be in place. The first was an inference stack. We started hosting models ourselves in 2024, adding standalone embed and rerank endpoints. This meant you could use one API key and one provider and begin to simplify some parts of your application. You still had to call the model yourself before every write and every search, but because the models now ran next to the database and under the same API key, that opened the door to combining them directly with the database workloads they supported. The second was a more flexible storage architecture for the database itself. In early 2025 we shipped the slab architecture: a new log-structured architecture for serverless indexes*. With slab, writes land in an in-memory buffer and flush to immutable sets of files we call “slabs”, and background compaction automatically merges small slabs into larger ones when warranted. Each slab is self-describing: it carries its own data, its own index, and a record of what kind of index it holds. Importantly, that made index selection composable for different kinds of indexes beyond just vectors, and full-text search would later become one of them. 🤯 Side note: We invested a lot of time in 2025 in rolling out the slab architecture to our entire customer base, so that we could migrate live indexes from our older clustering architecture to the new slab architecture. Not only could customers keep reading, but they could also keep writing throughout the migration process. Akin to not just building an airplane while flying, but also transferring the passengers from the old plane to the new one in mid-air! To learn more about our architecture, see how Pinecone works and our guide to the slab architecture. The Records API: proving the pattern (2024) While we were rebuilding the database architecture, we took our first crack at integrated embedding via the Records API, which launched in late 2024. We wrote about it shortly after, including how we sped it up by separating latency-sensitive query embedding traffic from slower batch upsert workloads. With the Records API, you specify a model when creating an index, and then upsert text records and search with text. Pinecone embeds the text on the way in and the query on the way out, using the model configured on the index. This proved the basic request pattern that we wanted. Users just bring text, the embedding happens inside Pinecone, and users never have to see a vector. 👉 Fun fact: we expected the Records API to grow into a “documents” API, so we actually named the new Kubernetes service behind it ‘docs-api’. But ultimately the Records API was premature: it wasn't a shape we could keep building on. Under the hood, it wraps our Vectors API, and the text still lives in a metadata field, with the same tradeoff costs as before. Each index was tied to one model and one text field, and didn’t give you the option to avoid storing the text. Because it was built directly on the Vectors API, the Records API had no room to evolve to support other kinds of search, like native full-text. Full-text search changes the shape (2025–2026) We’d known for a long time that we needed full-text search alongside vector search. It wins on exact string matching, like product codes and error messages, and combining the two is powerful. But full-text search needs the text itself tokenized and indexed by a native text search engine. With the slab architecture we had theoretically made room for a new kind of index, but it was unproven. Our APIs at the time also had no place for it. Making full-text search a reality would mean addressing both. So in late November 2025 we brought the engineering team together in NYC, locked ourselves in a conference room, and didn’t emerge until we had a whiteboard nobody dared to erase and a clear picture of what we needed to build. It came down to three significant new pieces: Native full-text search to be supported in our slab architecture A flexible schema language to drive indexing configuration and query orchestration A true document API and flexible search API we can continue to build on We've written about how full-text search works, the architecture behind it, and its general availability. In short, document indexes support a user-defined schema of typed fields, which at the first release were: string fields for full-text search, dense and sparse vector fields for native vector search, and metadata for filtering. And the search API lets you choose how documents are scored: by keyword relevance on text fields, or by semantics on a vector field. Besides improving retrieval quality in keyword-heavy use cases, full-text search fixes the text-as-metadata storage problem. Metadata indexing is built to filter on exact values, and raw text almost never benefits from exact value-matching. A full-text field is stored differently: Pinecone keeps the raw text once in the slab and builds a Tantivy inverted index next to it, which maps each term to the documents and positions where it appears. The index scores search results with BM25, and it also supports powerful text-match filters like $match_phrase, so any query, including a vector search, can require that a document contain a specific word or phrase. The Documents API fronts an entirely new index mechanism built from the top down, driven by a document schema, with native full-text search and vector search side by side in the same slab architecture. It also inverts the request shape: the Vectors API asks you to conform your data to its shape, while the Documents API is designed to conform to yours. You write documents with your own field names, and the schema says how each field should be indexed. The first release of document indexes and the Documents API focused on full-text search and native vectors. But integrated embedding was always part of the plan. The Documents API was designed up front to accommodate it and further evolution to support things like hybrid search, integrated reranking, multiple vector fields, and more. Integrated embedding as a schema field (2026) You can now configure both integrated embedding and full-text search on a string field: pc.indexes.create( name="space-missions", schema={ "fields": { "body": { "type": "string", "full_text_search": {"stemming": True, "stop_words": True}, "embed": {"model": "llama-text-embed-v2"}, } } } ) Create a document index with one string field that has both full-text search and integrated embedding. When you write a document, you send plain text. Pinecone embeds it with the field's model (specified by the schema), stores the vector next to the text, and indexes the text for full-text search—all in the same write. If embedding fails, nothing in the request is written. When you search, you send the query as text, and Pinecone embeds it with the same model. The goal is that loading data should feel less like building a data pipeline and more like configuring one. With integrated embedding, one string field holds the text, its full-text index, and its vector. Both kinds of search can run against it. Integrated embedding also makes search faster. When you embed the query yourself, every search takes two round trips: one to an embedding API and another to Pinecone with the vector. With integrated embedding it's one call. In our testing, a search with text in one call was about 3.5 times faster at the median than calling our standalone embedding API and then searching with the vector. Most of the difference comes from the extra network round trip that the single call avoids. Here’s how the Documents API compares to the APIs that came before it: Documents APIVectors APIRecords API You writeJSON documents with your own fieldsID, vector, metadataID, text field and metadata, in JSONL format Who embedsPinecone, per field in the schemaYou, outside PineconePinecone, one model per index Source text livesIn its own string fieldSeparate store or metadataMetadata Search byFull-text, vector, or text (embedded)VectorText (embedded) Configured bySchemaDimension and metricModel and text field Two kinds of search Here’s an example index with a handful of short descriptions of space missions in a single body field, with both s [truncated for AI cost control]