翻訳待ち：Introducing FILE type: a native column type for multimodal data

AI サービスが一時的に利用できないため、復旧後に翻訳を補完します。

Introducing FILE type: a native column type for multimodal data | Databricks Blog Your data estate holds far more than structured tables, metrics, and transaction logs - it holds contracts, product images, call recordings, and video. AI can now turn that unstructured data into something you can query and analyze, but only if it is governed and managed alongside everything else. Today we're announcing the beta of FILE type: a new column type that stores unstructured data as a native, governed column in your tables. With FILE, your unstructured data is AI-ready: queryable, secured, and managed right alongside your structured data, instead of living in a separate system. Benefits of FILE include: Unified governance. Apply the exact same fine-grained access controls and security policies to your raw files as you do to your standard tables. Automated compliance. When you delete a row containing a FILE, the file’s binary is also deleted in object storage, making GDPR "right to be forgotten" effortless. SQL and Python support. Run standard SQL and Python UDFs directly on unstructured files just like any normal column, and build materialized views to run AI functions incrementally. High performance by design. FILE columns only store lightweight pointers rather than heavy file binaries, and only process the actual file content when a query explicitly needs it FILE is an innovation we are driving in the open: we are working with the community to build support directly into Parquet and Delta Lake, so the entire ecosystem can leverage it. That keeps your multimodal data open and portable, so you're never locked into a single vendor or model provider. What you can do with FILE type Today, your data teams want to build applications on multimodal data to drive business impact: Enterprise document assistants over contracts, policies, and research - turning paperwork into an interactive knowledge base. Visual inspection over product images - catching defects at scale instead of eyeballing every one. Call analytics over audio recordings - surfacing why customers convert or churn from conversations you already have. Video understanding over events - making hours of footage searchable and queryable. Multimodal retrieval for agents that need to cite and act on real evidence, not just text. FILE reduces friction through the entire lifecycle of multimodal data - from exploratory analysis, to model training, to serving data to agents in real-time. To demonstrate this, we will walk through one example, but you can swap in your own use case as you read along. Let's start our example with a pointed question from the CEO of an autonomous-driving company: Can we reduce the number of random stops? When our self-driving car stops for no apparent reason, passengers lose confidence in our service The data needed to address this is unstructured, dashcam videos captured from each self-driving car. Here is how FILE makes that data AI ready so we can address the CEO’s request You start by creating a column of FILEs - create a table with a FILE column that reads from the storage volume holding the dashcam videos Figure 1: Conceptual diagram demonstrating a FILE column storing videos Add a footage column typed as FILE, mapping each clip in object storage to a row The table now holds that FILE column next to structured metadata One of those dashcam clips shows the car stopping for no clear reason - an example of what we need to discover and investigate Below is example syntax for creating a table with a FILE column. To try out the complete run through of a file-processing pipeline using FILE, see the accompanying documentation which includes example notebooks. Process the dashcam videos performantly. Because the column is typed as FILE, you can work with it like any other column: reach for built-in AI functions or your own Python UDFs. Here, a UDF samples a frame from each clip into a new FILE column of images, and an object-detection model adds one more column to the metadata you already have - indicating whether the car actually has a hazard in front of it that requires a stop. Figure 2: Diagram demonstrating processed video FILEs A sampled frame lands in a new frame column, again typed as FILE We run a function directly on that FILE column to get the hazard value for each frame The stopped clip comes back with hazard = none - this the insight we are after The payoff with using FILE is that you can easily work with gigabytes of video without taking a hit on query performance. Because the FILE column holds a lightweight reference, the engine pulls the actual bytes only at the step that needs them. Contrast that with encoding the raw binary in the table, where every operation drags the large binary through the engine’s memory and hinders performance. Answer the CEO's question. Now an agent can reason over your entire multimodal data estate - joining the extracted information to the structured trip data to surface every video where the car came to a sudden stop with no hazard in front of it. Figure 3: Diagram demonstrating querying multimodal table One query joins the extracted hazard column to the structured trip data, filtering for the videos where the car came to a full stop with a clear road ahead From here, your machine learning team can easily curate training sets to improve the self-driving system The agent can now accurately answer the question quickly because everything lives in one row: the original video frame as ground truth, the embedding, extracted insights like present hazards, and the trip metadata (speed, timestamp, etc.) all sit side by side. We often see customers try to solve use cases like this by storing a URL string containing a path to the file in the table. However, those strings are governed by an entirely different system of broad, folder-level permissions. You now are forced to maintain two permission models for one dataset, and a row filter that protects a path does nothing to protect the video at the end of it. We have solved this governance challenge by integrating FILE into Unity Catalog, where FILEs are also secured by row and column-level access controls as well as attribute-based access control (ABAC), ensuring the right people have access to your data. Another problem that teams face today is coordinating the lifecycle of their unstructured data. If someone deletes a video in object storage, the table has no idea of the storage lifecycle policy and the table knows nothing about each other, so you're left with a reference that points to nothing. With FILE, the video's lifecycle travels with its row in the dataset: delete a row, and the data and its reference stay in sync - no orphaned files, no compliance gaps. Now your teams can move quickly without worrying about right-to-be-forgotten requests (e.g. GDPR) and a manual scramble to find every copy of the data. FILE is integrated with the tools you already use FILE now enables your unstructured data to benefit from the same open foundation, governance model, and AI stack you already use for structured data. Ingest effortlessly or reference in place. Connect directly to sources like SharePoint and Google Drive with point-and-click Lakeflow connectors. Or, if your data already lives in cloud storage or on-prem, FILE columns can reference it right where it is - no data movement required. Build multimodal pipelines with Spark Declarative Pipelines. Processing unstructured data incrementally matters more than ever, because every document you reprocess can mean an expensive model API call. And just like your traditional data pipelines, you should not have engineers up at 3 a.m. when one file fails. With FILE and SDP you declare the multimodal tables you want and the DAG resolves itself. Ingestion is incremental, so only new documents are processed. Retry and recovery are built in, so a transient failure does not mean starting over. Use any model to process your unstructured data, including best-in-class AI functions. Run AI_PARSE_DOCUMENT, AI_QUERY, and your own UDFs directly over the FILE column to turn raw documents, images, and video into the structured columns your agents and analysts already query. And because Databricks is built for multi-cloud environments from the ground up, FILE fits naturally across cloud providers while providing your team one unified lakehouse. Get started with FILE type FILE type is available now in Beta and we would love your feedback. Read the Beta documentation for the full list of what you can try out today and step-by-step guidance. This is just the start for FILE type and we have an exciting roadmap of features coming soon that will accelerate your AI projects, including: Experiment faster without extra cost. Training and iterating on a dataset should not mean duplicating terabytes of unstructured data. FILE lets you version and clone data without copying the underlying binaries, and reference the same bytes from many tables at once. For example, a machine learning agent can spin up an isolated sandbox on real data, and lineage traces any regression back to the exact asset that caused it. Stream datasets directly into PyTorch. Execute SQL queries against your FILE data and instantly load the results into native PyTorch Datasets, delivering GPU-ready tensors straight to your training loops without extra data prep. Feature engineering without a table rewrite. Add or backfill a derived column such as extracted text, an embedding, or a classification without rewriting the whole table, including large binaries of the source unstructured data. One copy of your unstructured data for both analytics and serving. Search and retrieve directly from the table, with vector, full-text, and point-lookup indexes on the table itself, so your retrieval layer and your source of truth are the same lakehouse. If you want to learn more about how we are optimizing AI workloads in Databricks, reach out to your Databricks account team. Get the latest posts in your inbox Subscribe to our blog and get the latest posts delivered to your inbox. Sign up View all blogs