AI 服务暂时不可用，以下为来源正文，待恢复后补全翻译。
A configurable, instruction-driven detector that runs on any large language model (LLM) managed on Amazon Bedrock, evaluated on five public PII corpora across nine LLM-based detectors, including the OpenAI PrivacyFilter. Fine-tuning a model on real-world text creates a personally identifiable information (PII) detection problem. Training corpora are full of PII: names, home addresses, email and phone numbers, national-ID and social-security numbers, bank accounts, dates of birth. A model trained on uncleaned text can memorize that data and later reproduce it, leaking a real person’s details through a prompt that was never meant to surface them. In this post, we describe a configurable, model-agnostic detector built on large language models (LLMs), walk through its implementation, benchmark it against an off-the-shelf tool, and show how to run it on your own data. Sample code: The detector described in this post ships as the pii-detector package, available in the sample-llm-pii-detection repository. Every code snippet that follows is drawn from that package, and the Running the detector end to end section walks through installing and running it on your own data. PII rarely sits in a tidy form field. It hides in customer-support transcripts, HR records, chat logs, and the long free-text columns that make up the custom datasets teams fine-tune on. It arrives in messy, multilingual formats that no fixed schema anticipated. The usual tools are bi-directional token-classification models: transformer taggers that label each token with a PII type fixed at training time. A domain-specific identifier like an employee ID or crypto-wallet address is exactly what a custom fine-tuning corpus introduces, and it falls outside that frozen schema. Adding it means relabeling and retraining. And they are locked to one model and one deployment. Large language models reframe the problem. An LLM reads its instructions at inference time, so the entities to detect, the output format, and the deployment backend all become configuration rather than code. One detector can target a new entity type by editing a prompt instead of retraining, run on a managed API or inside your own virtual private cloud (VPC), and reason about context across eight languages without a translation step. The rest of this post describes such a detector, walks through the engineering behind it, and shows how it measures up against existing tools. Solution overview The detector treats the language model as a configurable, swappable component. You wrap the input text in instructions that define the PII entities to detect and the expected output. The model then returns a structured list of detected entities. Two design choices make it model-agnostic: Instruction-driven detection: The detection logic lives entirely in the instructions and a thin parsing layer. That makes it independent of any single model’s idiosyncrasies. Configurable backend: The model is reached through a uniform inference interface, the Inferencer. The package ships an adapter for Amazon Bedrock (managed, for example Mistral or OSS-GPT). The same interface accepts a custom adapter for an open model, such as OSS-GPT 20B served on your own infrastructure with a GPU. That covers secure or air-gapped environments that cannot reach Amazon Bedrock. Any object that takes a list of messages and returns the assistant’s text satisfies the interface, so the detector is agnostic to which backend it holds. Customization comes from two independent components. The first is the model, which sets accuracy, latency, and cost: you choose a frontier Amazon Bedrock model or a small open model on a single GPU. The second is the entity set, which defines what counts as PII. To extend it, you add a domain-specific identifier or drop one that you don’t need. Changing the entity set is a one-line edit to the instructions, with no retraining and no redeployment. The LLM’s job is narrow and well-defined. It reads the text, identifies all PII spans, and labels each with an entity type from the schema. It returns those spans as structured JSON, and a post-processing step computes exact character offsets and removes duplicates. To place the approach in context, we evaluate it span-for-span alongside eight other LLM-based detectors, including the OpenAI PrivacyFilter. All are scored on a common ground truth. Technical implementation The detector is built from four parts. A prompt defines the schema, a backend runs the model, a parsing-and-offset layer turns the response into located spans, and a thin call sequence ties them together. This section walks through each part in the order a request flows through the system, pointing to the module in the package repository that implements it. PII schema and detection prompt The schema lives in a single system-prompt template, the heart of the detector: fifteen entity categories each with a one-line definition, a do-not-flag list, optional few-shot examples, and the input text. Because the schema is text, adding or removing a category is a one-line edit. The model is instructed to respond with a JSON list, one object per detected entity carrying the entity type and the exact text value found. It does not return character offsets, which an LLM cannot produce reliably. Those are recovered in post-processing: [{"pii_entity_type": "FULL_ADDRESSES", "pii_entity_value": "82 Oak Street"}, {"pii_entity_type": "CONTACT_INFO", "pii_entity_value": "[email protected]"}] The full prompt is in pii_detector/templates.py, and the end-to-end walkthrough that follows runs it as-is against a sample string. LLM backend integration Because detection lives in the prompt, the backend is a free choice. In our provided implementation, the detector talks to a small interface, the Inferencer: messages in, text out. The same detector therefore runs against a managed model on Amazon Bedrock or an open model you host yourself on Amazon Elastic Compute Cloud (Amazon EC2). The package ships the Amazon Bedrock adapter (pii_detector/bedrock_inferencer.py), a thin wrapper over the Converse API. The walkthrough that follows runs that path end to end. Post-processing The model’s raw text becomes a clean list of located spans in three steps, all in pii_detector/detector.py: JSON parsing: The text response is converted into a list of dictionaries, each item corresponding to a detected PII (or an empty list if the record contains none). Offset calculation: Because the model returns values and not positions, each value is located in the source text with a regular expression. Hallucinated-label recovery: LLMs routinely emit near-miss labels (DATE for DATES, EMAIL for CONTACT_INFO), so each emitted label is re-homed onto the prompt’s own vocabulary by morphology and a curated alias table. A label that no tier can map is marked UNK rather than force-fitted, so genuine hallucinations stay visible. Running the detector end to end This section walks through running the detector on your own data, from prerequisites to cleanup. Every step uses the pii-detector package referenced at the top of this post. Prerequisites To follow along, you must have the following prerequisites. Python: Python 3.11 or later. AWS account with Amazon Bedrock model access: An AWS account whose credentials can call the Amazon Bedrock Converse API. You also need model access enabled in the Amazon Bedrock console for the model that you choose, for example a Mistral or OSS-GPT model. For model availability by AWS Region, see Supported models by AWS Region in Amazon Bedrock. The detector resolves credentials through the standard AWS credential chain, so set AWS_PROFILE (or an IAM role or SSO profile) and AWS_REGION in your environment. Python dependency: Boto3, the only runtime dependency, installed into a virtual environment (see step 1). The following steps assume you have cloned the pii-detector repository and are working from its root directory. Step 1: Install the package and its dependency Create a virtual environment and install boto3. The package runs from the repository root, so set PYTHONPATH to make the pii_detector module resolve. cd pii-detector python -m venv .venv && source .venv/bin/activate pip install boto3 export PYTHONPATH=. # so import pii_detector resolves from the repo root Step 2: Configure AWS credentials for Amazon Bedrock Point Boto3 at an account with Amazon Bedrock access and select the Region where you enabled model access. export AWS_PROFILE=my-bedrock-profile export AWS_REGION=us-east-1 If you aren’t using a named profile, Boto3 also supports AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY, but we recommend an AWS Identity and Access Management (IAM) role or SSO profile instead of long-lived static keys. Step 3: Run the bundled example The repository ships a runnable example (examples/detect.py) that detects PII in a sample string. Run it as a module from the repository root. It fails fast with actionable guidance if credentials or model access are missing. python -m examples.detect Step 4: Call the detector on your own text Construct an Amazon Bedrock inferencer with any Amazon Bedrock Converse model id, wrap it in a PiiDetector, and call the detector on a string. It returns the list of located spans, each with its exact character offsets, ready to feed a downstream redaction step. There are no servers to manage, because Amazon Bedrock is fully managed. from pii_detector import BedrockInferencer, PiiDetector inferencer = BedrockInferencer( model_id="openai.gpt-oss-20b-1:0", region="us-east-1", ) detector = PiiDetector(inferencer) spans = detector("Email [email protected] or call Jane at 555-0142.") # -> [{'pii_entity_type': 'CONTACT_INFO', 'pii_entity_value': '[email protected]', # 'start': 6, 'end': 21}, ...] The model_id is any Amazon Bedrock Converse model id or inference-profile id, for example amazon.nova-lite-v1:0 or mistral.mistral-large-3-675b-instruct. Switching models is a one-line change. The detector and the call site stay identical. Step 5: Clean up Amazon Bedrock is serverless, so there’s no infrastructure to tear down and you pay only for the tokens you use. To clean up, deactivate the virtual environment (deactivate) and, if you no longer need it, disable the model access you enabled in the Amazon Bedrock console. If you supply your own self-hosted backend instead of Amazon Bedrock, remember to shut down that host yourself, since the detector doesn’t manage backend infrastructure. Benchmarking Evaluation uses five public PII corpora from Hugging Face, each carrying ground-truth spans, sampling roughly 10,000 rows per dataset. Together they cover 49,365 records and 222,114 ground-truth core spans across eight languages (de, en, es, fr, hi, it, nl, te). Their domains run from multilingual synthetic profiles to English HR and customer-service documents, which makes the aggregate a fair stress test. # Dataset Records Core GT Notes 1 ai4privacy_500k 9,947 23,822 Multilingual. Adds sex/gender, organization 2 ai4privacy 9,936 70,720 6 langs. Names / addresses / email / phone 3 gretel 9,991 41,967 English. HR / financial / customer-service docs 4 isotonic 9,498 21,674 15+ extra domain categories 5 nemotron 9,993 63,931 US/UK. 30+ raw entity categories . Total 49,365 222,114 . A predicted span is matched to ground truth by exact (start, end, label) overlap (IoU = 1.0) and scored with Precision, Recall, and F1. Comparing detectors across these datasets is harder than it looks, because labels do not line up. Each detector and each dataset uses its own vocabulary: PRIVATE_NAMES compared to NAME, street_address compared to street. To make the comparison fair, every raw label, from detector output and dataset ground truth alike, is mapped onto a single canonical taxonomy of twelve common entities. Each detector is then scored only on the intersection of the label scopes it and the dataset both declare. T [truncated for AI cost control]