翻訳待ち：Deepgram deepens Amazon SageMaker AI observability with Enhanced Metrics

AI サービスが一時的に利用できないため、復旧後に翻訳を補完します。

Self-hosted speech AI has historically carried an observability trade-off. The service can tell you an endpoint is up and how many requests it served. The questions that actually drive capacity planning and cost management stay locked inside the vendor’s container: what you are billed for, which features your traffic uses, and what the inference engine is doing on each GPU. If you run Deepgram’s speech-to-text (STT) and text-to-speech (TTS) models on SageMaker AI, audio and transcripts stay inside your own AWS account. This can help support your data residency and compliance efforts without giving up a managed control plane for deployment, scaling, and monitoring. Your specific obligations depend on your own controls and assessments, so consult your compliance team and review the AWS shared responsibility model. Deepgram is closing the gap on billing, feature usage, and engine behavior with the following two innovations, available today on Deepgram SageMaker AI deployments. Deepgram Enhanced Metrics: Usage and billing metrics that the Deepgram container publishes directly into your Amazon CloudWatch account, with no agent, no sidecar, and no additional IAM permissions. These are the same consumed-unit values that drive AWS Marketplace metered billing, so you can reconcile your AWS bill against actual traffic down to the model and transport. Prometheus and OpenTelemetry support: Engine-level Prometheus metrics scraped straight from the Deepgram container, and per-GPU accelerator and host metrics. Both are collected through SageMaker AI detailed observability and queryable with PromQL from CloudWatch, Grafana, or any Prometheus-compatible tool. In this post, we walk through both capabilities: how they work, what they show you that you couldn’t see before, and how to start using them on a Deepgram SageMaker AI endpoint. Deepgram on Amazon SageMaker AI Deepgram’s speech models are available as model packages on AWS Marketplace and deploy as SageMaker AI real-time endpoints in your account. With SageMaker AI, you get baseline observability on your endpoints automatically: invocation metrics such as ConcurrentRequestsPerModel and FirstChunkLatency in CloudWatch, container logs in Amazon CloudWatch Logs, and alarm-driven automatic scaling. AWS Marketplace model packages run with network isolation. The container cannot make outbound connections, which is why security-conscious customers choose this deployment model. The same isolation, however, is what normally makes vendor-level telemetry hard. The container can’t push metrics to an external collector, and you can’t see inside the container. Both capabilities in this post work within that constraint: neither requires the container to open a network path, and both land the data in your own CloudWatch account. A note on terminology: SageMaker AI has its own endpoint feature called enhanced metrics (EnableEnhancedMetrics in MetricsConfig), which adds per-instance and per-GPU utilization dimensions to the standard endpoint metrics. The SageMaker AI feature is distinct from and complementary to Deepgram Enhanced Metrics, the Deepgram-published billing and usage metrics described in the following sections. This post covers the Deepgram capability. For the SageMaker AI capability, see Deepgram’s SageMaker observability guide. Innovation 1: Billing and usage transparency with Deepgram Enhanced Metrics Deepgram Enhanced Metrics answer two questions no standard endpoint metric can: exactly what you’re being billed for, and how your traffic actually uses Deepgram’s features. How it works: Metrics that ride the logging path The Deepgram container writes CloudWatch Embedded Metric Format (EMF) records to container stdout. SageMaker AI already forwards container output to the endpoint’s CloudWatch log group, and CloudWatch Logs extracts EMF records into metrics automatically. The EMF path gives you the following benefits. No agent, sidecar, or collector to deploy. No additional IAM permissions beyond what the endpoint already has for logging. Works under AWS Marketplace network isolation, because metrics travel over the existing SageMaker-to-CloudWatch logging path rather than any outbound network connection. Classic CloudWatch metrics: They appear in aws cloudwatch list-metrics and work with get-metric-statistics, dashboards, alarms, and metric math, with nothing to enable on the endpoint configuration. All dimensions are low-cardinality and contain no personally identifiable information (PII): no transcripts, TTS input, or per-request identifiers. Deepgram Enhanced Metrics aggregate across the Deepgram endpoints in your AWS account and Region. The dimensions cover category, model, and transport, but not endpoint name or instance ID, so you cannot filter this stream to a single endpoint or instance. For per-endpoint, per-instance, or per-GPU breakdowns, use the Prometheus and OpenTelemetry metrics in the next section. Reconcile your AWS Marketplace bill: The Deepgram/SageMakerInference namespace The billing namespace emits one record per completed request, covering each streaming session, pre-recorded request, and TTS request. The ConsumedUnits metric carries the same billable-unit values that drive AWS Marketplace metered billing. The following table lists the metrics this namespace publishes. Metric Unit Description ConsumedUnits Count Billable inference units for the request. Sum over a period is the total billed volume. AudioDurationSeconds Seconds Duration of audio processed (speech-to-text). CharCount Count Characters synthesized (text-to-speech). Dimensions are published at three granularities ([Category], [Category, Model], and [Category, Model, Transport]), so you can answer what did streaming STT cost this month and how much of that was nova-3 with the same namespace. For example, total consumed units per hour for streaming speech-to-text: aws cloudwatch get-metric-statistics \ --namespace Deepgram/SageMakerInference \ --metric-name ConsumedUnits \ --dimensions Name=Category,Value=stt_streaming \ --start-time $(date -u -d '24 hours ago' +%Y-%m-%dT%H:%M:%S) \ --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \ --period 3600 \ --statistics Sum SampleCount \ --region YOUR_AWS_REGION SampleCount is the number of billed requests. Because these are ordinary CloudWatch metrics, you can build a finance-facing billing dashboard or set a budget alarm on consumed units. Understand how your endpoint is used: The Deepgram/SelfHosted namespace A second stream, emitted by the Deepgram API server itself, breaks raw usage down by method, model tier, and enabled feature, independent of billing. It answers product-level questions: how much traffic is streaming versus pre-recorded, which model tiers process the most audio, and which features (diarization, smart formatting, redaction, keyterm prompting) your applications actually enable. The following table lists the metrics in this namespace and the dimensions each one carries. Metric Dimensions What it tells you AudioMs, Requests Deployment, Method Volume by streaming compared to pre-recorded TierAudioMs Deployment, Tier Audio processed per model tier (for example, nova-3, flux) FeatureAudioMs, FeatureTokens Deployment, Feature Utilization per enabled feature (for example, diarize, smart_format, redact) TtsCharacters, Tokens, VoiceAgentMs Deployment, Method TTS, intelligence-feature, and voice-agent volume For example, how much audio ran with diarization enabled in the last 24 hours: aws cloudwatch get-metric-statistics \ --namespace Deepgram/SelfHosted \ --metric-name FeatureAudioMs \ --dimensions Name=Deployment,Value=sagemaker Name=Feature,Value=diarize \ --start-time $(date -u -d '24 hours ago' +%Y-%m-%dT%H:%M:%S) \ --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \ --period 3600 \ --statistics Sum \ --region YOUR_AWS_REGION The usage stream is on by default and can be disabled with a single environment variable override in the endpoint configuration (DEEPGRAM_API_01: emf.enabled=false). The billing stream cannot be disabled. It is part of the metering pipeline. Innovation 2: Engine-level and per-GPU visibility with Prometheus and OpenTelemetry Deepgram containers serve a Prometheus metrics endpoint, and SageMaker AI detailed observability runs an AWS managed OpenTelemetry Collector on each instance backing the endpoint. The collector scrapes the container and exports the results to CloudWatch, and it exports per-GPU and host-level metrics. With detailed observability enabled, three metric sources publish to the CloudWatch OTel-compatible metric store. The following table describes each source and why it matters. Source Example metrics Why it matters GPU (DCGM exporter) DCGM_FI_DEV_GPU_UTIL, DCGM_FI_DEV_FB_USED Per-GPU series. On multi-GPU instances, each GPU reports separately, so a saturated device can’t hide behind a summed or averaged utilization number. Host (node exporter) node_cpu_seconds_total, node_memory_MemTotal_bytes Standard Prometheus node-exporter metrics for each instance. Deepgram engine engine_active_requests{kind="stream"}, engine_estimated_stream_capacity The collector scrapes the Deepgram container’s Prometheus endpoint directly. These are the same API and Engine metrics Deepgram’s self-hosted customers use for capacity planning, now available on SageMaker AI without running any collector yourself. See Deepgram’s Metrics Guide for the complete reference. engine_estimated_stream_capacity is the Deepgram engine’s own estimate of how many concurrent streams the instance can sustain. Comparing it against engine_active_requests gives you an engine-reported headroom signal for scaling decisions. Every series carries SageMaker resource labels, including aws.sagemaker.endpoint.name, the variant name, and the instance ID. You can filter to a single endpoint, isolate one instance in a scaled-out fleet, or compare GPUs within an instance. Because the collector runs on the host, outside the model container, this works under AWS Marketplace network isolation. Enabling it Detailed observability is on by default for newly created endpoints, publishing every 60 seconds. To set it explicitly, for example on an endpoint created before the feature launched, or to change the publish frequency, use MetricsConfig on the endpoint configuration: aws sagemaker create-endpoint-config \ --endpoint-config-name YOUR_CONFIG_NAME \ --production-variants file://production-variants.json \ --metrics-config '{"EnableDetailedObservability": true, "MetricPublishFrequencyInSeconds": 60}' \ --region YOUR_AWS_REGION For an endpoint already serving traffic, create a new endpoint configuration and run update-endpoint. The update is a blue/green deployment, and the endpoint stays in service. Query with PromQL from CloudWatch, Grafana, or anything Prometheus-compatible The metrics land in the CloudWatch OTel metric store and are queried with PromQL. In the CloudWatch console’s PromQL editor, filter per-GPU utilization to one endpoint: DCGM_FI_DEV_GPU_UTIL{"aws.sagemaker.endpoint.name"="YOUR_ENDPOINT_NAME"} CloudWatch also exposes a standard Prometheus-compatible HTTP API for these metrics, authenticated with SigV4: awscurl --service monitoring --region YOUR_AWS_REGION \ "https://monitoring.YOUR_AWS_REGION.amazonaws.com/api/v1/query?query=engine_active_requests" If you’ve standardized on Grafana or another Prometheus-native observability stack, you can point your existing tooling at your SageMaker AI fleet: no parallel monitoring stack, no export pipeline. Choosing the right metric stream The two innovations complement the built-in metrics of SageMaker AI rather than replacing them. The following table maps common questions to the metric stream that answers them. Question Where to look Is the endpoint healthy? Are requests erroring or slow to start? Standard SageMaker metrics (ConcurrentRequestsPerModel, FirstChunkLatency, Invocation5XXErrors [truncated for AI cost control]