AWS adds regional routing and native audio to SageMaker
Amazon Web Services added three components to SageMaker AI on November 19, 2026: guided deployment for an audio model, cross-region routing, and default monitoring. Use cases gain continuity, but operational load remains on the client side.
Amazon Web Services expanded SageMaker AI on November 19, 2026, with three new features aimed at production, not demonstration: the arrival of PolyAI's Dialog-RSN-1 in JumpStart, cross-region inference routing, and default monitoring for certain generative endpoints. Together, these additions reduce some integration work but leave telephony, speech synthesis, session resumption, and part of operational control to clients.
Deploying faster does not mean delegating everything
Dialog-RSN-1, PolyAI's audio model, joins the Amazon SageMaker JumpStart catalog with one-click deployment, evaluation tools, and four optimization profiles. From the SageMaker AI console, a user can select the model and then launch a real-time endpoint on a preconfigured runtime stack, without manually selecting a container or handpicking among dozens of infrastructure combinations.
The model itself processes the client's audio, combining speech recognition, response generation, turn-taking management, and function calling. PolyAI claims this direct processing preserves tone, hesitations, and timing that a more classic chain might smooth out after transcription. However, the service returns text or a function call, not synthetic voice: each deployment must therefore add its own text-to-speech layer. The bot talks, but it does not sing yet.
Before deployment, the model card displays notably the p50 latency, time to first token, and throughput. The offered profiles prioritize either latency, throughput, cost, or a general balance. The benefit is simple: adapt the configuration to call volume and response constraints, rather than applying a single setting to all uses.
Le routage interrégions règle la panne, pas la capacité magique
On the same November 19, 2026, AWS made cross-region inference routing generally available in SageMaker AI. The idea is to direct a request to existing capacity in another AWS region or account when local capacity is lacking, without forcing teams to build their own failover, fault detection, and recovery mechanisms.
The router chooses already deployed destinations: it does not create capacity or endpoints. When a region can no longer add the requested instance, requests can still be served elsewhere, but only if a target already exists. Clients therefore retain responsibility for positioning capacity, which is less romantic than "automatic failover" but more realistic.
AWS specifies that decisions rely on a health score calculated from recent requests. Server errors, throttles, and timeouts lower the score, while model latency is not part of the evaluation. An instance can thus remain in play as long as it responds correctly, even if slowly. The router also applies gradual traffic return rules after recovery, without letting clients adjust thresholds, weights, or failover order.
For real-time endpoints, retry behavior remains controlled: a second attempt is launched only if the model did not start, for example after throttling, error 503, or capacity refusal. Timeout errors or model errors are not retried. On governance, AWS exposes the served recipient, routing reasons, and OpenTelemetry metrics in CloudWatch.
What the router changes for operations teams
The service shifts part of failover engineering into SageMaker AI, but not the bill or capacity strategy. AWS does not charge extra for routing; however, clients pay standard endpoint rates and CloudWatch metrics. And if a region must remain ready to take over traffic, capacity must be maintained there. High availability is rarely free, even when it has a nice product name.
Technical limits also matter. The router does not support response streaming, limits real-time requests to about one minute to get a destination response, and accepts up to 6 MB per request. Each router can handle up to 20 routing targets, with up to 10 destinations per target. AWS also recommends having a router in two regions and switching client-side if the main entry must survive a regional outage.
Default monitoring targets drifts, not surprises
AWS also enabled default monitoring for certain generative inference SageMaker AI endpoints compatible with vLLM containers and GPU instances. This monitoring relies on AWS Health and targets progressive drifts: increasing latency, underutilized GPUs, missing service optimizations, or configurations becoming suboptimal after deployment.
Alerts do not rely on request or response content but on endpoint metrics, GPU usage, and service configuration. In case of drift, AWS Health sends a notification that opens in SageMaker Studio an agentic flow prefilled with the concerned endpoint. The user can then launch a SageMaker Python SDK notebook that compares the current configuration to alternatives on separate hardware before any production change.
Recommendations notably concern tensor parallelism, instance choice or size, speculative decoding, GPU kernels, or a newer container version. The examples given by AWS remain illustrative: one mentions a median time to first token increase of about 200 milliseconds over a week, another a GPU whose utilization stays at 12% or less for most of the week. Thresholds are not predefined since the system first compares to recent history.
Key points
- November 19, 2026: three AWS announcements around SageMaker AI.
- PolyAI Dialog-RSN-1 arrives in JumpStart with guided deployment.
- Cross-region routing remains limited to already active targets.
- Default monitoring targets latency and GPU drifts.
- Clients retain telephony, voice, and business logic.
- AWS does not charge for routing but charges for infrastructure.
By the numbers
- 30 minutes — maximum duration of a real-time connection before disconnection, according to AWS.
- 32,000 tokens — context window of the Dialog-RSN-1 model.
- $45/hour — hourly rate on ml.p4d.24xlarge for real-time inference.
- $110/hour — hourly rate on ml.p5.48xlarge for real-time inference.
- 20 — maximum number of routing targets per router.