Securely launch and scale your brokers and instruments on Amazon Bedrock AgentCore Runtime

Organizations are more and more excited concerning the potential of AI brokers, however many discover themselves caught in what we name “proof of idea purgatory”—the place promising agent prototypes battle to make the leap to manufacturing deployment. In our conversations with clients, we’ve heard constant challenges that block the trail from experimentation to enterprise-grade deployment:

“Our builders wish to use totally different frameworks and fashions for various use circumstances—forcing standardization slows innovation.”

“The stochastic nature of brokers makes safety extra advanced than conventional functions—we want stronger isolation between person periods.”

“We battle with identification and entry management for brokers that have to act on behalf of customers or entry delicate techniques.”

“Our brokers have to deal with numerous enter varieties—textual content, photographs, paperwork—usually with massive payloads that exceed typical serverless compute limits.”

“We will’t predict the compute assets every agent will want, and prices can spiral when overprovisioning for peak demand.”

“Managing infrastructure for brokers which may be a mixture of quick and long-running requires specialised experience that diverts our focus from constructing precise agent performance.”

Amazon Bedrock AgentCore Runtime addresses these challenges with a safe, serverless internet hosting setting particularly designed for AI brokers and instruments. Whereas conventional software internet hosting techniques weren’t constructed for the distinctive traits of agent workloads—variable execution instances, stateful interactions, and sophisticated safety necessities—AgentCore Runtime was purpose-built for these wants.

The service alleviates the infrastructure complexity that has stored promising agent prototypes from reaching manufacturing. It handles the undifferentiated heavy lifting of container orchestration, session administration, scalability, and safety isolation, serving to builders deal with creating clever experiences reasonably than managing infrastructure. On this put up, we focus on how one can accomplish the next:

• Use totally different agent frameworks and totally different fashions
• Deploy, scale, and stream agent responses in 4 strains of code
• Safe agent execution with session isolation and embedded identification
• Use state persistence for stateful brokers together with Amazon Bedrock AgentCore Reminiscence
• Course of totally different modalities with massive payloads
• Function asynchronous multi-hour brokers
• Pay just for used assets

Use totally different agent frameworks and fashions

One benefit of AgentCore Runtime is its framework-agnostic and model-agnostic strategy to agent deployment. Whether or not your group has invested in LangGraph for advanced reasoning workflows, adopted CrewAI for multi-agent collaboration, or constructed customized brokers utilizing Strands, AgentCore Runtime can use your present code base with out requiring architectural adjustments or any framework migrations. Refer to those samples on Github for examples.

With AgentCore Runtime, you’ll be able to combine totally different massive language fashions (LLMs) out of your most popular supplier, similar to Amazon Bedrock managed fashions, Anthropic’s Claude, OpenAI’s API, or Google’s Gemini. This makes certain your agent implementations stay transportable and adaptable because the LLM panorama continues to evolve whereas serving to you choose the fitting mannequin to your use case to optimize for efficiency, value, or different enterprise necessities. This provides you and your group the pliability to decide on your favourite or most helpful framework or mannequin utilizing a unified deployment sample.

Let’s study how AgentCore Runtime helps two totally different frameworks and mannequin suppliers:

For the total code examples, seek advice from langgraph_agent_web_search.py and strands_openai_identity.py on GitHub.

Each examples above present how you should use AgentCore SDK, whatever the underlying framework or mannequin selection. After you’ve gotten modified your code as proven in these examples, you’ll be able to deploy your agent with or with out the AgentCore Runtime starter toolkit, mentioned within the subsequent part.

Be aware that there are minimal additions, particular to AgentCore SDK, to the instance code above. Allow us to dive deeper into this within the subsequent part.

Deploy, scale, and stream agent responses with 4 strains of code

Let’s study the 2 examples above. In each examples, we solely add 4 new strains of code:

• Import – from bedrock_agentcore.runtime import BedrockAgentCoreApp
• Initialize – app = BedrockAgentCoreApp()
• Enhance – @app.entrypoint
• Run – app.run()

After getting made these adjustments, essentially the most easy approach to get began with agentcore is to make use of the AgentCore Starter toolkit. We recommend utilizing uv to create and handle native improvement environments and package deal necessities in python. To get began, set up the starter toolkit as follows:

uv pip set up bedrock-agentcore-starter-toolkit

Run the suitable instructions to configure, launch, and invoke to deploy and use your agent. The next video supplies a fast walkthrough.

To your chat fashion functions, AgentCore Runtime helps streaming out of the field. For instance, in Strands, find the next synchronous code:

consequence = agent(user_message)

Change the previous code to the next and deploy:

agent_stream = agent.stream_async(user_message)
    async for occasion in agent_stream:
        yield occasion #you'll be able to course of/filter these occasions earlier than yielding

For extra examples on streaming brokers, seek advice from the next GitHub repo. The next is an instance streamlit software streaming again responses from an AgentCore Runtime agent.

Safe agent execution with session isolation and embedded identification

AgentCore Runtime essentially adjustments how we take into consideration serverless compute for agentic functions by introducing persistent execution environments that may preserve an agent’s state throughout a number of invocations. Quite than the standard serverless mannequin the place features spin up, execute, and instantly terminate, AgentCore Runtime provisions devoted microVMs that may persist for as much as 8 hours. This allows subtle multi-step agentic workflows the place every subsequent name builds upon the collected context and state from earlier interactions inside the similar session. The sensible implication of that is that you could now implement advanced, stateful logic patterns that will beforehand require exterior state administration options or cumbersome workarounds to keep up context between perform executions. This doesn’t obviate the necessity for exterior state administration (see the next part on utilizing AgentCore Runtime with AgentCore Reminiscence), however is a standard want for sustaining native state and recordsdata briefly, inside a session context.

Understanding the session lifecycle

The session lifecycle operates by means of three distinct states that govern useful resource allocation and availability (see diagram under for a excessive degree view of this session lifecycle). Whenever you first invoke a runtime with a novel session identifier, AgentCore provisions a devoted execution setting and transitions it to an Energetic state throughout request processing or when background duties are operating.

The system routinely tracks synchronous invocation exercise, whereas background processes can sign their standing by means of HealthyBusy responses to well being verify pings from the service (see the later part on asynchronous workloads). Classes transition to Idle when not processing requests however stay provisioned and prepared for rapid use, lowering chilly begin penalties for subsequent invocations.

Lastly, periods attain a Terminated state after they presently exceed a 15-minute inactivity threshold, hit the 8-hour most period restrict, or fail well being checks. Understanding these state transitions is essential for designing resilient workflows that gracefully deal with session boundaries and useful resource cleanup. For extra particulars on session lifecycle-related quotas, seek advice from AgentCore Runtime Service Quotas.

The ephemeral nature of AgentCore periods implies that runtime state exists solely inside the boundaries of the lively session lifecycle. The information your agent accumulates throughout execution—similar to dialog context, person desire mappings, intermediate computational outcomes, or transient workflow state—stays accessible solely whereas the session persists and is totally purged when the session terminates.

For persistent information necessities that reach past particular person session boundaries, AgentCore Reminiscence supplies the architectural resolution for sturdy state administration. This purpose-built service is particularly engineered for agent workloads and provides each short-term and long-term reminiscence abstractions that may preserve person dialog histories, discovered behavioral patterns, and significant insights throughout session boundaries. See documentation right here for extra data on getting began with AgentCore Reminiscence.

True session isolation

Session isolation in AI agent workloads addresses elementary safety and operational challenges that don’t exist in conventional software architectures. In contrast to stateless features that course of particular person requests independently, AI brokers preserve advanced contextual state all through prolonged reasoning processes, deal with privileged operations with delicate credentials and recordsdata, and exhibit non-deterministic habits patterns. This creates distinctive dangers the place one person’s agent may doubtlessly entry one other’s information—session-specific data might be used throughout a number of periods, credentials may leak between periods, or unpredictable agent habits may compromise system boundaries. Conventional containerization or course of isolation isn’t adequate as a result of brokers want persistent state administration whereas sustaining absolute separation between customers.

Let’s discover a case research: In Might 2025, Asana deployed a brand new MCP server to energy agentic AI options (integrations with ChatGPT, Anthropic’s Claude, Microsoft Copilot) throughout its enterprise software program as a service (SaaS) providing. Attributable to a logic flaw in MCP’s tenant isolation and relying solely on person however not agent identification, requests from one group’s person may inadvertently retrieve cached outcomes containing one other group’s information. This cross-tenant contamination wasn’t triggered by a focused exploit however was an intrinsic safety fault in dealing with context and cache separation throughout agentic AI-driven periods.

The publicity silently continued for 34 days, impacting roughly 1,000 organizations, together with main enterprises. After it was found, Asana halted the service, remediated the bug, notified affected clients, and launched a repair.

AgentCore Runtime solves these challenges by means of full microVM isolation that goes past easy useful resource separation. Every session receives its personal devoted digital machine with remoted compute, reminiscence, and file system assets, ensuring agent state, software operations, and credential entry stay fully compartmentalized. When a session ends, the whole microVM is terminated and reminiscence sanitized, minimizing the chance of information persistence or cross-contamination. This structure supplies the deterministic safety boundaries that enterprise deployments require, even when coping with the inherently probabilistic and non-deterministic nature of AI brokers, whereas nonetheless enabling the stateful, personalised experiences that make brokers invaluable. Though different choices may present sandboxed kernels, with the power to handle your individual session state, persistence, and isolation, this shouldn’t be handled a strict safety boundary. AgentCore Runtime supplies constant, deterministic isolation boundaries no matter agent execution patterns, delivering the predictable safety properties required for enterprise deployments. The next diagram exhibits how two separate periods run in remoted microVM kernels.

AgentCore Runtime embedded identification

Conventional agent deployments usually battle with identification and entry administration, significantly when brokers have to act on behalf of customers or entry exterior companies securely. The problem turns into much more advanced in multi-tenant environments—for instance, the place it is advisable to make sure that Agent A accessing Google Drive on behalf of Consumer 1 can by no means unintentionally retrieve information belonging to Consumer 2.

AgentCore Runtime addresses these challenges by means of its embedded identification system that seamlessly integrates authentication and authorization into the agent execution setting. First, every runtime is related to a novel workload identification (you’ll be able to deal with this as a novel agent identification). The service helps two major authentication mechanisms for brokers utilizing this distinctive agent identification: IAM SigV4 Authentication for brokers working inside AWS safety boundaries, and OAuth primarily based (JWT Bearer Token Authentication) integration with present enterprise identification suppliers like Amazon Cognito, Okta, or Microsoft Entra ID.

When deploying an agent with AWS Id and Entry Administration (IAM) authentication, customers don’t have to include different Amazon Bedrock AgentCore Id particular settings or setup—merely configure with IAM authorization, launch, and invoke with the fitting person credentials.

When utilizing JWT authentication, you configure the authorizer through the CreateAgentRuntime operation, specifying your identification supplier (IdP)-specific discovery URL and allowed shoppers. Your present agent code requires no modification—you merely add the authorizer configuration to your runtime deployment. When a calling entity or person invokes your agent, they go their IdP-specific entry token as a bearer token within the Authorization header. AgentCore Runtime makes use of AgentCore Id to routinely validate this token in opposition to your configured authorizer and rejects unauthorized requests. The next diagram exhibits the circulation of knowledge between AgentCore runtime, your IdP, AgentCore Id, different AgentCore companies, different AWS companies (in orange), and different exterior APIs or assets (in purple).

Behind the scenes, AgentCore Runtime routinely exchanges validated person tokens for workload entry tokens (by means of the bedrock-agentcore:GetWorkloadAccessTokenForJWT API). This supplies safe outbound entry to exterior companies by means of the AgentCore credential supplier system, the place tokens are cached utilizing the mix of agent workload identification and person ID because the binding key. This cryptographic binding makes certain, for instance, Consumer 1’s Google token can by no means be accessed when processing requests for Consumer 2, no matter software logic errors. Be aware that within the previous diagram, connecting to AWS assets will be achieved just by modifying the AgentCore Runtime execution position, however connections to Amazon Bedrock AgentCore Gateway or to a different runtime would require reauthorization with a brand new entry token.

Probably the most easy approach to configure your agent with OAuth-based inbound entry is to make use of the AgentCore starter toolkit:

1. With the AWS Command Line Interface (AWS CLI), observe the prompts to interactively enter your OAuth discovery URL and allowed Shopper IDs (comma-separated).

2. With Python, use the next code:

 bedrock_agentcore_starter_toolkit  Runtime
 boto3.session  Session
boto_session  Session()
area  boto_sessionregion_name
area

discovery_url  ''

client_id  ''

agentcore_runtime  Runtime()
response  agentcore_runtimeconfigure(
    entrypoint"strands_openai.py",
    auto_create_execution_role,
    auto_create_ecr,
    requirements_file"necessities.txt",
    regionregion,
    agent_nameagent_name,
    authorizer_configuration{
        "customJWTAuthorizer": {
        "discoveryUrl": discovery_url,
        "allowedClients": [client_id]
        }
    }
    )

3. For outbound entry (for instance, in case your agent makes use of OpenAI APIs), first arrange your keys utilizing the API or the Amazon Bedrock console, as proven within the following screenshot.

4. Then entry your keys from inside your AgentCore Runtime agent code:

from bedrock_agentcore.identification.auth import requires_api_key

@requires_api_key(
    provider_name="openai-apikey-provider" # substitute with your individual credential supplier title
)
async def need_api_key(*, api_key: str):
    print(f'obtained api key for async func: {api_key}')
    os.environ["OPENAI_API_KEY"] = api_key

For extra data on AgentCore Id, seek advice from Authenticate and authorize with Inbound Auth and Outbound Auth and Internet hosting AI Brokers on AgentCore Runtime.

Use AgentCore Runtime state persistence with AgentCore Reminiscence

AgentCore Runtime supplies ephemeral, session-specific state administration that maintains context throughout lively conversations however doesn’t persist past the session lifecycle. Every person session preserves conversational state, objects in reminiscence, and native momentary recordsdata inside remoted execution environments. For brief-lived brokers, you should use the state persistence supplied by AgentCore Runtime without having to save lots of this data externally. Nonetheless, on the finish of the session lifecycle, the ephemeral state is completely destroyed, making this strategy appropriate just for interactions that don’t require information retention throughout separate conversations.

AgentCore Reminiscence addresses this problem by offering persistent storage that survives past particular person periods. Brief-term reminiscence captures uncooked interactions as occasions utilizing create_event, storing the whole dialog historical past that may be retrieved with get_last_k_turns even when the runtime session restarts. Lengthy-term reminiscence makes use of configurable methods to extract and consolidate key insights from these uncooked interactions, similar to person preferences, essential info, or dialog summaries. By retrieve_memories, brokers can entry this persistent information throughout fully totally different periods, enabling personalised experiences. The next diagram exhibits how AgentCore Runtime can use particular APIs to work together with Brief-term and Lengthy-term reminiscence in AgentCore Reminiscence.

This fundamental structure, of utilizing a runtime to host your brokers, and a mix of short- and long-term reminiscence has turn out to be commonplace in most agentic AI functions immediately. Invocations to AgentCore Runtime with the identical session ID helps you to entry the agent state (for instance, in a conversational circulation) as if it have been operating domestically, with out the overhead of exterior storage operations, and AgentCore Reminiscence selectively captures and buildings the precious data price preserving past the session lifecycle. This hybrid strategy means brokers can preserve quick, contextual responses throughout lively periods whereas constructing cumulative intelligence over time. The automated asynchronous processing of long-term recollections in keeping with every technique in AgentCore Reminiscence makes certain insights are extracted and consolidated with out impacting real-time efficiency, making a seamless expertise the place brokers turn out to be progressively extra useful whereas sustaining responsive interactions. This structure avoids the standard trade-off between dialog velocity and long-term studying, enabling brokers which are each instantly helpful and repeatedly bettering.

Course of totally different modalities with massive payloads

Most AI agent techniques battle with massive file processing attributable to strict payload dimension limits, usually capping requests at just some megabytes. This forces builders to implement advanced file chunking, a number of API calls, or exterior storage options that add latency and complexity. AgentCore Runtime removes these constraints by supporting payloads as much as 100 MB in dimension, enabling brokers to course of substantial datasets, high-resolution photographs, audio, and complete doc collections in a single invocation.

Think about a monetary audit state of affairs the place it is advisable to confirm quarterly gross sales efficiency by evaluating detailed transaction information in opposition to a dashboard screenshot out of your analytics system. Conventional approaches would require utilizing exterior storage similar to Amazon Easy Storage Service (Amazon S3) or Google Drive to obtain the Excel file and picture into the container operating the agent logic. With AgentCore Runtime, you’ll be able to ship each the excellent gross sales information and the dashboard picture in a single payload from the consumer:

large_payload = {
"immediate": "Examine the This autumn gross sales information with the dashboard metrics and determine any discrepancies",
"sales_data": base64.b64encode(excel_sales_data).decode('utf-8'),
"dashboard_image": base64.b64encode(dashboard_screenshot).decode('utf-8')
}

The agent’s entrypoint perform will be modified to course of each information sources concurrently, enabling this cross-validation evaluation:

@app.entrypoint
def audit_analyzer(payload, context):
    inputs = [
        {"text": payload.get("prompt", "Analyze the sales data and dashboard")},
        {"document": {"format": "xlsx", "name": "sales_data", 
                     "source": {"bytes": base64.b64decode(payload["sales_data"])}}},
        {"picture": {"format": "png", 
                  "supply": {"bytes": base64.b64decode(payload["dashboard_image"])}}}
    ]
    
    response = agent(inputs)
    return response.message['content'][0]['text']

To check out an instance of utilizing massive payloads, seek advice from the next GitHub repo.

Function asynchronous multi-hour brokers

As AI brokers evolve to sort out more and more advanced duties—from processing massive datasets to producing complete stories—they usually require multi-step processing that may take important time to finish. Nonetheless, most agent implementations are synchronous (with response streaming) that block till completion. Whereas synchronous, streaming brokers are a standard approach to expose agentic chat functions to customers, customers can’t work together with the agent when a process or software continues to be operating, view the standing of, or cancel background operations, or begin extra concurrent duties whereas others have nonetheless not accomplished.

Constructing asynchronous brokers forces builders to implement advanced distributed process administration techniques with state persistence, job queues, employee coordination, failure restoration, and cross-invocation state administration whereas additionally navigating serverless system limitations like execution timeouts (tens of minutes), payload dimension restrictions, and chilly begin penalties for long-running compute operations—a big heavy elevate that diverts focus from core performance.

AgentCore Runtime alleviates this complexity by means of stateful execution periods that preserve context throughout invocations, so builders can construct upon earlier work incrementally with out implementing advanced process administration logic. The AgentCore SDK supplies ready-to-use constructs for monitoring asynchronous duties and seamlessly managing compute lifecycles, and AgentCore Runtime helps execution instances as much as 8 hours and request/response payload sizes of 100 MB, making it appropriate for many asynchronous agent duties.

Getting began with asynchronous brokers

You may get began with simply a few code adjustments:

pip set up bedrock-agentcore

To construct interactive brokers that carry out asynchronous duties, merely name add_async_task when beginning a process and complete_async_task when completed. The SDK routinely handles process monitoring and manages compute lifecycle for you.

# Begin monitoring a process
task_id = app.add_async_task("data_processing")

# Do your work...
# (your enterprise logic right here)

# Mark process as full
app.complete_async_task(task_id)

These two technique calls remodel your synchronous agent into a completely asynchronous, interactive system. Check with this pattern for extra particulars.

The next instance exhibits the distinction between a synchronous agent that streams again responses to the person instantly vs. a extra advanced multi-agent state of affairs the place longer operating, asynchronous background buying brokers use Amazon Bedrock AgentCore Browser to automate a buying expertise on amazon.com on behalf of the person.

Pay just for Used Sources

Amazon Bedrock AgentCore Runtime introduces a consumption-based pricing mannequin that essentially adjustments the way you pay for AI agent infrastructure. In contrast to conventional compute fashions that cost for allotted assets no matter utilization, AgentCore Runtime payments you just for what you really use nevertheless lengthy you employ it; stated otherwise, you don’t should pre-allocate assets like CPU or GB Reminiscence, and also you don’t pay for CPU assets throughout I/O wait durations. This distinction is especially invaluable for AI brokers, which usually spend important time ready for LLM responses or exterior API calls to finish. Here’s a typical Agent occasion loop, the place we solely count on the purple bins to be processed inside Runtime:

The LLM name (mild blue) and gear name (inexperienced) bins take time, however are run exterior the context of AgentCore Runtime; customers solely pay for processing that occurs in Runtime itself (purple bins). Let’s take a look at some real-world examples to grasp the influence:

Buyer help agent instance

Think about a buyer help agent that handles 10,000 person inquiries per day. Every interplay includes preliminary question processing, information retrieval from Retrieval Augmented Era (RAG) techniques, LLM reasoning for response formulation, API calls to order techniques, and closing response technology. In a typical session lasting 60 seconds, the agent may actively use CPU for less than 18 seconds (30%) whereas spending the remaining 42 seconds (70%) ready for LLM responses or API calls to finish. Reminiscence utilization can fluctuate between 1.5 GB to 2.5 GB relying on the complexity of the shopper question and the quantity of context wanted. With conventional compute fashions, you’d pay for the total 60 seconds of CPU time and peak reminiscence allocation. With AgentCore Runtime, you solely pay for the 18 seconds of lively CPU processing and the precise reminiscence consumed moment-by-moment:

CPU value: 18 seconds × 1 vCPU × ($0.0895/3600) = $0.0004475
 Reminiscence value: 60 seconds × 2GB common × ($0.00945/3600) = $0.000315
 Complete per session: $0.0007625

For 10,000 every day periods, this represents a 70% discount in CPU prices in comparison with conventional fashions that will cost for the total 60 seconds.

Information evaluation agent instance

The financial savings turn out to be much more dramatic for information processing brokers that deal with advanced workflows. A monetary evaluation agent processing quarterly stories may run for 3 hours however have extremely variable useful resource wants. Throughout information loading and preliminary parsing, it would use minimal assets (0.5 vCPU, 2 GB reminiscence). When performing advanced calculations or operating statistical fashions, it would spike to 2 vCPU and eight GB reminiscence for simply quarter-hour of the entire runtime, whereas spending the remaining time ready for batch operations or mannequin inferences at a lot decrease useful resource utilization. By charging just for precise useful resource consumption whereas sustaining your session state throughout I/O waits, AgentCore Runtime aligns prices straight with worth creation, making subtle agent deployments economically viable at scale.

Conclusion

On this put up, we explored how AgentCore Runtime simplifies the deployment and administration of AI brokers. The service addresses important challenges which have historically blocked agent adoption at scale, providing framework-agnostic deployment, true session isolation, embedded identification administration, and help for giant payloads and long-running, asynchronous brokers, all with a consumption primarily based mannequin the place you pay just for the assets you employ.

With simply 4 strains of code, builders can securely launch and scale their brokers whereas utilizing AgentCore Reminiscence for persistent state administration throughout periods. For hands-on examples on AgentCore Runtime masking easy tutorials to advanced use circumstances, and demonstrating integrations with numerous frameworks similar to LangGraph, Strands, CrewAI, MCP, ADK, Autogen, LlamaIndex, and OpenAI Brokers, seek advice from the next examples on GitHub:

Concerning the authors

Shreyas Subramanian is a Principal Information Scientist and helps clients by utilizing Generative AI and deep studying to unravel their enterprise challenges utilizing AWS companies like Amazon Bedrock and AgentCore. Dr. Subramanian contributes to cutting-edge analysis in deep studying, Agentic AI, basis fashions and optimization methods with a number of books, papers and patents to his title. In his present position at Amazon, Dr. Subramanian works with numerous science leaders and analysis groups inside and out of doors Amazon, serving to to information clients to greatest leverage state-of-the-art algorithms and methods to unravel enterprise important issues. Outdoors AWS, Dr. Subramanian is a specialist reviewer for AI papers and funding by way of organizations like Neurips, ICML, ICLR, NASA and NSF.

Kosti Vasilakakis is a Principal PM at AWS on the Agentic AI group, the place he has led the design and improvement of a number of Bedrock AgentCore companies from the bottom up, together with Runtime. He beforehand labored on Amazon SageMaker since its early days, launching AI/ML capabilities now utilized by hundreds of firms worldwide. Earlier in his profession, Kosti was a knowledge scientist. Outdoors of labor, he builds private productiveness automations, performs tennis, and explores the wilderness together with his household.

Vivek Bhadauria is a Principal Engineer at Amazon Bedrock with virtually a decade of expertise in constructing AI/ML companies. He now focuses on constructing generative AI companies similar to Amazon Bedrock Brokers and Amazon Bedrock Guardrails. In his free time, he enjoys biking and mountaineering.