Deploy a full stack voice AI agent with Amazon Nova Sonic

AI-powered speech options are reworking contact facilities by enabling pure conversations between prospects and AI brokers, shortening wait occasions, and dramatically decreasing operational prices—all with out sacrificing the human-like interplay prospects anticipate. With the current launch of Amazon Nova Sonic in Amazon Bedrock, now you can construct subtle conversational AI brokers that talk naturally by way of voice, with out the necessity for separate speech recognition and text-to-speech parts. Amazon Nova Sonic is a speech-to-speech mannequin in Amazon Bedrock that permits real-time, human-like voice conversations.

Whereas many early Amazon Nova Sonic implementations targeted on native improvement, this resolution supplies a whole cloud-deployed structure that you should use as a basis for constructing actual proof of idea functions. This asset is deployable by way of the AWS Cloud Growth Equipment (AWS CDK) and supplies a basis for constructing additional Amazon Nova use instances utilizing preconfigured infrastructure parts, whereas permitting you to customise the structure to handle your particular enterprise necessities.

On this publish, we present create an AI-powered name middle agent for a fictional firm referred to as AnyTelco. The agent, named Telly, can deal with buyer inquiries about plans and companies whereas accessing real-time buyer knowledge utilizing customized instruments applied with the Mannequin Context Protocol (MCP) framework.

Resolution overview

The next diagram supplies an outline of the deployable resolution.

The answer consists of the next layers:

• Frontend layer – The frontend layer of this method is constructed with scalability and efficiency in thoughts:
• Communication layer – The communication layer facilitates seamless real-time interactions:
  • Community Load Balancer manages WebSocket connections. WebSockets allow two-way interactive communication periods between a consumer’s browser and the server, which is important for real-time audio streaming functions.
  • Amazon Cognito supplies consumer authentication and JSON net token (JWT) validation. Amazon Cognito supplies consumer authentication, authorization, and consumer administration for net and cell functions, assuaging the necessity to construct and keep your personal identification techniques.
• Processing layer – The processing layer types the computational spine of the system:
  • Amazon Elastic Container Service (Amazon ECS) runs the containerized backend service.
  • AWS Fargate supplies the serverless compute backend. Orchestration is supplied by the Amazon ECS engine.
  • The Python backend processes audio streams and manages Amazon Nova Sonic interactions.
• Intelligence layer – The intelligence layer makes use of AI and knowledge applied sciences to energy the core functionalities:
  • The Amazon Nova Sonic mannequin in Amazon Bedrock handles speech processing.
  • Amazon DynamoDB shops buyer info.
  • Amazon Bedrock Information Bases connects basis fashions (FMs) together with your group’s knowledge sources, permitting AI functions to reference correct, up-to-date info particular to what you are promoting.

The next sequence diagram highlights the move when a consumer initiates dialog. The consumer solely indicators in a single time, however authentication Steps 3 and 4 occur each time the consumer begins a brand new session. The conversational loop in Steps 6–12 is repeated all through the conversational interplay. Steps a–c solely occur when the Amazon Nova Sonic agent decides to make use of a software. In eventualities with out software use, the move goes instantly from Step 9 to Step 10.

Stipulations

Earlier than getting began, confirm that you’ve got the next:

Deploy the answer

You’ll find the answer and full deployment directions on the GitHub repository. The answer makes use of the AWS CDK to automate infrastructure deployment. Use the next code terminal instructions to get began in your AWS Command Line Interface (AWS CLI) setting:

git clone https://github.com/aws-samples/sample-sonic-cdk-agent.git 
cd nova-s2s-call-center 

# Configure setting variables
cp template.env .env

# Edit .env together with your settings

# Deploy the answer 
./deploy.sh 

The deployment creates two AWS CloudFormation stacks:

• Community stack for digital non-public cloud (VPC) and networking parts
• Stack for software assets

The output of the second stack offers you a CloudFront distribution hyperlink, which takes you to the login web page.

You possibly can create an Amazon Cognito admin consumer with the next AWS CLI command:

aws cognito-idp admin-create-user 
  --user-pool-id YOUR_USER_POOL_ID 
  --username USERNAME 
  --user-attributes Identify=electronic mail,Worth=USER_EMAIL 
  --temporary-password TEMPORARY_PASSWORD 
  --region YOUR_AWS_REGION

The previous command makes use of the next parameters:

• YOUR_USER_POOL_ID: The ID of your Amazon Cognito consumer pool
• USERNAME: The specified consumer identify for the consumer
• USER_EMAIL: The e-mail tackle of the consumer
• TEMPORARY_PASSWORD: A brief password for the consumer
• YOUR_AWS_REGION: Your AWS Area (for instance, us-east-1)

Log in together with your momentary password from the CloudFront distribution hyperlink, and you may be requested to set a brand new password.

You possibly can select Begin Session to begin a dialog together with your assistant. Experiment with prompts and completely different instruments on your use case.

Customizing the applying

A key characteristic of this resolution is its flexibility—you’ll be able to tailor the AI agent’s capabilities to your particular use case. The pattern implementation demonstrates this extensibility by way of customized instruments and information integration:

• Buyer info lookup – Retrieves buyer profile knowledge from DynamoDB utilizing telephone numbers as keys
• Information base search – Queries an Amazon Bedrock information base for firm info, plan particulars, and pricing

These options showcase improve the performance of Amazon Nova Sonic with exterior knowledge sources and domain-specific information. The structure is designed for seamless customization in a number of key areas.

Modifying the system immediate

The answer features a UI in which you’ll be able to regulate the AI agent’s conduct by modifying its system immediate. This allows fast iteration on the agent’s character, information base, and dialog model with out redeploying all the software.

Including new instruments

You may also prolong the AI agent’s capabilities by implementing extra instruments utilizing the MCP framework. The method includes:

• Implementing the software logic, sometimes as a brand new Python module
• Registering the software with the MCP server through the use of the @mcp_server.software customized decorator and defining the software specification, together with its identify, description, and enter schema in /backend/instruments/mcp_tool_registry.py

For instance, the next code illustrates add a information base lookup software:

@mcp_server.software(
    identify="lookup",
    description="Runs question towards a information base to retrieve info."
)
async def lookup_tool(
    question: Annotated[str, Field(description="the query to search")]
) -> dict:
    """Lookup info within the information base"""
    outcomes = knowledge_base_lookup.foremost(question)
    return outcomes

The decorator handles registration with the MCP server, and the operate physique accommodates your software’s implementation logic.

Increasing the information base

The answer makes use of Amazon Bedrock Information Bases to supply the AI agent with company-specific info. You possibly can replace this information base with:

• Continuously requested questions and their solutions
• Product catalogs and specs
• Firm insurance policies and procedures

Clear up

You possibly can take away the stacks with the next command:

# transfer to the cdk folder, assuming you might be within the undertaking root folder
cd cdk
# Removes each stacks sequentially
npx cdk destroy --all

Conclusion

AI brokers are reworking how organizations strategy customer support, with options providing the flexibility to deal with a number of conversations concurrently, present constant service across the clock, and scale immediately whereas sustaining high quality and decreasing operational prices. This resolution makes these advantages accessible by offering a deployable basis for Amazon Nova Sonic functions on AWS. The answer demonstrates how AI brokers can successfully deal with buyer inquiries, entry real-time knowledge, and supply personalised service—all whereas sustaining the pure conversational move that prospects anticipate.

By combining the Amazon Nova Sonic mannequin with a strong cloud structure, safe authentication, and versatile software integration, organizations can shortly transfer from idea to proof of idea. This resolution is not only serving to construct voice AI functions, it’s serving to firms drive higher buyer satisfaction and productiveness throughout a spread of industries.

To study extra, confer with the next assets:

Concerning the authors

Reilly Manton is a Options Architect in AWS Telecoms Prototyping. He combines visionary considering and technical experience to construct modern options. Specializing in generative AI and machine studying, he empowers telco prospects to boost their technological capabilities.

Shuto Araki is a Software program Growth Engineer at AWS. He works with prospects in telecom business specializing in AI safety and networks. Outdoors of labor, he enjoys biking all through the Netherlands.

Ratan Kumar is a Principal Options Architect at Amazon Net Companies.A trusted expertise advisor with over 20 years of expertise working throughout a spread of business domains, Ratan’s ardour lies in empowering enterprise prospects innovate and remodel their enterprise by unlocking the potential of AWS cloud.

Chad Hendren is a Principal Options Architect at Amazon Net Companies. His ardour is AI/ML and Generative AI utilized to Buyer Expertise. He’s a broadcast creator and inventor with 30 years of telecommunications expertise.