This publish is co-written with Taras Tsarenko, Vitalil Bozadzhy, and Vladyslav Horbatenko.
As organizations worldwide search to make use of AI for social influence, the Danish humanitarian group Bevar Ukraine has developed a complete digital generative AI-powered assistant known as Victor, aimed toward addressing the urgent wants of Ukrainian refugees integrating into Danish society. This publish particulars our technical implementation utilizing AWS providers to create a scalable, multilingual AI assistant system that gives automated help whereas sustaining knowledge safety and GDPR compliance.
Bevar Ukraine was established in 2014 and has been on the forefront of supporting Ukrainian refugees in Denmark for the reason that full-scale struggle in 2022, offering help to over 30,000 Ukrainians with housing, job search, and integration providers. The group has additionally delivered greater than 200 tons of humanitarian assist to Ukraine, together with medical provides, mills, and important gadgets for civilians affected by the struggle.
Background and challenges
The combination of refugees into host nations presents a number of challenges, notably in accessing public providers and navigating complicated authorized procedures. Conventional assist methods, relying closely on human social employees, typically face scalability limitations and language limitations. Bevar Ukraine’s answer addresses these challenges by means of an AI-powered system that operates repeatedly whereas sustaining excessive requirements of service high quality.
Resolution overview
The answer’s spine contains a number of AWS providers to ship a dependable, safe, and environment friendly generative AI-powered digital assistant for Ukrainian refugees. The group consisting of three volunteer software program builders developed the answer inside weeks.
The next diagram illustrates the answer structure.
Amazon Elastic Compute Cloud (Amazon EC2) serves as the first compute layer, utilizing Spot Situations to optimize prices. Amazon Easy Storage Service (Amazon S3) gives safe storage for dialog logs and supporting paperwork, and Amazon Bedrock powers the core pure language processing capabilities. Bevar Ukraine makes use of Amazon DynamoDB for real-time knowledge entry and session administration, offering low-latency responses even beneath excessive load.
Within the means of implementation, we found that Anthropic’s Claude 3.5 giant language mannequin (LLM) is finest suited because of its superior dialogue logic and talent to take care of a human-like tone. It’s finest for thorough, reasoned responses and producing extra artistic content material, which makes Victor’s replies extra pure and fascinating.
Amazon Titan Embeddings G1 – Textual content v1.2 excels at producing high-quality vector representations of multilingual textual content, enabling environment friendly semantic search and similarity comparisons. That is notably helpful when Victor must retrieve related data from a big information base or match customers’ queries to beforehand seen inputs. Amazon Titan Embeddings additionally integrates easily with AWS, simplifying duties like indexing, search, and retrieval.
In real-world interactions with Victor, some queries require quick, particular solutions, whereas others want artistic technology or contextual understanding. By combining Anthropic’s Claude 3.5. for technology and Amazon Titan Embeddings G1 for semantic retrieval, Victor can route every question by means of essentially the most applicable pipeline, retrieving related context by means of embeddings and producing a response, leading to extra correct and context-aware solutions.
Amazon Bedrock gives a exceptional interface to name Anthropic’s Claude 3.5 and Amazon Titan Embeddings G1 (together with different fashions) with out creating separate integrations for every supplier, simplifying improvement and upkeep.
For multilingual assist, we used embedders that assist multi-language embeddings and translated our supplies utilizing Amazon Translate. This enhances the resilience of our Retrieval Augmented Era (RAG) system. The applying is constructed securely and makes use of AWS providers to perform this. AWS Key Administration Service (AWS KMS) simplifies the method of encrypting knowledge throughout the software, and Amazon API Gateway helps the purposes REST endpoints. Consumer authentication and authorization capabilities are supported by Amazon Cognito, which gives safe and scalable buyer id and entry administration (CIAM) capabilities.
The applying runs on AWS infrastructure utilizing providers which can be designed to be safe and scalable like Amazon S3, AWS Lambda, and DynamoDB.
Suggestions and suggestions
Constructing an AI assistant answer for refugees utilizing Amazon Bedrock and different AWS providers has supplied helpful insights into creating impactful AI-powered humanitarian options. By means of this implementation, we found key concerns that organizations ought to take into account when growing comparable options. The expertise highlighted the significance of balancing technical capabilities with human-centric design, offering multilingual assist, sustaining knowledge privateness, and creating scalable but cost-effective options. These learnings can function a basis for organizations wanting to make use of AI and cloud applied sciences to assist humanitarian causes, notably in creating accessible and useful digital help for displaced populations. The next are the primary
- Use the Amazon Bedrock playground to check a number of LLMs facet by facet utilizing the identical immediate. This helps you discover the mannequin that offers the highest quality, model, and tone of response in your particular use case (for instance, factual accuracy vs. conversational tone).
- Experiment with prompts and settings to enhance responses.
- Maintain prices in thoughts; arrange monitoring and budgets in AWS.
- For duties involving data retrieval or semantic search, choose an embedding mannequin whereas ensuring to choose the suitable settings. Take note of the scale of the embeddings, as a result of bigger vectors can seize extra which means however may improve prices. Additionally, examine that the mannequin helps the languages your software requires.
- In the event you’re utilizing a information base, use the Amazon Bedrock information base playground to experiment with how content material is chunked and what number of passages are retrieved for every question. Discovering the proper variety of retrieved passages could make a giant distinction in how clear and centered the ultimate solutions are—typically fewer, high-quality chunks work higher than sending an excessive amount of context.
- To implement security and privateness, use Amazon Bedrock Guardrails. Guardrails might help stop the mannequin from leaking delicate data, comparable to private knowledge or inner enterprise content material, and you’ll block dangerous responses or implement a particular tone and formatting model.
- Begin with a easy prototype, take a look at the embedding high quality in your area, and broaden iteratively.
Integration and enhancement layer
Bevar Ukraine has prolonged the core AWS infrastructure with a number of complementary applied sciences:
- Pinecone vector database – For environment friendly storage and retrieval of semantic embeddings
- DSPy framework – For structured immediate engineering and optimization of Anthropic’s Claude 3.5 Sonnet responses
- EasyWeek – For appointment scheduling and useful resource administration
- Telegram API – For UI supply
- Amazon Bedrock Guardrails – For safety coverage enforcement
- Amazon Rekognition – For doc verification
- GitHub-based steady integration and supply (CI/CD) pipeline – For fast function deployment
Key technical insights
The implementation revealed a number of essential technical concerns. The DSPy framework was essential in optimizing and enhancing our language mannequin prompts. By integrating extra layers of reasoning and context consciousness instruments, DSPy notably improved response accuracy, consistency, and depth. The group discovered that designing a sturdy information base with complete metadata was elementary to the system’s effectiveness.
GDPR compliance required cautious architectural selections, together with knowledge minimization, safe storage, and clear person consent mechanisms. Price optimization was achieved by means of strategic use of EC2 Spot Situations and implementation of API request throttling, leading to vital operational financial savings with out compromising efficiency.
Future enhancements
Our roadmap consists of a number of technical enhancements to reinforce the system’s capabilities:
- Implementing superior context dispatching utilizing machine studying algorithms to enhance service coordination throughout a number of domains
- Creating a complicated human-in-the-loop validation system for complicated circumstances requiring knowledgeable oversight
- Migrating appropriate parts to a serverless structure utilizing Lambda to optimize useful resource utilization and prices
- Enhancing the information base with superior semantic search capabilities and automatic content material updates
Outcomes
This answer, which serves a whole bunch of Ukrainian refugees in Denmark each day, demonstrates the potential of AWS providers in creating scalable, safe, and environment friendly AI-powered methods for social influence. In consequence, volunteers and staff of Bevar Ukraine have saved hundreds of hours, and as a substitute of answering repetitive questions from refugees, can assist them in additional difficult life conditions. For refugees, the digital assistant Victor is a lifeline assist that permits customers to get responses to essentially the most urgent questions on public providers in Denmark and plenty of different questions in seconds as a substitute of getting to attend for an accessible volunteer to assist. Given the huge information base Victor is utilizing to generate responses, the standard of assist has improved as nicely.
Conclusion
By means of cautious structure design and integration of complementary applied sciences, we’ve created a platform that successfully addresses the challenges confronted by refugees whereas sustaining excessive requirements of safety and knowledge safety.
The success of this implementation gives a blueprint for comparable options in different social service domains, probably supporting refugees and different folks in want all over the world, highlighting the significance of mixing sturdy cloud infrastructure with considerate system design to create significant social influence.
In regards to the Authors
Taras Tsarenko is a Program Supervisor at Bevar Ukraine. For over a decade on this planet of know-how, Taras has led the whole lot from tight-knit agile groups of 5 or extra to an organization of 90 people who turned the most effective small IT firm in Ukraine beneath 100 folks in 2015. Taras is a builder who thrives on the intersection of technique and execution, the place technical experience meets human influence, whether or not it’s streamlining workflows, fixing complicated issues, or empowering groups to create significant merchandise. Taras makes a speciality of AI-driven options and knowledge engineering, leveraging applied sciences like machine studying and generative AI utilizing Amazon SageMaker AI, Amazon Bedrock, Amazon OpenSearch Service, and extra. Taras is an AWS Licensed ML Engineer Affiliate.
Anton Garvanko is a Senior Analytics Gross sales Specialist for Europe North at AWS. As a finance skilled turned salesman, Anton spent 15 years in varied finance management roles in provide chain and logistics in addition to monetary providers industries. Anton joined Amazon over 5 years in the past and has been a part of specialist gross sales groups specializing in enterprise intelligence, analytics, and generative AI for over 3 years. He’s captivated with connecting the worlds of finance and IT by ensuring that enterprise intelligence and analytics powered by generative AI assist on a regular basis decision-making throughout industries and use circumstances.
Vitalii Bozadzhy is a Senior Developer with in depth expertise in constructing high-load, cloud-based options, specializing in Java, Golang, SWIFT, and Python. He makes a speciality of scalable backend methods, microservice architectures designed to automate enterprise processes, in addition to constructing dependable and safe cloud infrastructures. Moreover, he has expertise in optimizing compute sources and constructing superior options built-in into merchandise. His experience covers the complete improvement cycle—from design and structure to deployment and upkeep—with a robust concentrate on efficiency, fault tolerance, and innovation.
Vladyslav Horbatenko is a pc science scholar, Professor Assistant, and Information Scientist with a robust concentrate on synthetic intelligence. Vladyslav started his journey with machine studying, reinforcement studying, and deep studying, and steadily turned extra fascinated with giant language fashions (LLMs) and their potential influence. This led him to deepen his understanding of LLMs, and now he works on growing, sustaining, and bettering LLM-based options. He contributes to progressive initiatives whereas staying updated with the most recent developments in AI.
