This submit is co-authored by Jay Kshirsagar and Ronald Quan from Qualtrics. The content material and opinions on this submit are these of the third-party creator and AWS isn’t answerable for the content material or accuracy of this submit.
Qualtrics, based in 2002, is a pioneering software program firm that has spent over twenty years creating distinctive frontline experiences, constructing high-performing groups, and designing merchandise that folks love. Because the creators and stewards of the Expertise Administration (XM) class, Qualtrics serves over 20,000 shoppers globally, bringing humanity, connection, and empathy again to companies throughout numerous industries, together with retail, authorities, and healthcare.
Qualtrics’s complete XM platform allows organizations to constantly perceive, measure, and enhance the experiences they ship for patrons, workers, and the broader market. With its three core product suites—XM for Buyer Expertise, XM for Worker Expertise, and XM for Analysis & Technique—Qualtrics supplies actionable insights and purpose-built options that empower firms to ship distinctive experiences.
Qualtrics harnesses the facility of generative AI, cutting-edge machine studying (ML), and the newest in pure language processing (NLP) to offer new purpose-built capabilities which might be precision-engineered for expertise administration (XM). These AI capabilities are purpose-built to assist organizations of all sizes deeply perceive and tackle the wants of each buyer, worker, and stakeholder—driving stronger connections, elevated loyalty, and sustainable development.
On this submit, we share how Qualtrics constructed an AI platform powered by Amazon SageMaker and Amazon Bedrock.
AI at Qualtrics
Qualtrics has a deep historical past of utilizing superior ML to energy its industry-leading expertise administration platform. Early 2020, with the push for deep studying and transformer fashions, Qualtrics created its first enterprise-level ML platform known as Socrates. Constructed on high of SageMaker, this new platform enabled ML scientists to effectively construct, take a look at, and ship new AI-powered capabilities for the Qualtrics XM suite. This robust basis in ML and AI has been a key driver of Qualtrics’s innovation in expertise administration.
Qualtrics AI, a strong engine that sits on the coronary heart of the corporate’s XM platform, harnesses the newest advances in ML, NLP, and AI. Educated on Qualtrics’s expansive database of human sentiment and expertise information, Qualtrics AI unlocks richer, extra customized connections between organizations and their clients, workers, and stakeholders. Qualtrics’s unwavering dedication to innovation and buyer success has solidified its place as the worldwide chief in expertise administration.
To be taught extra about how AI is reworking expertise administration, go to this weblog from Qualtrics.
Socrates platform: Powering AI at Qualtrics
Qualtrics AI is powered by a custom-built ML platform, a synergistic suite of instruments and companies designed to allow a various set of Qualtrics personae—researchers, scientists, engineers, and data staff—to harness the transformative energy of AI and ML. Qualtrics refers to it internally because the “Socrates” platform. It makes use of managed AWS companies like SageMaker and Amazon Bedrock to allow your entire ML lifecycle. Information staff can supply, discover, and analyze Qualtrics information utilizing Socrates’s ML workbenches and AI Information Infrastructure. Scientists and researchers are enabled to conduct analysis, prototype, develop, and practice fashions utilizing a bunch of SageMaker options. ML engineers can take a look at, productionize, and monitor a heterogeneous set of ML fashions possessing a variety of capabilities, inference modes, and manufacturing visitors patterns. Accomplice software groups are supplied with an abstracted mannequin inference interface that makes the combination of an ML mannequin into the Qualtrics product a seamless engineering expertise. This holistic strategy allows inner groups to seamlessly combine superior AI and ML capabilities into their workflows and decision-making processes.
Science Workbench
The Socrates Science Workbench, purpose-built for Qualtrics Information and Information Employees, supplies a strong platform for mannequin coaching and hyperparameter optimization (HPO) with a JupyterLab interface, help for a variety of programming languages, and safe, scalable infrastructure via SageMaker integration, giving customers the pliability and reliability to give attention to their core ML duties. Customers can reap the benefits of the strong and dependable infrastructure of SageMaker to take care of the confidentiality and integrity of their information and fashions, whereas additionally benefiting from the scalability that SageMaker supplies to deal with even essentially the most demanding ML workloads.
AI Information Infrastructure
Socrates’s AI Information Infrastructure is a complete and cohesive end-to-end ML information ecosystem. It encompasses a safe and scalable information retailer built-in with the Socrates Science Workbench, enabling customers to effortlessly retailer, handle, and share datasets with capabilities for anonymization, schematization, and aggregation. The AI Information Infrastructure additionally supplies scientists with interfaces for distributed compute, information pulls and enrichment, and ML processing.
AI Playground
The AI Playground is a user-friendly interface that gives Socrates customers with direct entry to the highly effective language fashions and different generative AI capabilities hosted on the Socrates platform utilizing backend instruments like SageMaker Inference, Amazon Bedrock, and OpenAI GPT, permitting them to experiment and quickly prototype new concepts with out intensive coding or technical experience. By constantly integrating the newest fashions, the AI Playground empowers Socrates customers to remain on the forefront of developments in massive language fashions (LLMs) and different cutting-edge generative AI applied sciences, exploring their potential and discovering new methods to drive innovation.
Mannequin deployment for inference
The Socrates platform encompasses a refined mannequin deployment infrastructure that’s important for the scalable implementation of ML and AI fashions. This infrastructure permits customers to host fashions throughout the number of {hardware} choices accessible for SageMaker endpoints, offering the pliability to pick a deployment surroundings that optimally meets their particular wants for inference, whether or not these wants are associated to efficiency optimization, cost-efficiency, or explicit {hardware} necessities.
One of many defining traits of the Socrates mannequin deployment infrastructure is its functionality to simplify the complexities of mannequin internet hosting. This enables customers to focus on the important job of deploying their fashions for inference inside the bigger Socrates ecosystem. Customers profit from an environment friendly and user-friendly interface that allows them to effortlessly package deal their fashions, modify deployment settings, and put together them for inference use.
By providing an adaptable mannequin deployment resolution, the Socrates platform makes positive ML fashions created inside the system are easily built-in into real-world purposes and workflows. This integration not solely hurries up the transition to manufacturing but additionally maximizes the utilization of Qualtrics’s AI-driven options, fostering innovation and offering vital enterprise worth to its clients.
Mannequin capability administration
Mannequin capability administration is a crucial part that provides environment friendly and dependable supply of ML fashions to Qualtrics customers by offering oversight of mannequin entry and the allocation of computing assets throughout a number of customers. The Socrates workforce intently displays useful resource utilization and units up price limiting and auto scaling insurance policies, the place relevant, to satisfy the evolving calls for of every use case.
Unified GenAI Gateway
The Socrates platform’s Unified GenAI Gateway simplifies and streamlines entry to LLMs and embedding fashions throughout the Qualtrics ecosystem. The Unified GenAI Gateway is an API that gives a standard interface for customers to work together with all the platform-supported LLMs and embedding fashions, no matter their underlying suppliers or internet hosting environments. Because of this Socrates customers can use the facility of cutting-edge language fashions with out having to fret concerning the complexities of integrating with a number of distributors or managing self-hosted fashions.
The standout function of the Unified GenAI Gateway is its centralized integration with inference platforms like SageMaker Inference and Amazon Bedrock. which permits the Socrates workforce to deal with the intricate particulars of mannequin entry, authentication, and attribution on behalf of customers. This not solely simplifies the consumer expertise but additionally allows price attribution and management mechanisms, ensuring the consumption of those highly effective AI assets is rigorously monitored and aligned with particular use instances and billing codes. Moreover, the Unified GenAI Gateway boasts capabilities like rate-limiting help, ensuring the system’s assets are effectively allotted, and an upcoming semantic caching function that may additional optimize mannequin inference and improve total efficiency.
Managed Inference APIs (powered by SageMaker Inference)
The Socrates Managed Inference APIs present a complete suite of companies that simplify the combination of superior ML and AI capabilities into Qualtrics purposes. This infrastructure, constructed on high of SageMaker Inference, handles the complexities of mannequin deployment, scaling, and upkeep, boasting a rising catalog of production-ready fashions.
Managed Inference APIs provide each asynchronous and synchronous modes to accommodate a variety of software use instances. Importantly, these managed APIs include assured production-level SLAs, offering dependable efficiency and cost-efficiency as utilization scales. With available pre-trained Qualtrics fashions for inference, the Socrates platform empowers Qualtrics software groups to give attention to delivering distinctive consumer experiences, with out the burden of constructing and sustaining AI infrastructure.
GenAI Orchestration Framework
Socrates’s GenAI Orchestration Framework is a group of instruments and patterns designed to streamline the event and deployment of LLM-powered purposes inside the Qualtrics ecosystem. The framework consists of such instruments/frameworks akin to:
- Socrates Agent Platform, constructed on high of LangGraph Platform offering a versatile orchestration framework to develop brokers as graphs that expedite supply of agentic options whereas centralizing core infrastructure and observability parts.
- A GenAI SDK, offering simple coding comfort for interacting with LLMs and third-party orchestration packages
- Immediate Lifecycle Administration Service (PLMS) for sustaining the safety and governance of prompts
- LLM guardrail tooling, enabling LLM customers to outline the protections they need utilized to their mannequin inference
- Synchronous and asynchronous inference gateways
These instruments all contribute to the general reliability, scalability, and efficiency of the LLM-powered purposes constructed upon it. Capabilities of the Socrates AI App Framework are anticipated to develop and evolve alongside the fast developments within the subject of LLMs. Because of this Qualtrics customers all the time have entry to the newest and most cutting-edge AI capabilities from generative AI inference platforms like SageMaker Inference and Amazon Bedrock, empowering them to harness the transformative energy of those applied sciences with higher ease and confidence.
Ongoing enhancements to the Socrates platform utilizing SageMaker Inference
Because the Socrates platform continues to evolve, Qualtrics is constantly integrating the newest developments in SageMaker Inference to additional improve the capabilities of their AI-powered ecosystem:
- Improved price, efficiency, and usefulness of generative AI inference – One outstanding space of focus is the combination of price and efficiency optimizations for generative AI inference. The SageMaker Inference workforce has launched progressive strategies to optimize the usage of accelerators, enabling SageMaker Inference to scale back basis mannequin (FM) deployment prices by 50% on common and latency by 20% on common with inference parts. Utilizing this function, we’re engaged on reaching vital price financial savings and efficiency enhancements for Qualtrics clients operating their generative AI workloads on the Socrates platform. As well as, SageMaker has streamlined deployment of open supply LLMs and FMs with simply three clicks. This user-friendly performance removes the complexity historically related to deploying these superior fashions, empowering extra Qualtrics clients to harness the facility of generative AI inside their workflows and purposes.
- Improved auto scaling speeds – The SageMaker workforce has developed a complicated auto scaling functionality to raised deal with the scaling necessities of generative AI fashions. These enhancements scale back considerably (from a number of minutes to underneath a minute), decreasing auto scaling instances by as much as 40% and auto scaling detection by six instances for Meta Llama 3 8B, enabling Socrates customers to quickly scale their generative AI workloads on SageMaker to satisfy spikes in demand with out compromising efficiency.
- Simple deployment of self-managed OSS LLMs – Utilizing the brand new functionality from SageMaker Inference for a extra streamlined and intuitive course of to package deal your generative AI fashions reduces the technical complexity that was historically related to this job. This, in flip, empowers a wider vary of Socrates customers, together with software groups and subject material specialists, to make use of the transformative energy of those cutting-edge AI applied sciences inside their workflows and decision-making processes.
- Generative AI inference optimization toolkit – Qualtrics can be actively utilizing the newest developments within the SageMaker Inference optimization toolkit inside the Socrates platform, which presents two instances increased throughput whereas decreasing prices by as much as 50% for generative AI inference. By integrating utilizing capabilities, Socrates is engaged on decreasing the price of generative AI inference. This breakthrough is especially impactful for Qualtrics’s clients, who depend on the Socrates platform to energy AI-driven purposes and experiences.
“By seamlessly integrating SageMaker Inference into our Socrates platform, we’re capable of ship inference developments in AI to our world buyer base. The generative AI inference from capabilities in SageMaker like inference parts, sooner auto scaling, straightforward LLM deployment, and the optimization toolkit have been a sport changer for Qualtrics to scale back the associated fee and enhance the efficiency for our generative AI workloads. The extent of sophistication and ease of use that SageMaker Inference brings to the desk is outstanding.”
– James Argyropoulos, Sr AI/ML Engineer at Qualtrics.
Partnership with SageMaker Inference
Since adopting SageMaker Inference, the Qualtrics Socrates workforce has been a key collaborator within the growth of AI capabilities in SageMaker Inference. Constructing on experience to serve Socrates customers, Qualtrics has labored intently with the SageMaker Inference workforce to reinforce and develop the platform’s generative AI functionalities. From the early phases of generative AI, they provided invaluable insights and experience to the SageMaker workforce. This has enabled the introduction of a number of new options and optimizations which have strengthened the platform’s generative AI choices, together with:
- Value and efficiency optimizations for generative AI inference – Qualtrics helped the SageMaker Inference workforce construct a new inference functionality for SageMaker Inference to scale back FM deployment prices by 50% on common and latency by 20% on common with inference parts. This function delivers vital price financial savings and efficiency enhancements for patrons operating generative AI inference on SageMaker.
- Quicker auto scaling for generative AI inference – Qualtrics has helped the SageMaker workforce develop These enhancements have diminished auto scaling instances by as much as 40% for fashions like Meta Llama 3 and elevated auto scaling detection pace by six instances sooner. With this, generative AI inference can scale with altering visitors with out compromising efficiency.
- Inference optimization toolkit for generative AI inference – Qualtrics has been instrumental in giving suggestions for AWS to launch the inference optimization toolkit, which will increase throughput by as much as two instances sooner and reduces latency by 50%.
- Launch of multi-model endpoint (MME) help for GPU – MMEs permit clients to scale back inference prices by as much as 90%. Qualtrics was instrumental in serving to AWS with the launch of this function by offering beneficial suggestions.
- Launch of asynchronous inference – Qualtrics was a launch accomplice for and has performed a key function in serving to AWS enhance the providing to provide clients optimum price-performance.
The partnership between Qualtrics and the SageMaker Inference workforce has been instrumental in advancing the state-of-the-art in generative AI inside the AWS ecosystem. Qualtrics’s deep area data and technical proficiency have performed a vital function in shaping the evolution of this quickly creating subject on the SageMaker Inference.
“Our partnership with the SageMaker Inference product workforce has been instrumental in delivering unimaginable efficiency and value advantages for Socrates platform customers operating AI Inference workloads. By working hand in hand with the SageMaker workforce, we’ve been capable of introduce sport altering optimizations which have diminished AI inference prices a number of folds for a few of our use instances. We sit up for continued innovation via beneficial partnership to enhance state-of-the-art AI inference capabilities.”
– Jay Kshirsagar, Senior Supervisor, Machine Studying
Conclusion
The Socrates platform underscores Qualtrics’s dedication to advancing innovation in expertise administration by flawlessly integrating superior AI and ML applied sciences. Because of a robust partnership with the SageMaker Inference workforce, the platform has seen enhancements that enhance efficiency, scale back prices, and improve the accessibility of AI-driven options inside the Qualtrics XM suite. As AI expertise continues to develop quickly, the Socrates platform is geared to empower Qualtrics’s AI groups to innovate and ship distinctive buyer experiences.
Concerning the Authors
Jay Kshirsagar is a seasoned ML chief driving GenAI innovation and scalable AI infrastructure at Qualtrics. He has constructed high-impact ML groups and delivered enterprise-grade LLM options that energy key product options.
Ronald Quan is a Workers Engineering Supervisor for the Information Intelligence Platform workforce inside Qualtrics. The workforce’s constitution is to allow, expedite and evolve AI and Agentic developments on the Socrates platform. He focuses on the workforce’s technical roadmap and strategic alignment with the enterprise wants.
Saurabh Trikande is a Senior Product Supervisor for Amazon Bedrock and SageMaker Inference. He’s obsessed with working with clients and companions, motivated by the aim of democratizing AI. He focuses on core challenges associated to deploying advanced AI purposes, inference with multi-tenant fashions, price optimizations, and making the deployment of Generative AI fashions extra accessible. In his spare time, Saurabh enjoys mountain climbing, studying about progressive applied sciences, following TechCrunch, and spending time together with his household.
Micheal Nguyen is a Senior Startup Options Architect at AWS, specializing in utilizing AI/ML to drive innovation and develop enterprise options on AWS. Michael holds 12 AWS certifications and has a BS/MS in Electrical/Laptop Engineering and an MBA from Penn State College, Binghamton College, and the College of Delaware.
Ranga Malaviarachchi is a Sr. Buyer Options Supervisor within the ISV Strategic Accounts group at AWS. He has been intently related to Qualtrics over the previous 4 years in supporting their AI initiatives. Ranga holds a BS in Laptop Science and Engineering and an MBA from Imperial School London.
