Managed GenAI in der Cloud: AWS Bedrock vs. Azure AI Studio

Introduction

Since the release of ChatGPT by OpenAI, it has become increasingly clear that generative AI will fundamentally change many business processes (1, 2). New models are published every day, new AI SaaS companies are founded, and new theoretical approaches are tested. However, this rapidly changing environment does not always align with the requirements of the business world, which prioritizes efficient and practical applications. To meet these challenges, the leading cloud providers now offer platforms that simplify the development of AI-supported applications and bring them closer to business needs. In this blog post, we will introduce and compare two of these Managed GenAI platforms, AWS Bedrock and Azure AI Studio, providing an overview of this new development at the intersection of AI and cloud computing.

Overview of the Services

But what exactly a constitutes such a platform? Essentially, it is a collection of foundational models and other technologies such as knowledge databases and tools for developing, testing, and deploying AI applications. Since these services are cloud-based, they offer scalable infrastructure and can be accessed through the consoles of their respective cloud providers or via their software development kits (SDKs). For example, to build a chatbot, you could select a foundational model with just a few clicks, write an instruction for a specific task (a system prompt), and test the bot directly in a playground. Once the results are satisfactory, this bot can be integrated into an app using an API endpoint.

The most common use cases for these Managed GenAI platforms are AI agents (often based on the Retrieval Augmented Generation (RAG) workflow (3)), which are generally more powerful than standard chatbots like ChatGPT. This is because they have access to relevant background information or may be able to perform specific tasks better than a base model. In the following sections, we will introduce and compare the core components of such an AI agent—base models, knowledge bases, functions, and guardrails—in AWS Bedrock and Azure AI Studio. Finally, we will demonstrate how an AI agent works in practice using two example architectures.

Foundation Models

Foundation models form the basis of most modern generative AI applications. On the one hand, these models act as an interface between humans and machines, allowing us to communicate with the program in natural language. On the other hand, they act as the "brain" that plans and executes the interaction with tools such as knowledge bases and functions, ultimately generating coherent responses in human language. As shown in the graphic below, both providers offer a wide range of proprietary and open-source models. It is important to note that both cloud providers closely collaborate with leading AI research labs and primarily offer models from these partnerships. Azure uses GPT models from OpenAI, while AWS integrates Claude models from AnthropicAI. Additionally, both have developed proprietary models (AWS Titan and Microsoft Phi) and also offer models from Meta's Llama series and CohereAI. Overall, both platforms provide a broad selection of language-based, multi-modal, or text-to-image models in different sizes, with Azure currently supporting a greater number of models. Model availability depends heavily on the region, with new models usually launched in the U.S. before becoming available in Europe.

Cost Structure

The cost structure in Azure AI Studio differs from model to model. Most large models are billed on a per-use basis, meaning you pay a certain amount per token in input or output, with one word equal to approximately one to two tokens, depending on the language (4). Some models can also be deployed on dedicated virtual machines, which allows better control but is significantly more expensive, especially for use cases with a small number of users. In AWS Bedrock, large models are also billed based on usage, but there is an option to save money by provisioning capacity upfront. You can also import your own models and run them on your own compute resources, although this is more complex and often more costly. To summarize, we see Azure AI Studio as having an advantage in foundational models, primarily due to the greater model availability. However, the market for foundational models is changing rapidly, and the influence of powerful open-source models remains to be seen.

Knowledge Bases

Another essential component of AI agents and RAG systems are knowledge bases, which provide background information that the base model uses to answer questions. This is typically achieved by dividing text-based information into chunks, converting these chunks into vectors using an embedding model, and storing them in a vector database. These databases are designed to retrieve the most semantically relevant vectors based on a search query. This contextual information allows the base model to reference specific knowledge when generating a response. Thus, the key components of knowledge bases in Managed GenAI platforms are the embedding models and vector databases.

AWS Bedrock currently offers two embedding models in Europe: AWS Titan Embeddings and Cohere Text Embed. Azure, on the other hand, offers embedding models from OpenAI's Ada series (002 and 003). All these models are billed based on usage, although tasks such as text extraction from tables or PowerPoint slides, as well as chunking, may incur additional costs. Overall, these models are powerful and versatile, but they do not represent the full range of embedding models used globally. It remains to be seen when alternatives trained in different languages or multi-modally will be introduced.

The vector database offerings of the two platforms differ primarily in billing models. AWS and Azure offer serverless databases as the default solution (AWS OpenSearch Serverless and Azure AI Search). While these are easy to set up, they are billed hourly and can quickly become significant cost drivers. Alternatively, you can use your own database services such as AWS Aurora or Azure Cosmos DB, which may offer cost savings but are more complex to configure. In contrast to Azure AI Studio, AWS Bedrock also allows integration with third-party vector databases such as Pinecone and Redis, which is often the cheapest option as they are billed per request (read or write entries). In both platforms, vector databases can be queried with filters to limit access to specific documents, enabling document-level access control or topic-specific searches. In our opinion, AWS Bedrock offers slightly more versatile solutions when it comes to knowledge bases due to its third party support as well as a more diverse vector store offering.

Functions (Actions)

Function calls, also known as actions, are essential in agent-based AI applications. They enable the base model to go beyond simple text generation and call external APIs or predefined functions. This is critical for handling complex tasks, such as querying an SQL database or sending emails. Function calls extend the model's capabilities by enabling it to perform specific actions. Interaction with a knowledge base, as described earlier, is also an example of a function call, built into both AWS Bedrock and Azure AI Studio. In this section, we will explore how generic function calls can be implemented in both platforms.

In AWS Bedrock, user-defined functions are managed via action groups. In these groups, you provide the agent with descriptions of up to ten functions in natural language, along with required input variables. When the agent processes a request, the base model uses these definitions during planning to decide whether a particular function should be executed. The function itself must be explicitly defined, either through AWS Lambda, using templates that contain the necessary authorizations, or through fully user-managed Lambda functions. These functions can be tested directly in the AWS Bedrock Playground, enabling seamless integration and rapid iterations.

In Azure AI Studio, functions are configured in the Assistants tab. Here, you define function schemas as JSON, specifying when a function should be executed and what input variables are required. However, there is no native integration with Azure Functions yet that would allow direct execution of functions in the playground. Instead, the results of function calls must be inserted manually to generate the final response. As Azure AI Studio Assistants is still in public preview, easier integration is expected in the future. One advantage of Azure AI Studio is the availability of commonly used tools, such as a code interpreter, which allows the agent to write and execute code to handle tasks requiring calculations. Overall, AWS is ahead when it comes to function calls, as it provides seamless integration and the ability to test functions directly in the Playground.

Content Filters (Guardrails)

Content filters are crucial for ensuring that generative AI applications are safe and responsible, preventing the generation of inappropriate content and protecting sensitive information. They also protect chatbots from malicious interactions, such as prompt injection attacks, where predefined behavior is bypassed through manipulative input.

In AWS Bedrock, the Guardrails tab provides comprehensive security mechanisms that go beyond the built-in capabilities of many base models. These include content filters that block categories such as violence, hate speech, sexual content, and insults. Topic filters prevent the processing of undesirable content, which is useful, for example, to stop a customer service bot from being used for irrelevant topics. Word filters block specific words or phrases, including profanity and custom terms. Additionally, filters are available to mask or block personally identifiable information (PII) and other sensitive data. Prompt attack protection is built into Bedrock to prevent prompt injection and jailbreak attacks. Lastly, a new feature in AWS Bedrock is a hallucination safeguard that prevents the base model from providing false information.

Azure AI Studio also offers robust content filters. These include thematic filters that block harmful content in a similar way to AWS, along with a blacklist for blocking specific words or regular expressions. Protection against prompt attacks is also included. As Azure AI Studio’s content filters are still in public preview, further improvements and simpler integrations are expected soon.

Both cloud providers offer advanced security mechanisms that go beyond the capabilities of individual base models. However, AWS Bedrock offers more detailed configuration options and comprehensive security mechanisms specifically designed to protect PII and prevent hallucinations.

Orchestration and Evaluation

Now that we understand the core components of AI applications with agents, we can explore the mechanism behind agent response generation—orchestration. This involves understanding how an agent moves from a user query to an answer. While primarily relevant for agents, both cloud providers offer features for building multi-step AI workflows based on similar principles.

AWS integrates orchestration directly into its Agents feature. Each agent follows an orchestration flow divided into four steps: First, the agent checks whether the query aligns with the system prompt and can be answered. Next, it plans which tools (knowledge bases and functions) should be used and in what order. During implementation, the tools are utilized, and follow-up questions are formulated if needed. Finally, the results are summarized into a user-friendly answer. The prompts used in the flow are predefined but can be customized in the Advanced Prompts tab to suit specific use cases. The responses to individual prompts can be reviewed in the Playground or during API calls to understand how the agent generates an answer. In Azure AI Studio Assistants, a similar workflow exists, but there is no option to view or customize the prompts.

Additionally, AWS and Azure both offer Prompt Flows, allowing workflows that combine specific prompts to a base model with user-defined logic, such as Python or regular expressions. These features are useful for applications like classification or more complex workflows.

While both platforms provide advanced orchestration solutions, AWS offers more customization options with adaptable orchestration prompts in its agents. In terms of evaluation, both companies still have room for improvement, but new evaluation mechanisms will likely emerge soon.

Example Architectures

We are now equipped to understand the example architecture of an AI agent using AWS Bedrock and Azure AI Studio. Let’s start with the data fed into the knowledge base. In AWS, this data resides in an S3 bucket, while in Azure, it is stored in a storage account. Both platforms offer specific options for connecting to sources like SharePoint, but the bucket/storage account setup is a more generic solution, allowing connection to various data sources via pipelines. In AWS, data is pulled in, chunked, and embeddings are generated using a model like Cohere Embed, with the results stored in a third-party vector database such as Pinecone. The required services (embedding model and vector database) can be set up directly from the AWS Bedrock console. In Azure, a similar process occurs, with Document Intelligence, Azure OpenAI Embedding Models, and AI Search working together to move the data from chunking to embedding and into storage in a vector database.

Both platforms allow the definition of functions (Action Groups in AWS Bedrock Agents and Tools in Azure AI Studio Assistants), with AWS allowing Lambda functions to be stored alongside the schema. As an orchestration and response model, you can configure different models in both AWS and Azure to serve as the core of the AI agent. When users ask questions, both platforms route queries and responses through guardrails to ensure security. We find it easier to set up a complete solution in AWS Bedrock, as all services—except for the S3 bucket—can be configured within Bedrock itself. In contrast, Azure requires separate setup and integration of services like Document Intelligence and Search Service into Azure AI Studio.

Conclusion

Both AWS Bedrock and Azure AI Studio are powerful platforms that facilitate the rapid, systematic development and scalable deployment of AI-supported applications, particularly AI agents. Azure provides excellent access to foundational models with sophisticated guardrails and integrates seamlessly into the Azure cloud and other Microsoft environments. AWS Bedrock, on the other hand, stands out for its advanced integration of functions and knowledge bases for agents, offering customizable orchestration and extensive guardrails. Both platforms are at the forefront of technology, making them ideal for developing enterprise AI solutions. As both continue to evolve rapidly, it will be exciting to follow their progress closely.