Analyzing Traffic of LLM API Calls in Amazon Bedrock
Amazon Bedrock is a fully managed and serverless service that provides access to a range of foundation models (FMs) from top AI companies such as Amazon, Anthropic, Cohere, Meta, AI21 Labs, Mistral AI, Stability AI etc. through a single API call. It enables the development of AI applications on Amazon's cloud computing platform Amazon Web Services (AWS).
Network Traffic Analysis
The ATI team in Keysight has analyzed the network traffic (decrypted) of Amazon Bedrock API calls and found some interesting insights, which can be helpful for other researchers.
When we call Amazon Bedrock’s “Converse” and “InvokeModel” APIs using AWS’s “boto3” library, it generally sends a POST request to the Amazon Bedrock endpoint which utilizes TLS 1.3 (by default) for encryption and HTTP/1.1 for communication.
Let's take a detailed look at the decrypted traffic:
Request Components:
Figure 1: Sample Amazon Bedrock API Request
- Request Line
When we call the Amazon Bedrock API, the POST request URL (request line) structure looks like below:
/model/<model name>/<api_name>
- Model Name: Indicates the name (or ID) of the Large Language Model (LLM) the user wants to use to generate the contents. List of some example foundation models available in Amazon Bedrock service are -
Table 1: List of LLMs available in Amazon Bedrock service
- API Name: Indicates the version of the Amazon Bedrock API that the user wants to use. As example: "converse" and "invoke".
- Headers
The POST request contains the following headers:
- Host: bedrock-runtime.<AWS region name>.amazonaws.com (indicating the request is being sent to this particular API server).
- Accept-encoding: identity
- Content-Type: application/json
- User-Agent: Boto3/1.35.39 md/Botocore#1.35.39 ua/2.0 os/linux#6.8.0-47-generic md/arch#x86_64 lang/python#3.10.12 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.35.39
- X-Amz-Date: The timestamp of the request in ISO 8601 basic format (<YYYYMMDD>T<HHMMSS>Z) and is used in AWS Signature Version 4 for request authentication.
- X-Amz-Security-Token: Temporary security credential token provided by AWS Security Token Service (STS), used to authenticate API requests made with IAM roles or temporary credentials
- Authorization: Used to authenticate requests to AWS service. It contains the following sections
- Credential: Specifies the access key, the date, the AWS region, the service (bedrock) and the termination string (aws4_request).
- SignedHeaders: Lists the headers included in the signature calculation like content-type, host, x-amz-date, x-amz-security-token.
- Signature: The cryptographic hash that verifies the request's authenticity, generated using the secret access key.
- amz-sdk-invocation-id: A unique identifier assigned to each AWS SDK request, used for tracking and debugging purposes. It helps correlate logs and monitor individual request executions across AWS services.
- amz-sdk-request: attempt=1
- Content-Length: Length of the POST JSON data.
- Payload
The POST request body/payload contains the prompt that the users want to send to the Bedrock API server in JSON format. It can be of two types:
- For Converse API
For Converse API, the JSON body looks like below -
Figure 2: Sample Amazon Bedrock API Request Payload for Converse API Call
- For InvokeModel API
For InvokeModel API, the JSON body looks like below -
Figure 3: Sample Amazon Bedrock API Request Payload for InvokeModel API Call
Inside the HTTP Response:
After the successful authentication and processing of the API request, the Amazon Bedrock API server responds with a 200 Ok HTTP response which looks like below –
Figure 4: Sample 200 OK Response from Amazon Bedrock API Server
The HTTP response header includes the following fields –
- Date: Indicates the date and time when the response was generated, formatted according to the HTTP-date standard defined.
- Content-Type: application/json
- Connection: keep-alive
- x-amzn-RequestId: A unique identifier assigned by AWS to each API request, enabling tracking and troubleshooting.
- Content-Length: Length of the HTTP response JSON data.
The response also contains the answer of the user prompt in JSON format. It also can be of two types:
- For Converse API
For Converse API call, the response data looks like below:
Figure 5: Sample Amazon Bedrock API Response Payload for Converse API Call
- For InvokeModel API
For InvokeModel API call, the response data looks like below:
Figure 6: Sample Amazon Bedrock API Response Payload for InvokeModel API Call
Amazon Bedrock LLM API Calls Traffic Simulation in Keysight ATI
At Keysight Technologies, our Application and Threat Intelligence (ATI) team, researchers have examined the traffic pattern of Amazon Bedrock API call for various LLMs and added their support in ATI-2024-22 and ATI-2024-23 StrikePacks released on November 05, 2024 and November 22, 2024 respectively.
We have added 1 new Application “Amazon Bedrock API Call” and 11 new Superflows for different LLMs till now as shown below -
Figure 7: Amazon Bedrock API Call and its Superflows in BPS
Here, the traffic for all the Superflows is customizable which allows the user to choose their own values for Hostname, Content Type, User Agent, Accept-Encoding, AWS Region Name, Model ID, API Endpoint Name, X-Amz-Security-Token, Authorization, amz-sdk-invocation-id, User Prompt and Response Prompt during the BreakingPoint System (BPS) simulation as shown below –
Figure 8: Anthropic Claude 3 Haiku API Call Superflow in BPS
Leverage Subscription Service to Stay Ahead of Attacks
Keysight's Application and Threat Intelligence subscription provides daily malware and bi-weekly updates of the latest application protocols and vulnerabilities for use with Keysight test platforms. The ATI Research Centre continuously monitors threats as they appear in the wild. Customers of BreakingPoint now have access to attack campaigns for different advanced persistent threats, allowing BreakingPoint Customers to test their currently deployed security control's ability to detect or block such attacks.