Issues with specifying Amazon Bedrock Guardrails

[robert-hanuschke]

Hi,

I'm trying to use litellm with Amazon Bedrock. However, I seem to do something wrong specifying Bedrock Guardrails.

In my code, I use the bedrock client of boto3 to list_guardrails and have my desired one here:

{'ResponseMetadata': {..., 'guardrails': [{'id': 'mts5z4bj1570', 'arn': 'arn:aws:bedrock:us-east-1:xxxxxxxxxxx:guardrail/mts5z4bj1570', 'status': 'READY', 'name': 'test', 'version': 'DRAFT', ...]}

However, when I try to use it, I get

{"message":"The model returned the following errors: The guardrail identifier or version provided in the request does not exist."}

The prompt works when I remove the guardrailConfig. The completion call:

            litellm_result = completion(
                model=f"bedrock/{os.environ['MODEL_ID']}",
                messages=[
                    {
                        "role": "system",
                        "content": RATING_PROMPT
                    },
                    {
                        "role": "user",
                        "content": "\n".join(
                            [
                                "Rate this story:",
                                f"Title: {db_article.title}",
                                f"Text: {db_article.text}",
                            ]
                        ),
                    },
                ],
                guardrailConfig={
                    "trace": "enabled",
                    "guardrailIdentifier": "mts5z4bj1570",
                    "guardrailVersion": "DRAFT",
                },
            )

I'm using highly privileged credentials here - in fact the same ones that I used to create the Guardrail in the first place. Should not be a permission error.

Any idea what my issue might be here and how to resolve it?