AWS Bedrock provider requires repetition_penalty to be set to 0

[raghotham]

System Info

Used in Colab

Information

• The official example scripts
• My own modified scripts

🐛 Describe the bug

Without setting repetition_penalty to 0 (or less than 0):

response = client.inference.chat_completion(
    model_id=model_id,
    messages=[
        {"role": "system", "content": "You are a friendly assistant."},
        {"role": "user", "content": "Write a two-sentence poem about llama."},
    ],
    #sampling_params={"repetition_penalty": 0},
)

print(response.completion_message.content)

There's an error:

ValidationException: An error occurred (ValidationException) when calling the InvokeModel operation: Malformed input request: #: extraneous key [repetition_penalty] is not permitted, please reformat your input and try again.

Setting the repetition penalty fixes the problem:

response = client.inference.chat_completion(
    model_id=model_id,
    messages=[
        {"role": "system", "content": "You are a friendly assistant."},
        {"role": "user", "content": "Write a two-sentence poem about llama."},
    ],
    sampling_params={"repetition_penalty": 0},
)

print(response.completion_message.content)

Output

With soft fur and gentle eyes so bright,
The llama roams, a peaceful sight.

Error logs

---------------------------------------------------------------------------
ValidationException                       Traceback (most recent call last)
[<ipython-input-10-4fb4c68b419c>](https://localhost:8080/#) in <cell line: 0>()
----> 1 response = client.inference.chat_completion(
      2     model_id=model_id,
      3     messages=[
      4         {"role": "system", "content": "You are a friendly assistant."},
      5         {"role": "user", "content": "Write a two-sentence poem about llama."},

<> 16 frames
[/usr/local/lib/python3.11/dist-packages/botocore/client.py](https://localhost:8080/#) in _make_api_call(self, operation_name, api_params)
   1021             )
   1022             error_class = self.exceptions.from_code(error_code)
-> 1023             raise error_class(parsed_response, operation_name)
   1024         else:
   1025             return parsed_response

ValidationException: An error occurred (ValidationException) when calling the InvokeModel operation: Malformed input request: #: extraneous key [repetition_penalty] is not permitted, please reformat your input and try again.

Expected behavior

Should work without setting repetition_penalty.