[Lex] Multi-language issue when using iOS SDK - AWSLexInteractionKit

[qing-at-work]

Describe the bug
When we are using AWSLexInteractionKit, we have noticed that for Japanese language, Lex bot response text contains question marks instead of actual Japanese characters. For European languages such as Spanish, French etc., they seem to work fine.

To Reproduce
Steps to reproduce the behavior:

1. Configure a Lex bot that returns text in Japanese
2. Use AWSLexInteractionKit to interact with the Lex bot, such as this example
3. Check the output text

Observed Behavior
Output text contains question marks.

Expected Behavior
Output text contains Japanese characters

Areas of the SDK you are using (AWSMobileClient, Cognito, Pinpoint, IoT, etc)?
AWSLex

Environment(please complete the following information):

• SDK Version: 2.24.2
• Dependency Manager: Cocoapods
• Swift Version : 5
• Xcode Version: 12.5.1

Device Information (please complete the following information):

• Device: iPhone 12 simulator
• iOS Version: iOS 14.6
• Specific to simulators: n/a

Additional context
Looks like the switchModeInput.outputText is set in AWSLexInteractionKit.m, and it’s using response.message field. According to the documentation, this field can only be used for a limit set of languages.
We also noticed this commit, which indicated that message field in PostContentResponse has been deprecated, so does this mean AWSLexInteractionKit should also expose encodedMessage field via its interface?

[NALanhnt2]

me too

[github-actions]

This issue is stale because it has been open for 14 days with no activity. Please, provide an update or it will be automatically closed in 7 days.

[palpatim]

Currently, Lex returns the message as part of HTTP response headers, which restricts the character set in the response. Currently to unblock customers wanting to use special characters in speech-to-text use cases, they would need to use base64 encoded strings when creating the bot and base64 decode the responses.

Next steps:

• Investigate whether this is something that we can do in the Lex interaction kit (mid-level client), or whether it has to be done at the low-level client
• Investigate what, if any, backend configuration options need to be set up to support this
• Investigate what, if any, backend code a customer would have to provide to support this

[atierian]

Thanks for opening this feature request. We don't plan on adding support for this in the AWS SDK for iOS at this time.