Updating SK to dotnet-0.24.230918.1-preview (#369)

### Motivation and Context

<!-- Thank you for your contribution to the chat-copilot repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

This pull request updates the SK package and includes several updates
and improvements to the project:
- Tokenizer change
- Error handling updates
- Chat optimizations

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

- The `SharpToken` package has been added, and `SemanticKernel` packages
have been updated to their latest preview versions
[dotnet-0.24.230918.1-preview](https://github.com/microsoft/semantic-kernel/releases/tag/dotnet-0.24.230918.1-preview).
- The ChatSkill has been updated to improve performance and bypass
audience extraction for the default user.
- Update `TokenUtilities.TokenCount` to use custom SharpToken token
counter implementation with cl100k_base encoding for calculating the
number of tokens in a string, improving the accuracy and consistency of
token counting in the project. Remove unused code for determining token
usage. Improve error handling by logging and throwing the exception.
- Refactor `ChatController` and `ChatSkill` to use `SafeInvokeAsync`
method for invoking asynchronous callback functions and tagging any
exception that occurs with function name. Also, update telemetry
tracking accordingly.
- Update `BotResponsePrompt.cs` constructor parameter name from
`systemResponse` to `systemInstructions`
- Simplified AlertMessage in client

![image](https://github.com/microsoft/chat-copilot/assets/125500434/8a838817-eecb-4af6-b532-db0146a54c18)


### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [Contribution
Guidelines](https://github.com/microsoft/chat-copilot/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/chat-copilot/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
~- [ ] All unit tests pass, and I have added new tests where possible~
- [x] I didn't break anyone 😄

webapi/Controllers/ChatController.cs

@@ -25,7 +25,6 @@
 using Microsoft.Extensions.Logging;
 using Microsoft.Extensions.Options;
 using Microsoft.Graph;
-using Microsoft.IdentityModel.Tokens;
 using Microsoft.SemanticKernel;
 using Microsoft.SemanticKernel.Diagnostics;
 using Microsoft.SemanticKernel.Orchestration;
@@ -147,23 +146,19 @@ public async Task<IActionResult> ChatAsync(
                 : null;
 
             result = await kernel.RunAsync(function!, contextVariables, cts?.Token ?? default);
+            this._telemetryService.TrackSkillFunction(ChatSkillName, ChatFunctionName, true);
         }
-        finally
+        catch (Exception ex)
         {
-            this._telemetryService.TrackSkillFunction(ChatSkillName, ChatFunctionName, (!result?.ErrorOccurred) ?? false);
-        }
-
-        if (result.ErrorOccurred)
-        {
-            if (result.LastException is OperationCanceledException || result.LastException?.InnerException is OperationCanceledException)
+            if (ex is OperationCanceledException || ex.InnerException is OperationCanceledException)
             {
                 // Log the timeout and return a 504 response
                 this._logger.LogError("The chat operation timed out.");
                 return this.StatusCode(StatusCodes.Status504GatewayTimeout, "The chat operation timed out.");
             }
 
-            var errorMessage = result.LastException!.Message.IsNullOrEmpty() ? result.LastException!.InnerException?.Message : result.LastException!.Message;
-            return this.BadRequest(errorMessage);
+            this._telemetryService.TrackSkillFunction(ChatSkillName, ChatFunctionName, false);
+            throw ex;
         }
 
         AskResult chatSkillAskResult = new()

webapi/CopilotChatWebApi.csproj

@@ -12,18 +12,19 @@
   <ItemGroup>
     <PackageReference Include="Azure.AI.FormRecognizer" Version="4.1.0" />
     <PackageReference Include="Microsoft.Azure.Cosmos" Version="3.35.4" />
-    <PackageReference Include="Microsoft.SemanticKernel" Version="0.23.230906.2-preview" />
-    <PackageReference Include="Microsoft.SemanticKernel.Connectors.AI.OpenAI" Version="0.23.230906.2-preview" />
-    <PackageReference Include="Microsoft.SemanticKernel.Connectors.Memory.AzureCognitiveSearch" Version="0.23.230906.2-preview" />
-    <PackageReference Include="Microsoft.SemanticKernel.Connectors.Memory.Chroma" Version="0.23.230906.2-preview" />
-    <PackageReference Include="Microsoft.SemanticKernel.Connectors.Memory.Qdrant" Version="0.23.230906.2-preview" />
-    <PackageReference Include="Microsoft.SemanticKernel.Connectors.Memory.Postgres" Version="0.23.230906.2-preview" />
-    <PackageReference Include="Microsoft.SemanticKernel.Skills.MsGraph" Version="0.23.230906.2-preview" />
-    <PackageReference Include="Microsoft.SemanticKernel.Skills.OpenAPI" Version="0.23.230906.2-preview" />
+    <PackageReference Include="Microsoft.SemanticKernel" Version="0.24.230918.1-preview" />
+    <PackageReference Include="Microsoft.SemanticKernel.Connectors.AI.OpenAI" Version="0.24.230918.1-preview" />
+    <PackageReference Include="Microsoft.SemanticKernel.Connectors.Memory.AzureCognitiveSearch" Version="0.24.230918.1-preview" />
+    <PackageReference Include="Microsoft.SemanticKernel.Connectors.Memory.Chroma" Version="0.24.230918.1-preview" />
+    <PackageReference Include="Microsoft.SemanticKernel.Connectors.Memory.Qdrant" Version="0.24.230918.1-preview" />
+    <PackageReference Include="Microsoft.SemanticKernel.Connectors.Memory.Postgres" Version="0.24.230918.1-preview" />
+    <PackageReference Include="Microsoft.SemanticKernel.Skills.MsGraph" Version="0.24.230918.1-preview" />
+    <PackageReference Include="Microsoft.SemanticKernel.Skills.OpenAPI" Version="0.24.230918.1-preview" />
     <PackageReference Include="Azure.Extensions.AspNetCore.Configuration.Secrets" Version="1.2.2" />
     <PackageReference Include="Microsoft.ApplicationInsights.AspNetCore" Version="2.21.0" />
     <PackageReference Include="Microsoft.Identity.Web" Version="2.13.4" />
     <PackageReference Include="PdfPig" Version="0.1.8" />
+    <PackageReference Include="SharpToken" Version="1.2.12" />
     <PackageReference Include="Swashbuckle.AspNetCore" Version="6.5.0" />
   </ItemGroup>
 

webapi/Extensions/SemanticKernelExtensions.cs

@@ -23,7 +23,6 @@
 using Microsoft.SemanticKernel.Connectors.Memory.Qdrant;
 using Microsoft.SemanticKernel.Diagnostics;
 using Microsoft.SemanticKernel.Memory;
-using Microsoft.SemanticKernel.Orchestration;
 using Microsoft.SemanticKernel.Skills.Core;
 using Npgsql;
 using Pgvector.Npgsql;
@@ -117,15 +116,22 @@ public static IKernel RegisterChatSkill(this IKernel kernel, IServiceProvider sp
     }
 
     /// <summary>
-    /// Propagate exception from within semantic function
+    /// Invokes an asynchronous callback function and tags any exception that occurs with function name.
     /// </summary>
-    public static void ThrowIfFailed(this SKContext context)
+    /// <typeparam name="T">The type of the result returned by the callback function.</typeparam>
+    /// <param name="callback">The asynchronous callback function to invoke.</param>
+    /// <param name="functionName">The name of the function that calls this method, for logging purposes.</param>
+    /// <returns>A task that represents the asynchronous operation and contains the result of the callback function.</returns>
+    public static async Task<T> SafeInvokeAsync<T>(Func<Task<T>> callback, string functionName)
     {
-        if (context.ErrorOccurred)
+        try
         {
-            var logger = context.LoggerFactory.CreateLogger(nameof(SKContext));
-            logger.LogError(context.LastException, "{0}", context.LastException?.Message);
-            throw context.LastException!;
+            // Invoke the callback and await the result
+            return await callback();
+        }
+        catch (Exception ex)
+        {
+            throw new SKException($"{functionName} failed.", ex);
         }
     }
 

webapi/Models/Response/BotResponsePrompt.cs

@@ -54,8 +54,7 @@ public class BotResponsePrompt
     public ChatCompletionContextMessages MetaPromptTemplate { get; set; } = new();
 
     public BotResponsePrompt(
-        string systemDescription,
-        string systemResponse,
+        string systemInstructions,
         string audience,
         string userIntent,
         string chatMemories,
@@ -65,7 +64,7 @@ public BotResponsePrompt(
         ChatCompletionContextMessages metaPromptTemplate
     )
     {
-        this.SystemPersona = string.Join("\n", systemDescription, systemResponse);
+        this.SystemPersona = systemInstructions;
         this.Audience = audience;
         this.UserIntent = userIntent;
         this.PastMemories = string.Join("\n", chatMemories, documentMemories).Trim();

webapi/Skills/ChatSkills/ChatSkill.cs

@@ -165,8 +165,6 @@ public async Task<string> ExtractUserIntentAsync(SKContext context, Cancellation
         // Get token usage from ChatCompletion result and add to context
         TokenUtilities.GetFunctionTokenUsage(result, context, this._logger, "SystemIntentExtraction");
 
-        result.ThrowIfFailed();
-
         return $"User intent: {result}";
     }
 
@@ -209,8 +207,6 @@ public async Task<string> ExtractAudienceAsync(SKContext context, CancellationTo
         // Get token usage from ChatCompletion result and add to context
         TokenUtilities.GetFunctionTokenUsage(result, context, this._logger, "SystemAudienceExtraction");
 
-        result.ThrowIfFailed();
-
         return $"List of participants: {result}";
     }
 
@@ -400,16 +396,21 @@ private async Task<ChatMessage> GetChatResponseAsync(string chatId, string userI
         var chatCompletion = this._kernel.GetService<IChatCompletion>();
         var promptTemplate = chatCompletion.CreateNewChat(systemInstructions);
 
-        // Get the audience
-        await this.UpdateBotResponseStatusOnClientAsync(chatId, "Extracting audience", cancellationToken);
-        var audience = await this.GetAudienceAsync(chatContext, cancellationToken);
-        chatContext.ThrowIfFailed();
-        promptTemplate.AddSystemMessage(audience);
+        // Bypass audience extraction if Auth is disabled
+        var audience = string.Empty;
+        if (!PassThroughAuthenticationHandler.IsDefaultUser(userId))
+        {
+            // Get the audience
+            await this.UpdateBotResponseStatusOnClientAsync(chatId, "Extracting audience", cancellationToken);
+            audience = await SemanticKernelExtensions.SafeInvokeAsync(
+                () => this.GetAudienceAsync(chatContext, cancellationToken), nameof(GetAudienceAsync));
+            promptTemplate.AddSystemMessage(audience);
+        }
 
         // Extract user intent from the conversation history.
         await this.UpdateBotResponseStatusOnClientAsync(chatId, "Extracting user intent", cancellationToken);
-        var userIntent = await this.GetUserIntentAsync(chatContext, cancellationToken);
-        chatContext.ThrowIfFailed();
+        var userIntent = await SemanticKernelExtensions.SafeInvokeAsync(
+            () => this.GetUserIntentAsync(chatContext, cancellationToken), nameof(GetUserIntentAsync));
         promptTemplate.AddSystemMessage(userIntent);
 
         // Calculate the remaining token budget.
@@ -419,8 +420,8 @@ private async Task<ChatMessage> GetChatResponseAsync(string chatId, string userI
         // Acquire external information from planner
         await this.UpdateBotResponseStatusOnClientAsync(chatId, "Acquiring external information from planner", cancellationToken);
         var externalInformationTokenLimit = (int)(remainingTokenBudget * this._promptOptions.ExternalInformationContextWeight);
-        var planResult = await this.AcquireExternalInformationAsync(chatContext, userIntent, externalInformationTokenLimit, cancellationToken);
-        chatContext.ThrowIfFailed();
+        var planResult = await SemanticKernelExtensions.SafeInvokeAsync(
+            () => this.AcquireExternalInformationAsync(chatContext, userIntent, externalInformationTokenLimit, cancellationToken), nameof(AcquireExternalInformationAsync));
 
         // Extract additional details about planner execution in chat context
         // TODO: [Issue #150, sk#2106] Accommodate different planner contexts once core team finishes work to return prompt and token usage.
@@ -480,7 +481,7 @@ private async Task<ChatMessage> GetChatResponseAsync(string chatId, string userI
             promptTemplate.AddSystemMessage(planResult);
         }
 
-        var promptView = new BotResponsePrompt(this._promptOptions.SystemDescription, this._promptOptions.SystemResponse, audience, userIntent, chatMemories, documentMemories, plannerDetails, chatHistory, promptTemplate);
+        var promptView = new BotResponsePrompt(systemInstructions, audience, userIntent, chatMemories, documentMemories, plannerDetails, chatHistory, promptTemplate);
         chatContext.Variables.Set(TokenUtilities.GetFunctionKey(this._logger, "SystemMetaPrompt")!, TokenUtilities.GetContextMessagesTokenCount(promptTemplate).ToString(CultureInfo.CurrentCulture));
 
         // Stream the response to the client
@@ -517,7 +518,6 @@ await SemanticChatMemoryExtractor.ExtractSemanticChatMemoryAsync(
     private async Task<string> GetAudienceAsync(SKContext context, CancellationToken cancellationToken)
     {
         SKContext audienceContext = context.Clone();
-
         var audience = await this.ExtractAudienceAsync(audienceContext, cancellationToken);
 
         // Copy token usage into original chat context
@@ -527,9 +527,6 @@ private async Task<string> GetAudienceAsync(SKContext context, CancellationToken
             context.Variables.Set(functionKey, tokenUsage);
         }
 
-        // Propagate the error
-        audienceContext.ThrowIfFailed();
-
         return audience;
     }
 
@@ -553,8 +550,6 @@ private async Task<string> GetUserIntentAsync(SKContext context, CancellationTok
             {
                 context.Variables.Set(functionKey!, tokenUsage);
             }
-
-            intentContext.ThrowIfFailed();
         }
 
         return userIntent;
@@ -598,13 +593,7 @@ private async Task<string> AcquireExternalInformationAsync(SKContext context, st
     {
         SKContext planContext = context.Clone();
         planContext.Variables.Set("tokenLimit", tokenLimit.ToString(new NumberFormatInfo()));
-
-        var plan = await this._externalInformationSkill.AcquireExternalInformationAsync(userIntent, planContext, cancellationToken);
-
-        // Propagate the error
-        planContext.ThrowIfFailed();
-
-        return plan;
+        return await this._externalInformationSkill.AcquireExternalInformationAsync(userIntent, planContext, cancellationToken);
     }
 
     /// <summary>

webapi/Skills/TokenUtilities.cs

@@ -7,7 +7,6 @@
 using Microsoft.Extensions.Logging;
 using Microsoft.SemanticKernel.AI.ChatCompletion;
 using Microsoft.SemanticKernel.Connectors.AI.OpenAI.AzureSdk;
-using Microsoft.SemanticKernel.Connectors.AI.OpenAI.Tokenizers;
 using Microsoft.SemanticKernel.Orchestration;
 using ChatCompletionContextMessages = Microsoft.SemanticKernel.AI.ChatCompletion.ChatHistory;
 
@@ -69,27 +68,40 @@ internal static Dictionary<string, int> EmptyTokenUsages()
     /// <returns> true if token usage is found in result context; otherwise, false.</returns>
     internal static void GetFunctionTokenUsage(SKContext result, SKContext chatContext, ILogger logger, string? functionName = null)
     {
-        var functionKey = GetFunctionKey(logger, functionName);
-        if (functionKey == null)
+        try
         {
-            return;
-        }
+            var functionKey = GetFunctionKey(logger, functionName);
+            if (functionKey == null)
+            {
+                return;
+            }
+
+            if (result.ModelResults == null || result.ModelResults.Count == 0)
+            {
+                logger.LogError("Unable to determine token usage for {0}", functionKey);
+                return;
+            }
 
-        if (result.ModelResults == null || result.ModelResults.Count == 0)
+            var tokenUsage = result.ModelResults.First().GetResult<ChatModelResult>().Usage.TotalTokens;
+            chatContext.Variables.Set(functionKey!, tokenUsage.ToString(CultureInfo.InvariantCulture));
+        }
+        catch (Exception e)
         {
-            logger.LogError("Unable to determine token usage for {0}", functionKey);
-            return;
+            logger.LogError(e, "Unable to determine token usage for {0}", functionName);
+            throw e;
         }
-
-        var tokenUsage = result.ModelResults.First().GetResult<ChatModelResult>().Usage.TotalTokens;
-        chatContext.Variables.Set(functionKey!, tokenUsage.ToString(CultureInfo.InvariantCulture));
     }
 
     /// <summary>
-    /// Calculate the number of tokens in a string.
+    /// Calculate the number of tokens in a string using custom SharpToken token counter implementation with cl100k_base encoding.
     /// </summary>
     /// <param name="text">The string to calculate the number of tokens in.</param>
-    internal static int TokenCount(string text) => GPT3Tokenizer.Encode(text).Count;
+    internal static int TokenCount(string text)
+    {
+        var tokenizer = SharpToken.GptEncoding.GetEncoding("cl100k_base");
+        var tokens = tokenizer.Encode(text);
+        return tokens.Count;
+    }
 
     /// <summary>
     /// Rough token costing of ChatHistory's message object.

webapp/src/libs/services/BaseService.ts

@@ -47,15 +47,17 @@ export class BaseService {
             });
 
             if (!response.ok) {
-                const responseText = await response.text();
-
                 if (response.status === 504) {
                     throw Object.assign(new Error('The request timed out. Please try sending your message again.'));
                 }
 
-                const errorMessage = `${response.status}: ${response.statusText}${
-                    responseText ? ` => ${responseText}` : ''
-                }`;
+                const responseText = await response.text();
+                const responseDetails = responseText.split('--->');
+                const errorDetails =
+                    responseDetails.length > 1
+                        ? `${responseDetails[0].trim()} ---> ${responseDetails[1].trim()}`
+                        : responseDetails[0];
+                const errorMessage = `${response.status}: ${response.statusText}${errorDetails}`;
 
                 throw Object.assign(new Error(errorMessage));
             }