Application logs are not emitted directly to Application Insights, but are being relayed through the Function host with host.json taking effect

[nlykkei]

Description

I'm configuring my Azure Function app to send logs directly to Application Insights, as documented here:

https://learn.microsoft.com/en-us/azure/azure-functions/dotnet-isolated-process-guide?tabs=hostbuilder%2Cwindows#application-insights

This should allow me to configure logging in the application only (e.g. appsettings.json), without host.json having affect on the Application Insights emitted logs.

However, when running the Function app in Azure, debug logs are not sent to Application Insights, unless I modify host.json with logging.logLevel.Function: "Debug". Doesn't this mean that application logs are actually sent over the Function host, i.e. Worker -> Functions host -> Application Insights instead of Worker -> Application Insights?

How can I sent application logs directly to Application Insights?

I have the required package references in .csproj:

<PackageReference Include="Microsoft.ApplicationInsights.WorkerService" Version="2.22.0" />
<PackageReference Include="Microsoft.Azure.Functions.Worker.ApplicationInsights" Version="2.0.0" />

Program.cs:

using System.Reflection;

using LEGO.IAM.AssetGovernance.Application;
using LEGO.IAM.AssetGovernance.Function;
using LEGO.IAM.AssetGovernance.Function.Extensions;
using LEGO.IAM.AssetGovernance.Infrastructure;

using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Builder;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;

var loggerFactory = LoggerFactory.Create(builder =>
{
    builder.AddConsole();

    var connectionString = Environment.GetEnvironmentVariable("APPLICATIONINSIGHTS_CONNECTION_STRING");

    if (!string.IsNullOrEmpty(connectionString))
    {
        builder.AddApplicationInsights(
            configureTelemetryConfiguration: (config) => config.ConnectionString = connectionString,
            configureApplicationInsightsLoggerOptions: (options) => { }
        );
    }
});
var logger = loggerFactory.CreateLogger("Startup");

logger.LogInformation("Starting application");

var builder = FunctionsApplication.CreateBuilder(args);
{
    // Configure application
    builder.ConfigureFunctionsWebApplication();

    var environment = builder.Environment.EnvironmentName;
    var basePath = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location) ?? Environment.CurrentDirectory;
    builder.Configuration.SetBasePath(basePath);

    builder.Configuration.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
        .AddJsonFile($"appsettings.{environment}.json", optional: true, reloadOnChange: true)
        .AddEnvironmentVariables();

    // Configure services
    builder.Services.AddApplicationInsightsTelemetryWorkerService();
    builder.Services.ConfigureFunctionsApplicationInsights();
    builder.Services.Configure<WorkerOptions>(options => { });

    builder.Services.AddFunction(builder.Configuration, environment);
    builder.Services.AddApplication(builder.Configuration, environment);
    builder.Services.AddInfrastructure(builder.Configuration, environment);

    // Configure logging
    builder.Logging.Services.Configure<LoggerFilterOptions>(options =>
    {
        var defaultRule = options.Rules.FirstOrDefault(rule => rule.ProviderName == "Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLoggerProvider");

        if (defaultRule is not null)
        {
            options.Rules.Remove(defaultRule);
        }
    });
}

var host = builder.Build();
{
    await host.SetupDatabase(logger);
}

await host.RunAsync();

Steps to reproduce

Create a Function app and configure it to sent application logs directly to Application Insights as documented here:

• https://learn.microsoft.com/en-us/azure/azure-functions/configure-monitoring?tabs=v2#custom-application-logs
• https://learn.microsoft.com/en-us/azure/azure-functions/dotnet-isolated-process-guide?tabs=hostbuilder%2Cwindows#application-insights

Configure the application with a logging filter for a category set to "Debug" and write debug logs with this category. Only logs with "Information" level or above are sent to Application Insights, as this is the default setting for host.json's logging.logLevel.Function setting

[mattchenderson]

We will need to reproduce the issue to understand it fully, but based on your description, it does seem like there is something unexpected happening here. We'll investigate.

Just as a note, one other way to check where the logs are coming from is to look at the sdkVersion field in the traces table for Application Insights. For example, in one of my apps, I can see azurefunctions-netiso: 2.0.0+d8b5fe998a8c92819b8ee41d2569d252541 (indicating the worker is sending it), as well as azurefunctions: 4.1037.0.23521 (indicating the host is sending it). A mix is expected, but we would also expect that debug logs written from the worker to flow through, given that you've removed the default filter rule.

[nlykkei]

Hi @mattchenderson,

Thanks for your response. I appreciate your support.

I just checked, and the logs are emitted with sdkVersion:
azurefunctions-netiso: 2.0.0+d8b5fe998a8c92819b8ee41d2569d252541.

So, they're originating from the worker (Microsoft.Azure.Functions.Worker.ApplicationInsights).

But for those to be emitted, I must set also set logging.logLevel.Function: "Debug" in host.json

[RohitRanjanMS]

Hi @nlykkei , can you confirm if you see duplicate logs? builder.AddConsole(); will write everything to console and this will be picked by host and sent to AppInsights. SDK Version would be different for these duplicate logs.

[nlykkei]

@RohitRanjanMS Sorry, I'm not sure what you want me to do.

The code you see in my example:

var loggerFactory = LoggerFactory.Create(builder =>
{
    builder.AddConsole();

    var connectionString = Environment.GetEnvironmentVariable("APPLICATIONINSIGHTS_CONNECTION_STRING");

    if (!string.IsNullOrEmpty(connectionString))
    {
        builder.AddApplicationInsights(
            configureTelemetryConfiguration: (config) => config.ConnectionString = connectionString,
            configureApplicationInsightsLoggerOptions: (options) => { }
        );
    }
});
var logger = loggerFactory.CreateLogger("Startup");

is only for creating a startup logger, as it wasn't otherwise possible to log errors in Program.cs. This logger is not used in the functions.

Isn't builder.AddConsole() called implicitly by the builder.ConfigureFunctionsWebApplication()? I can see console logs locally and filesystem logs in Azure without explicitly calling builder.AddConsole().

Currently, I don't see duplicate logs in Application Insights, they're all emitted by the Application Insights SDK

[RohitRanjanMS]

Hi @nlykkei , can you share what do you see for sdkVersion and Category attributes for these debug logs?

[nlykkei]

@RohitRanjanMS Thanks for getting back. How are you progressing on this? Have you created a simple Function app (.NET 9-isolated) to assert the bug?

Severity level: Verbose
CategoryName: XYZ.IAM.AssetGovernance.Application.Notifications.NotificationDispatcher
SDK version: azurefunctions-netiso: 2.0.0+d8b5fe998a8c92819b8ee41d2569d252541