Pulling refs/heads/main into main

apm/profiling/get-data-in-profiling.rst

@@ -86,6 +86,10 @@ AlwaysOn Profiling requires APM tracing data to correlate stack traces to your a
    * - .NET
      - SignalFx Instrumentation for .NET version 1.0.0 or higher
      - :ref:`instrument-dotnet-applications`
+   * - Python (in beta)
+     - Splunk Distribution of OpenTelemetry Python version 1.15 or higher
+     - * :ref:`instrument-python-applications`
+       * :ref:`profiling-configuration-python`
 
 .. note:: See :ref:`apm-data-retention` for information on profiling data retention.
 
@@ -199,6 +203,52 @@ To activate AlwaysOn Profiling, follow the steps for the appropriate programming
 
       For more configuration options, including setting a separate endpoint for profiling data, see :ref:`profiling-configuration-dotnet`.
 
+   .. group-tab:: Python
+
+      .. note::
+         AlwaysOn Profiling for Python is in beta development. This feature is provided by Splunk to you "as is" without any warranties, maintenance and support, or service-level commitments. Use of this feature is subject to the :new-page:`Splunk General Terms <https://www.splunk.com/en_us/legal/splunk-general-terms.html>`.
+
+      :strong:`Requirements`
+
+      AlwaysOn Profiling requires Python 3.7.2 or higher.
+
+      :strong:`Instrumentation`
+
+      Activate the profiler by setting the ``SPLUNK_PROFILER_ENABLED`` environment variable to ``true`` or call the ``start_profiling`` function in your application code. 
+
+      Check the OTLP endpoint in the ``SPLUNK_PROFILER_LOGS_ENDPOINT`` environment variable:
+
+         - For non-Kubernetes environments, make sure that the ``SPLUNK_PROFILER_LOGS_ENDPOINT`` environment variable points to \http://localhost:4317.
+         - For Kubernetes deployments, the OTLP endpoint has to point to \http://$(K8S_NODE_IP):4317 where the ``K8S_NODE_IP`` is fetched from the Kubernetes downstream API by setting the environment configuration on the Kubernetes pod running the application. For example:
+
+            .. code-block:: yaml
+
+               env:  
+               - name: K8S_NODE_IP
+                 valueFrom:
+                   fieldRef:
+                     apiVersion: v1
+                     fieldPath: status.hostIP
+
+      The following example shows how to activate the profiler from your application's code:
+
+      .. code-block:: python
+
+         from splunk_otel.profiling import start_profiling
+
+         # Activates CPU profiling
+         # All arguments are optional
+         start_profiling(
+            service_name='my-python-service', 
+            resource_attributes={
+               'service.version': '3.1'
+               'deployment.environment': 'production', 
+            }
+            endpoint='http://localhost:4317'
+         ) 
+
+      For more configuration options, see :ref:`profiling-configuration-python`.
+
 .. _profiling-check-data-coming-in:
 
 Check that Splunk Observability Cloud is receiving profiling data

apm/profiling/intro-profiling.rst

@@ -85,6 +85,10 @@ The following programming languages have instrumentation available:
    * - .NET
      - SignalFx Instrumentation for .NET version 1.0.0 or higher
      - :ref:`instrument-dotnet-applications`
+   * - Python (in beta)
+     - Splunk Distribution of OpenTelemetry Python version 1.15 or higher
+     - * :ref:`instrument-python-applications`
+       * :ref:`profiling-configuration-python` 
 
 
 .. raw:: html

gdi/get-data-in/application/java/instrumentation/instrument-java-application.rst

@@ -112,7 +112,7 @@ If you need to add custom attributes to spans or want to manually generate spans
 .. _enable_profiling_java:
 
 Activate AlwaysOn Profiling
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+-------------------------------------
 
 To activate AlwaysOn Profiling, use the following system property argument. You can also use the ``SPLUNK_PROFILER_ENABLED`` environment variable. For more information, see :ref:`profiling-intro`.
 
@@ -135,7 +135,7 @@ See :ref:`get-data-in-profiling` for more information. For more settings, see :r
 .. _enable_automatic_metric_collection:
 
 Activate metrics collection
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+---------------------------------------
 
 To activate automatic metric collection, activate the metrics feature using a system property argument. You can also use the ``SPLUNK_METRICS_ENABLED`` environment variable.
 
@@ -153,7 +153,7 @@ If your metrics endpoint is different than the default value, set the ``SPLUNK_M
 .. _ignore_endpoints_java:
 
 Ignore specific endpoints
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+------------------------------------------
 
 By default, the Java agent collects traces from all the endpoints of your application. To ignore specific endpoints, use the ``rules`` sampler and define ``drop`` rules.
 

gdi/get-data-in/application/nodejs/instrumentation/instrument-nodejs-application.rst

@@ -107,7 +107,7 @@ If no data appears in APM, see :ref:`common-nodejs-troubleshooting`.
 .. _enable_profiling_nodejs:
 
 Activate AlwaysOn Profiling
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+--------------------------------------------
 
 To activate AlwaysOn Profiling, set the ``SPLUNK_PROFILER_ENABLED`` environment variable to ``true``.
 
@@ -130,7 +130,7 @@ See :ref:`get-data-in-profiling` for more information. For more settings, see :r
 .. _enable_automatic_metric_collection_nodejs:
 
 Activate metrics collection
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+------------------------------------------------
 
 To activate automatic runtime metric collection, activate the metrics feature using the ``SPLUNK_METRICS_ENABLED`` environment variable. See :ref:`metrics-configuration-nodejs` for more information.
 
@@ -216,7 +216,7 @@ For a list of supported instrumentations, see :new-page:`https://github.com/open
 .. _kubernetes_nodejs_agent:
 
 Deploy the Node.js distribution in Kubernetes
------------------------------------------------------
+=======================================================
 
 To deploy the Splunk Distribution of OpenTelemetry JS in Kubernetes, configure the Kubernetes Downward API to expose environment variables to Kubernetes resources.
 
@@ -250,7 +250,7 @@ The following example shows how to update a deployment to expose environment var
 .. _export-directly-to-olly-cloud-nodejs:
 
 Send data directly to Splunk Observability Cloud
------------------------------------------------------
+=============================================================
 
 By default, all telemetry is sent to the local instance of the Splunk Distribution of OpenTelemetry Collector.
 

gdi/get-data-in/application/python/configuration/advanced-python-otel-configuration.rst

@@ -115,6 +115,28 @@ For backward compatibility with the SignalFx Python Tracing Library, use the b3m
 
       $env:OTEL_PROPAGATORS=b3multi
 
+.. _profiling-configuration-python:
+
+Python settings for AlwaysOn Profiling
+====================================================
+
+The following settings control the AlwaysOn Profiling feature for the Python agent:
+
+.. list-table::
+   :header-rows: 1
+   :widths: 40, 60
+
+   * - Environment variable
+     - Description
+   * - ``SPLUNK_PROFILER_ENABLED``
+     - Activates AlwaysOn Profiling. The default value is ``false``. 
+   * - ``SPLUNK_PROFILER_LOGS_ENDPOINT``
+     - The collector endpoint for profiler logs. By default, it takes the value of ``http://localhost:4317``.
+   * - ``SPLUNK_PROFILER_CALL_STACK_INTERVAL``
+     - The frequency of call stack sampling, in milliseconds. The default value is ``1000``.
+   * - ``SPLUNK_PROFILER_INCLUDE_INTERNAL_STACKS``
+     - Determines whether to include stack traces from internal profiler threads. The default value is ``false``.
+
 .. _server-trace-information-python:
 
 Server trace information

gdi/get-data-in/application/python/instrumentation/instrument-python-application.rst

@@ -123,6 +123,35 @@ Application metrics are collected by default. See :ref:`python-otel-metrics` for
 
 If no data appears in APM, see :ref:`common-python-troubleshooting`.
 
+.. _enable-profiling-python:
+
+Activate AlwaysOn Profiling
+------------------------------------------------
+
+.. note::
+   AlwaysOn Profiling for Python is in beta development. This feature is provided by Splunk to you "as is" without any warranties, maintenance and support, or service-level commitments. Use of this feature is subject to the :new-page:`Splunk General Terms <https://www.splunk.com/en_us/legal/splunk-general-terms.html>`.
+
+To activate AlwaysOn Profiling, set the ``SPLUNK_PROFILER_ENABLED`` environment variable to ``true`` or call the ``start_profiling`` function in your application code.
+
+The following example shows how to activate the profiler from your application code:
+
+.. code-block:: python
+
+         from splunk_otel.profiling import start_profiling
+
+         # Activates CPU profiling
+         # All arguments are optional
+         start_profiling(
+            service_name='my-python-service', 
+            resource_attributes={
+               'service.version': '3.1'
+               'deployment.environment': 'production', 
+            }
+            endpoint='http://localhost:4317'
+         ) 
+
+See :ref:`get-data-in-profiling` for more information. For additional settings, see :ref:`profiling-configuration-python`.
+
 .. _configure-python-instrumentation:
 
 Configure the Python agent
@@ -135,7 +164,7 @@ For advanced configuration of the Python agent, like changing trace propagation
 .. _kubernetes_python_agent:
 
 Deploy the Python agent in Kubernetes
-----------------------------------------------------
+===================================================
 
 To deploy the Python agent in Kubernetes, configure the Kubernetes Downward API to expose environment variables to Kubernetes resources.
 
@@ -168,7 +197,7 @@ The following example shows how to update a deployment to expose environment var
 .. _export-directly-to-olly-cloud-python:
 
 Send data directly to Splunk Observability Cloud
-----------------------------------------------------
+==========================================================
 
 By default, the agent sends all telemetry to the local instance of the Splunk Distribution of OpenTelemetry Collector.
 
@@ -203,6 +232,6 @@ The realm name appears in the :guilabel:`Organizations` section.
 .. _instrument_aws_python_functions:
 
 Instrument Lambda functions
-----------------------------------------------------
+=============================================
 
 You can instrument AWS Lambda functions using the Splunk OpenTelemetry Lambda Layer. See :ref:`instrument-aws-lambda-functions` for more information.

gdi/opentelemetry/kubernetes-config-advanced.rst

@@ -45,6 +45,8 @@ This custom configuration is merged into the default agent configuration.
 
 .. caution:: After merging the files you need to fully redefine parts of the configuration, for example ``service``, ``pipelines``, ``logs``, and ``processors``.
 
+.. _otel-kubernetes-config-advanced-control-plane:
+
 Override a control plane configuration
 ==============================================================