[HW Accel Support]: TensorRT improper library search: libcuda.so vs libcuda.so.1

[JoshuaPK]

Describe the problem you are having

When first starting up the Frigate container, the yolov7-320.trt model fails to build. It stops with an error: Could not load library libcudnn_cnn_infer.so.8. Error: libcuda.so: cannot open shared object file: No such file or directory This is due to the fact that there is no libcuda.so in the image. The provided Dockerfile fixes this issue (note the 'entrypoint' entry). I am not sure if this is a problem with the NVidia base image, the way the frigate image is built, or the fact I'm using podman and podman-compose instead of Docker.

Version

frigate:stable-tensorrt

Frigate config file

N/A at this stage

docker-compose file or Docker CLI command

version: "3.9"
services:
  frigate:
    container_name: frigate
    #privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable-tensorrt
    shm_size: "4gb" # update for your cameras based on calculation above
    devices:
      - nvidia.com/gpu=all
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/vcs/frigate/config:/config
      - /srv/vcs/frigate/media:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "password"
    entrypoint: ['/bin/bash', '-c', 'ln /usr/lib64/libcuda.so.1 /usr/lib64/libcuda.so; ldconfig; /init']

Relevant log output

[frigate] | Generating yolov7-320.trt. This may take a few minutes.
[frigate] | 
/usr/local/src/tensorrt_demos/yolo/onnx_to_tensorrt.py:147: DeprecationWarning: Use network created with NetworkDefinitionCreationFlag::EXPLICIT_BATCH flag instead.
  builder.max_batch_size = MAX_BATCH_SIZE
/usr/local/src/tensorrt_demos/yolo/onnx_to_tensorrt.py:149: DeprecationWarning: Use set_memory_pool_limit instead.
  config.max_workspace_size = 1 << 30
/usr/local/src/tensorrt_demos/yolo/onnx_to_tensorrt.py:172: DeprecationWarning: Use build_serialized_network instead.
  engine = builder.build_engine(network, config)
Could not load library libcudnn_cnn_infer.so.8. Error: libcuda.so: cannot open shared object file: No such file or directory
/etc/s6-overlay/s6-rc.d/trt-model-prepare/run: line 82:   175 Aborted                 (core dumped) $cmd > /tmp/onnx_to_tensorrt.log

FFprobe output from your camera

N/A at this point

Operating system

Other Linux

Install method

Docker Compose

Network connection

Wired

Camera make and model

N/A at this point

Any other information that may be helpful

The Cuda libraries on the host OS (Alma Linux 9) are 555.42.02 and the Cuda Tookit version is 12.5.40.

[wernight]

I get the same running using podman-compose with a very similar docker-compose file (just without the entrypoint).

Usually, for other projects, I can use CUDA using:

    devices:
      - nvidia.com/gpu=all

I do have the CUDA toolkit but I didn't find a find like libcudnn_cnn_infer* on my host.

What confuses me is the Docker image to use.

• frigate:stable is mentioned on https://docs.frigate.video/configuration/hardware_acceleration#configuring-nvidia-gpus-in-docker for nVidia CUDA (non-Jetson). Frigate runs with that, but I see an error: TensorRT libraries not found, tensorrt detector not present.
• frigate:stable-tensorrt is mentioned on https://docs.frigate.video/frigate/installation/#docker for Amd64 running an nVidia GPU, and also TensorRT is mentioned on https://docs.frigate.video/frigate/hardware#tensorrt---nvidia-gpu. However it fails with the error like above regarding libcudnn_cnn_infer.so.8.

[wernight]

Same error when using podman run --rm --gpus all ...:

Generating yolov7-320.trt. This may take a few minutes.
...
Could not load library libcudnn_cnn_infer.so.8. Error: libcuda.so: cannot open shared object file: No such file or directory

/usr/lib/libcuda.so exists though outside of the container.

I tried manually mounting that file: -v /usr/lib/libcuda.so.1:/usr/lib/libcuda.so:ro which does seem to work.

[wernight]

Seem that the fix is mounting libcuda.so like:

    volumes:                                                                                                                                                                                                       
      - /usr/lib/libcuda.so:/usr/lib/libcuda.so:ro
      - ...

[martinlindhe]

Seem that the fix is mounting libcuda.so like:

    volumes:                                                                                                                                                                                                       
      - /usr/lib/libcuda.so:/usr/lib/libcuda.so:ro
      - ...

I too had success with this solution (libcudnn_cnn_infer.so.8 not found). Podman 5.2.5