need RGB and DEPTH image topic of the same size

[IacopoFornai]

hi @Serafadam ! i'm working with a turtlebot4 equipped with oak-d camera. i'm launching
ros2 launch turtlebot4_bringup oakd.launch.py
and i see that this launch file runs something from depthai-ros-driver.
changing config file https://github.com/turtlebot/turtlebot4_robot/blob/humble/turtlebot4_bringup/config/oakd_pro.yaml
i change
i_pipeline_type: RGB to RGBD to get depth image topic
i_height: 720 to 250 to match preview_image size 250x250 (rgb image)
i_width: 1280 to 250 to match preview_image size 250x250 (rgb image)
i need bgr and depth images to be the same size beacuse it is a requirement i need for running a visual odometry algorithm (rtabmap)
but when i change i_height and i_width to 250 and run the launch file, camera is not working anymore and i'm getting an error on terminal.
Is there a way to fix it or maybe another approach to have bgr and depth images of the same size? i tried all depthai-ros-driver and depthai-example and i found images of the same size but i had big delays, intead using
ros2 launch turtlebot4_bringup oakd.launch.py and changing i_pipeline_type: RGB to RGBD i'm getting fewer delays so it could be the best solution.

another requirement is that i need ros2 message of type Image and not CompressedImage
Thank you for your help!

[Serafadam]

Hi @IacopoFornai, resizing images is a little bit tricky since we have two separate resize mechanisms, this is explained on this and this documentation pages. Although I don't think you can set it to output 250x250 image without issues in the alignment, you can get close to that and resize further by the means of writing some additional ROS nodes to do that.

1. One can be achieved by using rgb._i_set_isp_scale which is enabled by default with scale (2/3), this resizes 1080P image to 720P. This preserves aspect ratio & FOV and you can this sheet to see what values can be obtained from different resolution and ISP num/den combination. Note that for alignment final width/height must be divisible by 16 (you can just look at the fields in the sheet and check where all fields are green)
2. Another way is to use video output which might crop the image depending on the sizes you set, this is controlled by i_width and i_height parameters when rgb.i_set_isp_scale and rgb.i_output_isp are set to false

You could also try setting stereo.i_width and stereo.i_height to manipulate stereo output size itself, although this might result in similar issues as with setting video size directly.

Regarding message types, by Image messages are being output by default, CompressedImage topic is provided by ROS ImageTransport plugin mechanism

[IacopoFornai]

Hi @Serafadam thanks for your help again, i'm still trying to find a good configuration to run rtabmap, now i'm working with
images of size 720x1280. running rtabmap from depthai_ros_driver i get these 2 warnings:

[component_container-1] [WARN] [1737645770.865692379] [rtabmap]: We received odometry message, but we cannot get the corresponding TF odom->oak at data stamp 1737645770.358590s (odom msg stamp is 1737645770.325299s). Make sure TF of odometry is also published to get more accurate pose estimation. This warning is only printed once.

[component_container-1] [WARN] [1737645771.551586273] [rgbd_odometry]: The time difference between rgb and depth frames is high (diff=0.033313s, rgb=1737645770.525279s, depth=1737645770.491966s). You may want to set approx_sync_max_interval lower than 0.02s to reject spurious bad synchronizations or use approx_sync=false if streams have all the exact same timestamp.

can you give me some hints for solving these problems? Regarding the second warnings i used to set
"approx_sync_max_interval:=0.02" but in this way i filter too many images, is there a way to have a perfect syncrhonization of the timestamps related to rbg images and depth images?

[Serafadam]

Hi @IacopoFornai, you can refer to this part of documentation w.r.t syncing

[IacopoFornai]

hi @Serafadam, i'm trying to launch
ros2 launch depthai_ros_driver rtabmap.launch.py
i just added in the yaml file the parameters for syncing and i changed the remappings in order to have the right input for a slam algorithm. But i have this error:

[component_container-1] terminate called after throwing an instance of 'std::system_error'
[component_container-1] what(): Device already closed or disconnected: Input/output error
[ERROR] [component_container-1]: process has died [pid 2661, exit code -6, cmd '/opt/ros/humble/lib/rclcpp_components/component_container --ros-args --log-level info --ros-args -r __node:=turtlebot4_0_container -r __ns:=/'].

i tried to launch just camera.launch.py and i have the same error, so i think the problem is there, i attach my terminal lines

issues_rtabmap.txt

do you have any hints on how to solve this problem?

[IacopoFornai]

small follow up: it seems like remappings the topics in the launch file brings problem, i explain what i've done.
i changed the remapping part of the launchfile, in order to have a matching between camera output topics and the topic i need for a SLAM algorithm.

this is the default setup:

i remapped in this way:

remappings = [
("rgb/image", name + "/turtlebot4_0/color/preview/image"),
("rgb/camera_info", name + "/turtlebot4_0/color/camera_info"),
("depth/image", name + "/turtlebot4_0/stereo/depth"),
]

if i make this change, then i have the problem i wrote in the issue above, if i leave the launch file as it is, rtabmap starts without problem

[Serafadam]

Hi, could you provide logs after enabling DEPTHAI_DEBUG=1?