Inaccurate Camera Positioning with UAV Data

[StonerLing]

Hi development team,

I've been testing your glomap software with UAV imagery, focusing on vegetated scenes. I've noticed that the software isn't performing as well as I'd hoped with these types of images. The original image layout should be nearly coplanar, and colmap has been able to achieve the slightly better results. However, with glomap, the final calculated positions of the images form a curved arc surface, which is not the expected outcome. I've tested this with several similar datasets, and they all seem to produce similar incorrect results.

These are results.

dataset sample

glomap result

colmap result

colmap result(with gps constrained)

The results above are from tests I conducted using a public dataset with scenes similar to my own data. The dataset can be accessed at this link: https://cloud.pix4d.com/site/48774/dataset/390556/files/inputs?shareToken=82c66f66-6d88-41a8-9dfa-98c70ce934bf. In addition to the abnormal camera positioning in the reconstruction, there's also a peculiar issue with point cloud coloring failure, which didn't occur in my own data tests. However, a similar issue with the curvature of the positions was present. I'm not sure if this is related to glomap's use of ray angle as an error term during the global positioning process instead of reprojection error. Or rather, for UAV data, should we adopt a GPS-constrained approach for SfM? Have you considered integrating this feature into the glomap library?

Additionally, the cmake install command does not seem to correctly install the glomap.lib static library to the specified directory.

[ichsan2895]

In my case, Glomap detected 238/374 images when Colmap only detected 120/374 images

Left = Colmap 3.11dev0 commit c238aec0
Right = Glomap commit 98fa29a

Additional info:

MXLinux KDE 19.4
Ceres-solver 2.1.0 (manually build with CUDA 11.4)
Cmake 3.28.4

For sake of experiment, I resize the dataset to 1/8 * original size for getting faster result, then I move it to "input" folder

Command used:

mkdir distorted
mkdir input


colmap feature_extractor \
    --image_path    input/ \
    --database_path distorted/database.db \
    --ImageReader.single_camera_per_folder 1

colmap exhaustive_matcher \
    --database_path distorted/database.db 

mkdir distorted/colmap
mkdir distorted/glomap

colmap mapper --database_path distorted/database.db \
    --image_path input/ \
    --output_path distorted/colmap

glomap mapper --database_path distorted/database.db \
    --image_path input/ \
    --output_path distorted/glomap

colmap model_converter \
  --input_path distorted/colmap/0 \
  --output_path distorted/colmap/0 \
  --output_type TXT

colmap model_converter \
  --input_path distorted/glomap/0 \
  --output_path distorted/glomap/0 \
  --output_type TXT

colmap image_undistorter \
  --image_path input \
  --input_path distorted/colmap/0 \
  --output_path distorted/colmap

colmap image_undistorter \
  --image_path input \
  --input_path distorted/glomap/0 \
  --output_path distorted/glomap

colmap model_converter \
  --input_path distorted/colmap/sparse \
  --output_path distorted/colmap/sparse \
  --output_type TXT

colmap model_converter \
  --input_path distorted/glomap/sparse \
  --output_path distorted/glomap/sparse \
  --output_type TXT

[ichsan2895]

In second attempt, Glomap detected 374/374 images when Colmap only detected 340/374 images

I resize the dataset to 1/4 * original size instead of 1/8, then I move it to "input" folder. I use spatial_matcher instead of exhausive_matcher

Left = Colmap 3.11dev0 commit c238aec0
Right = Glomap commit 98fa29a

Additional info:

MXLinux KDE 19.4
Ceres-solver 2.1.0 (manually build with CUDA 11.4)
Cmake 3.28.4

Command used:

mkdir distorted
mkdir input


colmap feature_extractor \
    --image_path    input/ \
    --database_path distorted/database.db \
    --ImageReader.single_camera_per_folder 1

colmap spatial_matcher \
    --database_path distorted/database.db 

mkdir distorted/colmap
mkdir distorted/glomap

colmap mapper --database_path distorted/database.db \
    --image_path input/ \
    --output_path distorted/colmap

glomap mapper --database_path distorted/database.db \
    --image_path input/ \
    --output_path distorted/glomap

colmap model_converter \
  --input_path distorted/colmap/0 \
  --output_path distorted/colmap/0 \
  --output_type TXT

colmap model_converter \
  --input_path distorted/glomap/0 \
  --output_path distorted/glomap/0 \
  --output_type TXT

colmap image_undistorter \
  --image_path input \
  --input_path distorted/colmap/0 \
  --output_path distorted/colmap

colmap image_undistorter \
  --image_path input \
  --input_path distorted/glomap/0 \
  --output_path distorted/glomap

colmap model_converter \
  --input_path distorted/colmap/sparse \
  --output_path distorted/colmap/sparse \
  --output_type TXT

colmap model_converter \
  --input_path distorted/glomap/sparse \
  --output_path distorted/glomap/sparse \
  --output_type TXT

[StonerLing]

@ichsan2895 Thanks for your comments and experiments!

Regarding the test results that I obtained when I initially opened the issue, I used the COLMAP GUI to set up some parameters:

• All steps use the CPU instead of the GPU, with three threads employed. (I believe these parameters do not affect the results.)
• Camera intrinsics are shared.
• The spatial matcher is used with a 150m constraint.
• The principal point is adjusted.

After considering your comments, I tried using the command interface you suggested. I was glad to see that GLOMAP produced the expected results :)

GLOMAP result:

However, I have a few more questions; I might need to open another issue in COLMAP 3.11.

• I'm not getting the desired results when using the GUI interface(3.11) for the reconstruction process, even though I don't believe I've changed many parameters. Why is that?
• The model generated by COLMAP seems to stick together with the camera positions. Is this related to reconstruction normalization?
• When I attempted to import the sparse point cloud models generated by COLMAP or GLOMAP into the GUI interface of version 3.11 for display, I encountered an error: Check failed: images_.emplace(image_id, std::move(image)).second. As a result, I switched to version 3.10 for visualization.

COLMAP result:

[ichsan2895]

All steps use the CPU instead of the GPU, with three threads employed. (I believe these parameters do not affect the results.)

In my test, feature_extractor and exhausive_matcher uses GPU. colmap mapper & glomap mapper uses CPU.

I'm not getting the desired results when using the GUI interface(3.11) for the reconstruction process, even though I don't believe I've changed many parameters. Why is that?

I never use GUI interface for reconstruction. But, colmap 3.11dev0 still got worse result than glomap.

What I expect : colmap & glomap gives similar good result, but glomap is a bit faster.

When I attempted to import the sparse point cloud models generated by COLMAP or GLOMAP into the GUI interface of version 3.11 for display, I encountered an error: Check failed: images_.emplace(image_id, std::move(image)).second.

It didn't happen to me. Maybe something wrong when it compiled? I use colmap 3.11dev0 commit c238aec0 and it can import the sparse data for display.

[StonerLing]

I used git pull to update my colmap from 3.11.0.dev0 commit a7a0db7d to the latest 3.11.0.dev0 commit 63b2cc00, and now I can import sparse data for display correctly. However, there is still this issue (I re-ran the reconstruction with the new version):

• The model generated by colmap seems to stick together with the camera positions. Is this related to reconstruction normalization

This really seems like some kind of normalization has been applied to the camera's pose.

Of course, regardless, glomap indeed shows better performance than colmap.

[ichsan2895]

Personally from my experience, colmap 3.8 gives best result despite it more slower. But, somehow, the glomap mapper from colmap 3.8 will gives bad result. See my experiment in here: #69

I think It need to be inspected with more various dataset.

[ahojnnes]

The type of data you are dealing with poses challenges for SfM in general, please refer for more details to: "Critical Configurations For Radial Distortion Self-Calibration", http://ccwu.me/file/radial.pdf

Apart from that, colmap is likely still the better choice for data with significant unknown distortion. glomap can deal quite robustly with unknown focal length but significant unknown distortion is still often a challenge. In glomap, you may want to try running additional rounds of retriangulation and bundle adjustment. It may (or may not) help with the issue.

[StonerLing]

@ahojnnes Thanks for your comment!

I will further study and understand the effects of radial distortion based on the paper you provided. In addition to performing additional rounds of triangulation and bundle adjustment, I might attempt to mitigate the bending of camera positions using GPS constraints.

[ahojnnes]

Colmap has a new pose prior mapper that can take advantage of GPS priors. GLOMAP currently doesn't have a built in way to take advantage of them but could be relatively easily modified to do so.