Mask resolution on big imagery

[jfulponi]

Hi, pleasure to write on this wonderful repo.

I was wondering if it is possible to tune the mask resolution. I cannot say this is an issue, but when I try to sam.predict() (specially with text prompts) on small images, the mask is done pretty well. In the other hand, when I try to predict prompts on the same area but with a bigger image (expanding the bbox but also adjusting the resolution to balance) the masking process gets weird and inaccurate.

Is there a way to adjust the masking resolution in order to have more accurate results on big areas? What I'm trying to avoid is to generate a loop with sub-bboxes.

This is a comparison of two examples:

Small bbox and great result.
Prompt: "grass", box_threshold=.4, text_threshold=.5

Larger bbox but higher resolution and incorrect result:
Prompt: "grass", box_threshold=.4, text_threshold=.5

Thanks in advance.

EDIT: I tried to tune the thresholds but I didn't get better results.

[giswqs]

Can you share the bounding box coordinates for debugging? Use m.bounds to get the coordinates

[jfulponi]

Of course! I'm sharing the actual part of the code.

small_bbox = [-58.374116,-34.610023,-58.366091,-34.604398]

s_image = "small_bbox.tif"

tms_to_geotiff(output=s_image, bbox=small_bbox, zoom=19, source="Satellite", overwrite=True) 

large_bbox = [-58.382421,-34.614208,-58.364310,-34.602199]

l_image = "large_bbox.tif"

tms_to_geotiff(output=l_image, bbox=large_bbox, zoom=19, source="Satellite", overwrite=True)

Thanks.

[LucasOsco]

Hello @jfulponi

We noticed that in our tests with the text-prompt approach too. We solved it by breaking the image into patches, processing them, and then returning it to its original size. The trick part is that we need to be careful with the patch size, mostly because SAM is a transformer and benefits from context information. Regardless, we hope to achieve an optimal point and soon I'll be working with @giswqs to implement this approach for bigger imagery.

[jfulponi]

Hi @LucasOsco!

Yeah, I've been trying to predict some "grass" prompts in contiguous areas in a similar approach, and I've noticed discrete jumps between masked zones because of what you say. Not a big problem for me, but definitely splitting the image, predicting on each part and joining the masks or vectors is a messy way to do it. I haven't tried this approach without using segment-geospatial, but I'm pretty sure it's a SAM problem.

[jfulponi]

By the way, I'd be more than happy to help you out with this if you need a hand.