GeoAI is an plug_in under development for QGIS whose objective is to allow to exploit with versatility the capabilities of the Artifical Intelligence (AI) models in geospatial data processing, starting with the Segment Anything image segmentation model (SAM) ) developed by META).
Segment Anything is a pre-trained artificial intelligence model that allows to generate masks on recognized objects in images for highlighting or extraction.
(Source pagina del proyecto SAM, github SAM project).
Segment Anything produces high-quality object masks from inputs such as dots or boxes, and can be used to generate masks for all objects in an image.
It has been trained on a dataset of 11 million images and 1.1 billion masks, and offers great performance on a wide variety of segmentation tasks
Source SAM project page, github SAM project).
When using the text and/or code add to the quote:
If you want to make a contribution to this development you can send it to PayPal account: [email protected]
Write in the note your name and "Contribution to the development of the GeoAI plugin"
The plugin is licensed under: GNU General Public License v3.0
- Version 1.1.0 of the GeoAI plugin implements a lightweight (mobile) version of the SAM model. Detailed description in the following article
- Article with details on the implementation of the lightweight SAM model
- Article outlining strategies and suggestions for improving Geospatial image processing
Although, GeoAI is a plugin under development, in this first version I have bet on four (04) fundamental aspects:
- Functionality GeoAIIt can work on 8-byte RGB georeferenced images (format accepted by the model) and also on uniband and multiband images such as satellite and drone images, which can have different formats. For this purpose the plugin performs a transformation process.
- Versatility The user interfaces have been designed for ease of use while exploiting the capabilities of the model, providing various configurations which are made available when appropriate.
- Compatibility
- Easy installation and portability The plug-in file is lightweight, weighing less than 2 megabytes, and its installation requirements are the minimum required to use the SAM model, such as: install PyTorch, download the pre-trained model files (checkpoints).
After configuring the images the user can segment the images using two modules:
With the implementation of the lightweight SAM model developed by ETH VIS Group, it is now possible to run segmentation with less resource consumption and very fast.
Source, repository and further details at: [Github ETH VIS Group](https://github.com/SysCV), [Paper: Ke, Lei and Ye, et. al, 2023 Segment Anything in High Quality, NeurIPS.](https://arxiv.org/abs/2306.01567).
1.- Segmentation of the whole image
It allows to segment the whole image, the user can do it using the default settings or by altering the advanced parameters. CAUTION modifying the parameters may alter the results and execution times.
2.- Interactive segmentation
This is the most complete and intuitive tool. It displays a wizard through which it is possible to segment objects in the image by selecting them on the screen using points and/or areas (drawing a rectangle). It also has the option to obtain a single segment per selection, multiple segments or the one with the largest area. On the other hand, the user can store the segments in a new layer or an existing one, even create fields and save an attribute. It is an advanced digitizing tool.
GeoAI has been tested on Windows 10 and 11 operating systems, QGIS versions 3.10, 3.16, 3.22 and the latest LTR 3.28.
You can download the plugin zip file from this repository and then install it in QGIS with the Install via Zip option..
However, there are aspects that could be improved, highlighting:
- Complete the documentation
- Translating the interface and documentation into English and other languages
- Explore in implementing the optimization options offered by the SAM model, Python and PyQGIS
- Add more functionalities
- Requirements
- Installation
- Quick Tutorial
- Tutorial
GeoAI is designed to minimize installation requirements. To use the plugin you only need to meet two requirements:
- Install the appropriate version of the PyTorch library.
- Download and place in an accessible directory the Check Points of the SAM model
- Install TIMM