Integrating SAM with Feature Interaction for Remote Sensing Change Detection

    

Introduction

The repository is the code implementation of the paper Integrating SAM with Feature Interaction for Remote Sensing Change Detection, based on MMSegmentation and TTP projects.

The current branch has been tested under PyTorch 2.x and CUDA 12.1, supports Python 3.7+, and is compatible with most CUDA versions.

Installation

Dependencies

• Linux or Windows
• Python 3.7+, recommended 3.10
• PyTorch 2.0 or higher, recommended 2.1
• CUDA 11.7 or higher, recommended 12.1
• MMCV 2.0 or higher, recommended 2.1

Environment Installation

We recommend using Miniconda for installation. The following command will create a virtual environment named sfcd and install PyTorch and MMCV.

Note: If you have experience with PyTorch and have already installed it, you can skip to the next section. Otherwise, you can follow these steps to prepare.

Step 0: Install Miniconda.

Step 1: Create a virtual environment named sfcd and activate it.

conda create -n sfcd python=3.10 -y
conda activate sfcd

Step 2: Install PyTorch2.1.x.

Linux/Windows:

pip install torch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 --index-url https://download.pytorch.org/whl/cu121

Or

conda install pytorch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 pytorch-cuda=12.1 -c pytorch -c nvidia

Step 3: Install MMCV2.1.x.

pip install -U openmim
mim install mmcv==2.1.0

Step 4: Install other dependencies.

pip install -U wandb einops importlib peft==0.8.2 scipy ftfy prettytable torchmetrics==1.3.1 transformers==4.38.1

Install SFCD

Download or clone the SFCD-Net repository.

git clone [email protected]:zhangda1018/SFCD-Net.git
cd SFCD

Dataset Preparation

Levir-CD Change Detection Dataset

Dataset Download

• Image and label download address: Levir-CD.

Organization Method

You can also choose other sources to download the data, but you need to organize the dataset in the following format:

${DATASET_ROOT} # Dataset root directory, for example: /home/username/data/levir-cd
├── train
│   ├── A
│   ├── B
│   └── label
├── val
│   ├── A
│   ├── B
│   └── label
└── test
    ├── A
    ├── B
    └── label

Note: In the project folder, we provide a folder named data, which contains an example of the organization method of the above dataset.

Other Datasets

If you want to use other datasets, you can refer to MMSegmentation documentation to prepare the datasets.

Model Training

SFCD Model

Config File and Main Parameter Parsing

We provide the configuration files of the SFCD model used in the paper, which can be found in the configs/SFCD folder. The Config file is completely consistent with the API interface and usage of MMSegmentation. Below we provide an analysis of some of the main parameters. If you want to know more about the meaning of the parameters, you can refer to MMSegmentation documentation.

Single Card Training

python tools/train.py configs/SFCD/xxx.py  # xxx.py is the configuration file you want to use

Model Testing

Single Card Testing:

python tools/test.py configs/SFCD/xxx.py ${CHECKPOINT_FILE}  # xxx.py is the configuration file you want to use, CHECKPOINT_FILE is the checkpoint file you want to use

Note: If you need to get the visualization results, you can uncomment default_hooks-visualization in the Config file.

Image Prediction

Single Image Prediction:

python demo/image_demo_with_cdinferencer.py ${IMAGE_FILE1} ${IMAGE_FILE2} configs/SFCD/image_infer.py --checkpoint ${CHECKPOINT_FILE} --out-dir ${OUTPUT_DIR}  # IMAGE_FILE is the image file you want to predict, xxx.py is the configuration file, CHECKPOINT_FILE is the checkpoint file you want to use, OUTPUT_DIR is the output path of the prediction result

Acknowledgements

The repository is the code implementation of the paper Integrating SAM with Feature Interaction for Remote Sensing Change Detection, based on MMSegmentation and TTP projects.

Citation

If you use the code or performance benchmarks of this project in your research, please refer to the following bibtex to cite SFCD.

@article{zhang2024integrating,
  title={Integrating SAM with Feature Interaction for Remote Sensing Change Detection},
  author={Zhang, Da and Wang, Feiyu and Ning, Lichen and Zhao, Zhiyuan and Gao, Junyu and Li, Xuelong},
  journal={IEEE Transactions on Geoscience and Remote Sensing},
  year={2024},
  publisher={IEEE}
}

License

The repository is licensed under the Apache 2.0 license.