From d8dba8685aa6dfb0ce9725280735d104e6f0e639 Mon Sep 17 00:00:00 2001 From: Dipti Sengupta Date: Sun, 9 Oct 2022 09:56:58 +0000 Subject: [PATCH 1/3] added empty Readme --- Zero-shot-Instance-Segmentation/README.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 Zero-shot-Instance-Segmentation/README.md diff --git a/Zero-shot-Instance-Segmentation/README.md b/Zero-shot-Instance-Segmentation/README.md new file mode 100644 index 0000000..e69de29 From d07f857abf7eee28945069b05bf8c7bd5ec54f34 Mon Sep 17 00:00:00 2001 From: Dipti Sengupta Date: Sun, 9 Oct 2022 12:07:08 +0200 Subject: [PATCH 2/3] Update annotations and results --- Zero-shot-Instance-Segmentation/README.md | 154 ++++++++++++++++++++++ 1 file changed, 154 insertions(+) diff --git a/Zero-shot-Instance-Segmentation/README.md b/Zero-shot-Instance-Segmentation/README.md index e69de29..fb7baf8 100644 --- a/Zero-shot-Instance-Segmentation/README.md +++ b/Zero-shot-Instance-Segmentation/README.md @@ -0,0 +1,154 @@ + +# Code for CVPR2021 paper + +# **Zero-shot Instance Segmentation** + +## Code requirements ++ python: python3.7 ++ nvidia GPU ++ pytorch1.1.0 ++ GCC >=5.4 ++ NCCL 2 ++ the other python libs in requirement.txt + +## Install + +``` +conda create -n zsi python=3.7 -y +conda activate zsi + +conda install pytorch=1.1.0 torchvision=0.3.0 cudatoolkit=10.0 -c pytorch + +pip install cython && pip --no-cache-dir install -r requirements.txt + +python setup.py develop +``` + +## Dataset prepare + + +- Download the train and test annotations files for zsi from [annotations](https://dagshub.com/f2010126/Zero-shot-Instance-Segmentation/src/main/data/coco/annotations), put all json label file to + ``` + data/coco/annotations/ + ``` + +- Download MSCOCO-2014 dataset and unzip the images it to path： + ``` + data/coco/train2014/ + data/coco/val2014/ + ``` + + +- **Training**: + - 48/17 split: + ``` + chmod +x tools/dist_train.sh + ./tools/dist_train.sh configs/zsi/train/zero-shot-mask-rcnn-BARPN-bbox_mask_sync_bg_decoder.py 4 + ``` + + - 65/15 split: + ``` + chmod +x tools/dist_train.sh + ./tools/dist_train.sh configs/zsi/train/zero-shot-mask-rcnn-BARPN-bbox_mask_sync_bg_65_15_decoder_notanh.py 4 + ``` + +- **Inference & Evaluate**: + + + **ZSI task**: + + - 48/17 split ZSI task: + - download [48/17](https://drive.google.com/file/d/1MMDcNWHjTTOaPvMYVlypMIXdvdOcSnao/view?usp=sharing) ZSI model, put it in checkpoints/ZSI_48_17.pth + + - inference: + ``` + chmod +x tools/dist_test.sh + ./tools/dist_test.sh configs/zsi/48_17/test/zsi/zero-shot-mask-rcnn-BARPN-bbox_mask_sync_bg_decoder.py checkpoints/ZSI_48_17.pth 4 --json_out results/zsi_48_17.json + ``` + - our results zsi_48_17.bbox.json and zsi_48_17.segm.json can also downloaded from [zsi_48_17_reults](https://dagshub.com/f2010126/Zero-shot-Instance-Segmentation/src/main/results). + - evaluate: + - for zsd performance + ``` + python tools/zsi_coco_eval.py results/zsi_48_17.bbox.json --ann data/coco/annotations/instances_val2014_unseen_48_17.json + ``` + - for zsi performance + ``` + python tools/zsi_coco_eval.py results/zsi_48_17.segm.json --ann data/coco/annotations/instances_val2014_unseen_48_17.json --types segm + ``` + - 65/15 split ZSI task: + - download [65/15](https://drive.google.com/file/d/1UZMNQ9a9Gpbn53JGPilzyjl1oiFYyPw5/view?usp=sharing) ZSI model, put it in checkpoints/ZSI_65_15.pth + + - inference: + ``` + chmod +x tools/dist_test.sh + ./toools/dist_test.sh configs/zsi/65_15/test/zsi/zero-shot-mask-rcnn-BARPN-bbox_mask_sync_bg_65_15_decoder_notanh.py checkpoints/ZSI_65_15.pth 4 --json_out results/zsi_65_15.json + ``` + - our results zsi_65_15.bbox.json and zsi_65_15.segm.json can also downloaded from [zsi_65_15_reults](https://dagshub.com/f2010126/Zero-shot-Instance-Segmentation/src/main/results). + - evaluate: + - for zsd performance + ``` + python tools/zsi_coco_eval.py results/zsi_65_15.bbox.json --ann data/coco/annotations/instances_val2014_unseen_65_15.json + ``` + - for zsi performance + ``` + python tools/zsi_coco_eval.py results/zsi_65_15.segm.json --ann data/coco/annotations/instances_val2014_unseen_65_15.json --types segm + ``` + + + **GZSI task**: + + - 48/17 split GZSI task: + - use the same model file ZSI_48_17.pth in ZSI task + - inference: + ``` + chmod +x tools/dist_test.sh + ./tools/dist_test.sh configs/zsi/48_17/test/gzsi/zero-shot-mask-rcnn-BARPN-bbox_mask_sync_bg_decoder_gzsi.py checkpoints/ZSI_48_17.pth 4 --json_out results/gzsi_48_17.json + ``` + - our results gzsi_48_17.bbox.json and gzsi_48_17.segm.json can also downloaded from [gzsi_48_17_results](https://dagshub.com/f2010126/Zero-shot-Instance-Segmentation/src/main/results). + - evaluate: + - for gzsd + ``` + python tools/gzsi_coco_eval.py results/gzsi_48_17.bbox.json --ann data/coco/annotations/instances_val2014_gzsi_48_17.json --gzsi --num-seen-classes 48 + ``` + - for gzsi + ``` + python tools/gzsi_coco_eval.py results/gzsi_48_17.segm.json --ann data/coco/annotations/instances_val2014_gzsi_48_17.json --gzsi --num-seen-classes 48 --types segm + ``` + - 65/15 split GZSI task: + - use the same model file ZSI_48_17.pth in ZSI task + - inference: + ``` + chmod +x tools/dist_test.sh + ./tools/dist_test.sh configs/zsi/65_15/test/gzsi/zero-shot-mask-rcnn-BARPN-bbox_mask_sync_bg_65_15_decoder_notanh_gzsi.py checkpoints/ZSI_65_15.pth 4 --json_out results/gzsi_65_15.json + ``` + - our results gzsi_65_15.bbox.json and gzsi_65_15.segm.json can also downloaded from [gzsi_65_15_results](https://dagshub.com/f2010126/Zero-shot-Instance-Segmentation/src/main/results). + - evaluate: + - for gzsd + ``` + python tools/gzsi_coco_eval.py results/gzsi_65_15.bbox.json --ann data/coco/annotations/instances_val2014_gzsi_65_15.json --gzsd --num-seen-classes 65 + ``` + - for gzsi + ``` + python tools/gzsi_coco_eval.py results/gzsi_65_15.segm.json --ann data/coco/annotations/instances_val2014_gzsi_65_15.json --gzsd --num-seen-classes 65 --types segm + ``` + + +# License + +ZSI is released under MIT License. + + +## Citing + +If you use ZSI in your research or wish to refer to the baseline results published here, please use the following BibTeX entries: + +```BibTeX + +@InProceedings{Zheng_2021_CVPR, + author = {Zheng, Ye and Wu, Jiahong and Qin, Yongqiang and Zhang, Faen and Cui, Li}, + title = {Zero-Shot Instance Segmentation}, + booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, + month = {June}, + year = {2021}, + pages = {2593-2602} +} + +``` From f3a0e431e3f9fa1bad74ec62b1bba2eeca4b7992 Mon Sep 17 00:00:00 2001 From: Dipti Sengupta Date: Sun, 9 Oct 2022 12:31:18 +0200 Subject: [PATCH 3/3] Checkpoints --- Zero-shot-Instance-Segmentation/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Zero-shot-Instance-Segmentation/README.md b/Zero-shot-Instance-Segmentation/README.md index fb7baf8..a4d167e 100644 --- a/Zero-shot-Instance-Segmentation/README.md +++ b/Zero-shot-Instance-Segmentation/README.md @@ -57,7 +57,7 @@ python setup.py develop + **ZSI task**: - 48/17 split ZSI task: - - download [48/17](https://drive.google.com/file/d/1MMDcNWHjTTOaPvMYVlypMIXdvdOcSnao/view?usp=sharing) ZSI model, put it in checkpoints/ZSI_48_17.pth + - download [48/17](https://dagshub.com/f2010126/Zero-shot-Instance-Segmentation/src/main/checkpoints) ZSI model, put it in checkpoints/ZSI_48_17.pth - inference: ``` @@ -75,7 +75,7 @@ python setup.py develop python tools/zsi_coco_eval.py results/zsi_48_17.segm.json --ann data/coco/annotations/instances_val2014_unseen_48_17.json --types segm ``` - 65/15 split ZSI task: - - download [65/15](https://drive.google.com/file/d/1UZMNQ9a9Gpbn53JGPilzyjl1oiFYyPw5/view?usp=sharing) ZSI model, put it in checkpoints/ZSI_65_15.pth + - download [65/15](https://dagshub.com/f2010126/Zero-shot-Instance-Segmentation/src/main/checkpoints) ZSI model, put it in checkpoints/ZSI_65_15.pth - inference: ```