Learned partition for vkitti

[yaping222]

Hi Loic,
Thank you very much for your great codes.
I met two issues in over-segmentation of vkitti.
I run the following code:
for FOLD in 1 2 3 4 5 6; do
python ./supervized_partition/supervized_partition.py --ROOT_PATH /media/yp/yaping/data/vkitti3d --dataset vkitti
--epochs 50 --test_nth_epoch 10 --cvfold $FOLD --reg_strength 0.5 --spatial_emb 0.02 --batch_size 15
--global_feat exyrgb --CP_cutoff 10 --odir results_part/vkitti/best;
done;
But I got 'Killed' during the training for Fold 5 and 6. I think it is because of my limited memory. So how much memory is required to process the data?

Also, when I run:
python supervized_partition/evaluate_partition.py --dataset vkitti --cvfold 123456 --folder best
I got an error like this:
FileNotFoundError: [Errno 2] No such file or directory: './results_partition/vkitti/best/results.txt'
What are the expected results from 'supervized_partition.py'?

I'm a little bit confused on how to implement learned partition could you give more explanation?

[loicland]

For some version of cuda the training can crash. This us an issue with cuda and not the code. If you have 12gb of gout tamoxifen you should be gold. Otherwise decrease the batch size.

Either way you can resume the training where it crashed with the --resume RESUME option.

Wait for the training to end before running evaluate_partition.

[yaping222]

Hi Loic,
Thank you very much for your reply.
I use the --resume RESUME option and I finish the training.
When I run:
python supervized_partition/evaluate_partition.py --dataset vkitti --cvfold 123456 --folder best
There's an error:
FileNotFoundError: [Errno 2] No such file or directory: './results_partition/vkitti/best/results.txt'
In my folder './results_part/vkitti/best/cv1', I got the following files:
cmdline.txt, model.pth.tar, res.h5, trainlog.json
So what is the expected results for 'results.txt'?

[kxhit]

Hi~ @yaping222 @loicland Sorry to bother you.
I'm dealing with vktti3d dataset and got memory error when generating .npy files. My memory is 32G. I don't know how much memory is enough for this. The same issue in vkitti3d. Could you help me? Thanks!

[loicland]

@yaping222 I assume the error is because ./results_partition and ./results_part are spelled differently

[loicland]

@kxhit vkitti3d is quite a small dataset, I am sure 32GB is more than anough? Which .npy are your eferring to?

[kxhit]

@loicland Ops! I used the v2 version of vkitti3d and now I find that v1 version should be used. The size of v1 version is only 971.7M, making me ignore it by mistake. The v2 version needs to be processed by this code which consumes a lot of memory.
Maybe you can remind others that v1 version should be used. Thanks a lot!