Clarification about Tanks & Temples results

[morsingher]

Hi @Xharlie and thanks for the awesome work. I have been trying to bias NeRF with geometry lately and I'm glad to see such a good work published at CVPR 2022.

I have a question about the evaluation process on Tanks & Temples. Let's take the truck scene as an example. You report a PSNR of 25.36 for basic NeRF, that actually matches the value in NSVF (I assume it's just taken from there). In that paper, they say: "We label the object masks ourselves with the software of Altizure".

Do you also use object masks and compute the metrics only on pixels from that mask? It seems plausible, as the results shown in your paper and your repo contain only the object, without the background.

I'm asking this because the PSNR of NeRF is significantly smaller, when computed on the whole image. It drops down to 20.85 as reported in NeRF++ and I have been able to reach a bit more than 21 with some tuning, but not more.

Thank you in advance for the answer!

[Xharlie]

Sure, we follow everything in NSVF, thank you!