Generating a Cosmoflow dataset is very slow

[Linzsd]

When I was generating the dataset for Cosmoflow, the generation was slow, and it took a minute interval to print the logs once. When I switch the data format to npz, the generation is very fast, why is that?

[daniarherikurniawan]

Try increasing the -n (--num-accelerators) to big number ( 20 or 50) during data generation.
I also modified line 262 https://github.com/mlcommons/storage/blob/main/benchmark.sh#L262C2-L262C37
Changing mpirun -hosts $hosts -np $parallel ... to mpirun -np $parallel ...
I haven't tried the npz format.