Gibson Data Structure for Reproduce

[Tllokn]

Hello,
Thanks for your great work. I am trying to reproduce the work to gibson datasets.
I followed #8 :

1. add a parameter "--task_config tasks/objectnav_gibson.yaml"
2. replace the line in the file agents/sem_exp.py from ObjectGoal_Env21 to ObjectGoal_Env
3. I also Force the task only work on the Darden scene, since I am sure I have all assets of this scene. I changed code: in envs/habitat/__init__.py

    if "*" in basic_config.DATASET.CONTENT_SCENES:
        content_dir = os.path.join("data/datasets/objectnav/gibson/v1.1/" + args.split, "content")
        scenes = _get_scenes_from_folder(content_dir)
        # force it to be only Darden, since we only have this in scene dataset.
        scenes = ['Darden.glb']

4. To match the dataset, I changed the objectnav_gibson.yaml with:

DATASET:
  TYPE: PointNav-v1
  SPLIT: train
  DATA_PATH: "data/datasets/objectnav/gibson/v1.1/{split}/{split}.json.gz"
  EPISODES_DIR: "data/datasets/objectnav/gibson/v1.1/{split}/"
  SCENES_DIR: "/data/scene_datasets"

With all these changes, I got an error:

Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/L3MVN/envs/habitat/utils/vector_env.py", line 176, in _worker_env
    env = env_fn(*env_fn_args)
  File "/L3MVN/envs/habitat/__init__.py", line 24, in make_env_fn
    env = Sem_Exp_Env_Agent(args=args, rank=rank,
  File "/L3MVN/agents/sem_exp.py", line 32, in __init__
    super().__init__(args, rank, config_env, dataset)
  File "/L3MVN/envs/habitat/objectgoal_env.py", line 26, in __init__
    super().__init__(config_env, dataset)
  File "/habitat-lab/habitat/core/env.py", line 377, in __init__
    self._env = Env(config, dataset)
  File "/habitat-lab/habitat/core/env.py", line 107, in __init__
    self._sim = make_sim(
  File "/habitat-lab/habitat/sims/registration.py", line 19, in make_sim
    return _sim(**kwargs)
  File "/habitat-lab/habitat/sims/habitat_simulator/habitat_simulator.py", line 276, in __init__
    super().__init__(self.sim_config)
  File "<attrs generated init habitat_sim.simulator.Simulator>", line 11, in __init__
    self.__attrs_post_init__()
  File "/usr/local/lib/python3.8/dist-packages/habitat_sim-0.2.1-py3.8-linux-x86_64.egg/habitat_sim/simulator.py", line 122, in __attrs_post_init__
    self.__set_from_config(self.config)
  File "/usr/local/lib/python3.8/dist-packages/habitat_sim-0.2.1-py3.8-linux-x86_64.egg/habitat_sim/simulator.py", line 260, in __set_from_config
    self._config_backend(config)
  File "/usr/local/lib/python3.8/dist-packages/habitat_sim-0.2.1-py3.8-linux-x86_64.egg/habitat_sim/simulator.py", line 204, in _config_backend
    super().__init__(config.sim_cfg, config.metadata_mediator)
AssertionError: ESP_CHECK failed: Missing (at least) one of scene dataset attributes, stage attributes, or dataset scene attributes for scene '/data/scene_datasets/gibson_semantic//Darden.glb'.  Likely an invalid scene name.

I guess this error is related to how I organize the data of Gibson, since I have seen similar problem when I didn't put data of hm3d correctly. But given these 2 dataset use different data format, I do not have a concrete idea on how to solving this problem. Can you share yours? My data structure looks like below

My data folder looks like this:
Data:

Data
|-- datasets
|-- matterport_category_mappings.tsv
|-- object_norm_inv_perplexity.npy
|-- objectgoal_hm3d
|-- scene_datasets
`-- versioned_data

datasets

datasets
|-- objectnav
|   `-- gibson
|       `-- v1.1
|           |-- train
|           |   |-- content
|           |   |   |-- Allensville.glb.json.gz
|           |   |   |-- Beechwood.glb.json.gz
|           |   |   |-- Benevolence.glb.json.gz
|           |   |   |-- Coffeen.glb.json.gz
|           |   |   |-- Cosmos.glb.json.gz
|           |   |   |-- Forkland.glb.json.gz
|           |   |   |-- Hanson.glb.json.gz
|           |   |   |-- Hiteman.glb.json.gz
|           |   |   |-- Klickitat.glb.json.gz
|           |   |   |-- Lakeville.glb.json.gz
|           |   |   |-- Leonardo.glb.json.gz
|           |   |   |-- Lindenwood.glb.json.gz
|           |   |   |-- Marstons.glb.json.gz
|           |   |   |-- Merom.glb.json.gz
|           |   |   |-- Mifflinburg.glb.json.gz
|           |   |   |-- Newfields.glb.json.gz
|           |   |   |-- Onaga.glb.json.gz
|           |   |   |-- Pinesdale.glb.json.gz
|           |   |   |-- Pomaria.glb.json.gz
|           |   |   |-- Ranchester.glb.json.gz
|           |   |   |-- Shelbyville.glb.json.gz
|           |   |   |-- Stockman.glb.json.gz
|           |   |   |-- Tolstoy.glb.json.gz
|           |   |   |-- Wainscott.glb.json.gz
|           |   |   `-- Woodbine.glb.json.gz
|           |   |-- train.json.gz
|           |   `-- train_info.pbz2
|           `-- val
|               |-- content
|               |   |-- Collierville.glb.json.gz
|               |   |-- Collierville_episodes.json.gz
|               |   |-- Corozal.glb.json.gz
|               |   |-- Corozal_episodes.json.gz
|               |   |-- Darden.glb.json.gz
|               |   |-- Darden_episodes.json.gz
|               |   |-- Markleeville.glb.json.gz
|               |   |-- Markleeville_episodes.json.gz
|               |   |-- Wiconisco.glb.json.gz
|               |   `-- Wiconisco_episodes.json.gz
|               |-- val.json.gz
|               `-- val_info.pbz2
`-- objectnav_gibson_v1.1.zip

scene_datasets

scene_datasets
|-- gibson_semantic
|   |-- Ackermanville.glb
|   |-- Ackermanville.navmesh
|   |-- Darden.glb
|   `-- Darden.navmesh
`-- hm3d_v0.2
    |-- hm3d -> /home/my-home-robot/data/versioned_data/hm3d-0.2/hm3d
    |-- hm3d_annotated_basis.scene_dataset_config.json
    `-- val

versioned_data

versioned_data
`-- hm3d-0.2
    |-- hm3d
    |   |-- hm3d -> /home/my-home-robot/data/versioned_data/hm3d-0.2/hm3d
    |   |-- hm3d_annotated_basis.scene_dataset_config.json
    |   `-- val
    |-- val-configs-files.json.gz
    |-- val-habitat-files.json.gz
    |-- val-semantic-annots-files.json.gz
    `-- val-semantic-configs-files.json.gz

[weizhenFrank]

I solved this problem by downloading the gibson with semantic annotation.

https://github.com/StanfordVL/3DSceneGraph/tree/master

According to https://github.com/facebookresearch/habitat-sim/blob/main/DATASETS.md#gibson-and-3dscenegraph-datasets.

And we need habitat-sim v-0.1.4 to install
https://github.com/facebookresearch/habitat-sim/releases/tag/v0.1.4

Also, by facebookresearch/habitat-sim#648

we need python setup.py build --build-datatool
then we copy newest tools/gen_gibson_semantics.sh

then we run in the v0.1.4 habitat-sim,
tools/gen_gibson_semantics.sh ~/projects/L3MVN/data/gibson_tiny_data_semantics/automated_graph ~/projects/L3MVN/data/gibson_tiny ~/projects/L3MVN/data/scene_datasets/gibson/gibson_semantic

This way help me to get the gibson with semantic info.

[hhhuxy]

I solved this problem by downloading the gibson with semantic annotation.

https://github.com/StanfordVL/3DSceneGraph/tree/master

According to https://github.com/facebookresearch/habitat-sim/blob/main/DATASETS.md#gibson-and-3dscenegraph-datasets.

And we need habitat-sim v-0.1.4 to install https://github.com/facebookresearch/habitat-sim/releases/tag/v0.1.4

Also, by facebookresearch/habitat-sim#648

we need python setup.py build --build-datatool then we copy newest tools/gen_gibson_semantics.sh

then we run in the v0.1.4 habitat-sim, tools/gen_gibson_semantics.sh ~/projects/L3MVN/data/gibson_tiny_data_semantics/automated_graph ~/projects/L3MVN/data/gibson_tiny ~/projects/L3MVN/data/scene_datasets/gibson/gibson_semantic

This way help me to get the gibson with semantic info.

@weizhenFrank Thank you for your advice, I follow the above instructions, but I am still confused.

When I download gibson scene dataset for habitat, there are only *.ply and *.navmesh files in the folder.
In tools/gen_gibson_semantics.sh L24, "${TOOLS_DIR}"/../build/utils/datatool/datatool create_gibson_semantic_mesh "${OBJ_PATH}"/"${scene}"/mesh.obj "${OUT_PATH}"/"${scene}".ids "${OUT_PATH}"/"${scene}"_semantic.ply, however, there is no mesh.obj file in my gibson dataset thus leads to errors.

Did I downloaded the wrong link for gibson? Could u please share which part of gibson dataset u downloaded, or how to obtain mesh.obj file?

Thank you in advance.