This repository was developed as a proof of concept for a pipeline for semantic segmentation of glTF models contained within 3D Tiles.
- Texture-based semantic segmentation of glTF models
- Support for glTF, glb as well as Draco compressed glb
- 3D Tiles conversion based on py3dtiles
- Extraction of 3D Tiles by bounding box or region of interest
- Python 3.9 or higher
- (Recommended) CUDA and cuDNN for GPU acceleration
- (Optional) Node.js for the webserver: Refer to the webserver README for more information
- (Optional) Unity v2022.3.5f1
pip install -r requirements.txt
To run the segmentation pipeline, run the following command:
python pipeline.py --file <tileset-file-name> --input <input-directory> --output <output-directory>
For instance, if your directory structure looks like this:
.
├── pipeline.py
├── input/
│ ├── tileset.json
│ └── model.glb
└── output/
Run the following command:
python pipeline.py --file tileset.json --input input/ --output output/
The pipeline will output a segmented .glb file containing the metadata of the class of each vertex in the model.
With a large tileset, having a smaller, selected area of region will significantly reduced the pipeline processing time. The extract_tileset.py script can be used to extract a specific branch of a tileset by specifying either the target file name or the latitude and longitude bounds of the region of interest.
To select a specific branch of the tileset by specifying the target file name, the --search-name argument can be specified.
python extract_tileset.py --file <tileset-file-name> --input <input-directory> --output <output-directory> --search-name <target-file-name>
Upon finding the target file, the script will isolate the branch the target file is on and exports the extracted branch to the output directory. Parent tilesets will be modified to contain only the specific child that is within the branch.
Alternatively, to extract by latitude and longitude bounds, the --bounds argument can be specified instead.
python extract_tileset.py --file <tileset-file-name> --input <input-directory> --output <output-directory> --bounds <lat_1> <lon_1> <lat_2> <lon_2>
The script will search for tiles and checks if the centre of the bounding box of the tile is within the given bounds. If the tile is within the bounds, the tile will be exported to the output directory. Parent tilesets will be modified to contain only the specific child that is within the bounds.
To extract the target tile and its neighbours, the --neighbours argument can be specified.
python extract_tileset.py --file <tileset-file-name> --input <input-directory> --output <output-directory> --search-name <target-file-name> --neighbours
The neighbours of the tile containing the target file will be kept in the extracted tileset.
Allows the segmentation of a .glb file into either a single .glb file containing metadata or into multiple .glb files split by class.
python glb/segment.py --file <glb file path> --output <output-directory>
To split by class, include the --submeshes argument.
Refer to the .glb segmentation README for more information.
The pipeline thereby consists of the following steps:
- Tileset Traversal - Traverses the tileset and retrieves the .glb file of each tile
- GLB Segmentation - Segments each .glb file into a single .glb file containing metadata
If the option to split by class is enabled with the inclusion of the --submeshes argument, the pipeline will have the following steps instead:
- Tileset Traversal - Traverses the tileset and retrieves the .glb file of each tile
- GLB Segmentation - Segments each .glb file into multiple .glb files split by class
- 3D Tiles Conversion - Exports a converted 3D Tileset with the necessary labels
If the pipeline is interrupted, restarting the pipeline will automatically resume from where it left off, by checking if the output file already exists.