From 8d0c552429ee9cc69662ce759a9c83949052222f Mon Sep 17 00:00:00 2001 From: Daniel Loos Date: Wed, 16 Oct 2024 10:57:15 +0200 Subject: [PATCH 1/2] Add thread-safety note for NetCDF4 --- docs/src/UserGuide/read.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/src/UserGuide/read.md b/docs/src/UserGuide/read.md index cf4024bc..6c4b33e9 100644 --- a/docs/src/UserGuide/read.md +++ b/docs/src/UserGuide/read.md @@ -43,6 +43,9 @@ Individual arrays can be accessed using subsetting: ds.tos ```` +Please note that netCDF4 uses HDF5 which is not thread-safe in Julia. +Use manual [locks](https://docs.julialang.org/en/v1/manual/multi-threading/#man-using-locks) in your own code to avoid any data-race. + ## Read GDAL (GeoTIFF, GeoJSON) All GDAL compatible files can be read as a `YAXArrays.Dataset` after loading [ArchGDAL](https://yeesian.com/ArchGDAL.jl/latest/): From c9f55a2045123b6514740f03879aa9c3ba6e35fc Mon Sep 17 00:00:00 2001 From: Daniel Loos Date: Wed, 16 Oct 2024 14:01:16 +0200 Subject: [PATCH 2/2] Add lock example --- docs/src/UserGuide/read.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/docs/src/UserGuide/read.md b/docs/src/UserGuide/read.md index 6c4b33e9..90286e3d 100644 --- a/docs/src/UserGuide/read.md +++ b/docs/src/UserGuide/read.md @@ -44,7 +44,16 @@ ds.tos ```` Please note that netCDF4 uses HDF5 which is not thread-safe in Julia. -Use manual [locks](https://docs.julialang.org/en/v1/manual/multi-threading/#man-using-locks) in your own code to avoid any data-race. +Add manual [locks](https://docs.julialang.org/en/v1/manual/multi-threading/#man-using-locks) in your own code to avoid any data-race: + +````@example read_netcdf +my_lock = ReentrantLock() +Threads.@threads for i in 1:10 + @lock my_lock @info ds.tos[1, 1, 1] +end +```` + +This code will ensure that the data is only accessed by one thread at a time, i.e. making it actual single-threaded but thread-safe. ## Read GDAL (GeoTIFF, GeoJSON)