Contact Support

NetCDF / HDF5 file locking

NeSI provides libraries for creating and accessing files in the NetCDF and HDF5 formats on both Mahuika and Māui.

On Māui, both libraries are part of the Cray Programming Environment and can be accessed using the modules 'cray-netcdf'/'cray-netcdf-hdf5parallel', and 'cray-hdf5'/'cray-hdf5-parallel'.

Recent versions of NetCDF and HDF5 (HDF5 1.10.x and newer) use a file locking feature. This prevents data corruption in rare cases of single-writer/multiple-reader and multiple writer access patterns.

On our current XC50 platform (Māui), file locking is not yet fully supported by Cray's DVS. (DVS allows accessing the parallel file system GPFS on the XC50 compute nodes).

Accordingly, NetCDF-4 and HDF5 applications that write data from Māui compute nodes need to disable file locking with:


If file locking is enabled, HDF5/NetCDF4 applications may experience errors such as

ncdump: /path/to/ NetCDF: HDF error


Error in EM_FOPEN: NetCDF: HDF error - /path/to/ 


(-101) // Error at HDF5 layer


HDF5-DIAG: Error detected in HDF5 (1.10.2) thread 0:
#000: ../../src/H5F.c line 445 in H5Fcreate(): unable to create file
major: File accessibilty
minor: Unable to open file
#001: ../../src/H5Fint.c line 1519 in H5F_open(): unable to lock the file
major: File accessibilty
minor: Unable to open file
#002: ../../src/H5FD.c line 1650 in H5FD_lock(): driver lock request failed
major: Virtual File Layer
minor: Can't update object
#003: ../../src/H5FDsec2.c line 941 in H5FD_sec2_lock(): unable to lock file, errno = 524, error message = 'Unknown error 524'
major: File accessibilty
minor: Bad file ID accessed


As file locking has to be disabled on the XC50, care should be taken to avoid concurrent reader/writer access in your application.

For more information see: