hdf#

Methods for dealing with HDF files.

Parts of the code in the module are based on: https://codereview.stackexchange.com/a/121308

hsr4hci.hdf.create_hdf_dir(experiment_dir, create_on_work=False)[source]#

Create a directory in which the HDF results files for an HSR experiment can be stores and return the Path to the directory.

Attention

Unless you are working on the MPI-IS cluster in Tübingen, you always want to use create_on_work=False!

Background

This is slightly complicated, because the exact location depends on the machine on which this code is running. When running locally, it should simply be created directly in the respective experiment_dir. However, when this code is running on the MPI-IS cluster, we want to store the (large) HDF files on /work, with a symlink connecting it to the rest of the experiment_dir.

Parameters:

  • experiment_dir (Path) – The Path to the experiment directory for which we are going to create a hdf results directory.

  • create_on_work (bool) – If True, the HDF directory is created on /work and a symlink is created in experiment_dir. If False, the HDF directory is created directly in experiment_dir.

Returns:

The Path to the hdf directory for the experiment_dir.

Return type:

Path

hsr4hci.hdf.load_dict_from_hdf(file_path)[source]#

Load the contents of an HDF file into a dictionary to replicate the internal structure (group, subgroups, …) of the HDF file.

Parameters:

file_path (Union[Path, str]) – The path to the target HDF file.

Returns:

A dict containing the contents of the specified HDF file.

Return type:

dict

hsr4hci.hdf.recursively_load_dict_contents_from_group(hdf_object, path='')[source]#

Auxiliary function for recursively looping over the contents of a given hdf_object and loading them into a dictionary.

Parameters:

  • hdf_object (Union[File, Group]) – A HDF object; either an HDF file (root) or a group.

  • path (str) – The path to the hdf_object in the actual HDF file.

Returns:

The contents of hdf_object[path] as a dictionary.

Return type:

dict

hsr4hci.hdf.recursively_save_dict_contents_to_group(hdf_object, prefix, dictionary)[source]#

Auxiliary function for recursively looping over the contents of a dictionary and saving them to an HDF file.

Parameters:

  • hdf_object (Union[File, Group]) – Either an open HDF file, or a group inside such a file.

  • prefix (str) – Path to the location inside the HDF file; e.g., the name of a group, or a path (for nested groups).

  • dictionary (dict) – The dictionary to be saved at the given location.

Return type:

None

hsr4hci.hdf.save_data_to_hdf(hdf_file, location, name, data, overwrite=True)[source]#

Auxiliary function to write data to an open HDF file that provides automatic overwriting (which requires deleting and re-creating data sets that already exist).

Parameters:

  • hdf_file (File) – An open HDF file (in write mode).

  • location (str) – The path (“group_1/group_2/…/group_n”) at which to create the new data set in the HDF file. Can be empty.

  • name (str) – The name of the data set.

  • data (Any) – The data to be written to the data set.

  • overwrite (bool) – Whether to overwrite a data set of the same name that already exists in the given location.

Return type:

None

hsr4hci.hdf.save_dict_to_hdf(dictionary, file_path, mode='a', prefix='')[source]#

Save the given dictionary as an HDF file at the file_path. If the dictionary is nested, the HDF file will replicate this structure using groups.

Parameters:

  • dictionary (dict) – A (possibly nested) dictionary to be saved.

  • file_path (Union[Path, str]) – The path to the target file (including name and file ending).

  • mode (str) – The mode (i.e., “w” or “a”) that is used when opening the HDF file for writing.

  • prefix (str) – Prefix to use when writing to the HDF file. This can be used, for example, to write the dictionary into its own group inside the HDF file.

Return type:

None