merging#

Methods for merging partial result files (FITS / HDF).

hsr4hci.merging.get_list_of_fits_file_paths(fits_dir, prefix)[source]#

Get a list of all FITS files in a given fits_dir whose file name begins with the given prefix.

Parameters:

  • fits_dir (Path) – Path to directory in which to look for FITS files.

  • prefix (str) – Only consider FITS files whose names begin with this. For example: “hypotheses” or “mean_mf”.

Returns:

A list of Paths to the matching FITS files in fits_dir.

Return type:

List[Path]

hsr4hci.merging.get_list_of_hdf_file_paths(hdf_dir, prefix='residuals')[source]#

Get a list of all HDF files in a given hdf_dir whose file name begins with the given prefix.

Parameters:

  • hdf_dir (Path) – Path to directory in which to look for HDF files.

  • prefix (str) – Only consider HDF files whose names begin with this. Usually, we only need HDF files starting with “residuals”.

Returns:

A list of Paths to the matching HDF files in hdf_dir.

Return type:

List[Path]

hsr4hci.merging.merge_fits_files(fits_file_paths)[source]#

Take a list of FITS files and merge all of them into a single array.

This function can merge the partial result files that are obtained in parallel with hsr4hci.hypotheses.get_all_hypotheses() and hsr4hci.match_fraction.get_all_match_fractions().

Merging works by stacking the arrays from the FITS files along a new axis and then taking the numpy.nanmean() along this axis. This, of course, assumes that each pixel only takes on a non- NaN value in at most one of the FITS files.

Parameters:

fits_file_paths (List[Path]) – List of FITS files to be merged.

Returns:

A numpy array containing the merged arrays from all FITS files.

Return type:

ndarray

hsr4hci.merging.merge_hdf_files(hdf_file_paths)[source]#

Take a list of HDF files and merge all of them into a single dict.

This function is intended to merge the (partial) results files that are produced by hsr4hci.training.train_all_models(); see there for more details on the expected internal structure of the files.

Parameters:

hdf_file_paths (Sequence[Path]) – A list of paths to the HDF files to be merged.

Returns:

A dictionary containing the “full” (i.e., merged) results from all HDF files.

Return type:

Dict[str, ndarray]