training#

Methods for training half-sibling regression models.

hsr4hci.training._train_default_model(base_model_creator, train_predictors, train_targets)[source]#

Train a default model (i.e., a “vanilla” HSR model with no signal fitting or masking).

Caution

This function should not be used directly, but rather through train_model_for_position() with train_mode set to “default”.

Parameters:

  • base_model_creator (BaseModelCreator) – Instance of BaseModelCreator that can be used to instantiate a new model.

  • train_predictors (ndarray) – A 2D numpy array containing the (normalized) predictors. Shape: (n_time_steps, n_features).

  • train_targets (ndarray) – A 1D numpy array containing the (normalized) targets. Shape (n_time_steps, ).

Returns:

The trained model instance, or None, if the training failed with a numpy.linalg.LinAlgError.

Return type:

Optional[RegressorModel]

hsr4hci.training._train_signal_fitting_model(base_model_creator, train_predictors, train_targets, expected_signal)[source]#

Train a model with signal fitting.

Caution

This function should not be used directly, but rather through train_model_for_position() with train_mode set to “signal_fitting”.

Parameters:

  • base_model_creator (BaseModelCreator) – Instance of BaseModelCreator that can be used to instantiate a new model.

  • train_predictors (ndarray) – A 2D numpy array containing the (normalized) predictors. Shape: (n_time_steps, n_features).

  • train_targets (ndarray) – A 1D numpy array containing the (normalized) targets. Shape: (n_time_steps, ).

  • expected_signal (ndarray) – A 1D numpy array containing the expected signal for the hypothesis under which we are training the current model. Shape: (n_time_steps, ).

Returns:

A 2-tuple, consisting of

  1. the trained noise model instance (i.e., with the planet coefficient removed), and

  2. the value of the planet coefficient that was removed.

If the training fails with a numpy.linalg.LinAlgError, this function returns (None, numpy.nan).

Return type:

Tuple[Optional[RegressorModel], float]

hsr4hci.training._train_signal_masking_model(base_model_creator, train_predictors, train_targets, expected_signal)[source]#

Train a model with signal masking.

Caution

This function should not be used directly, but rather through train_model_for_position() with train_mode set to “signal_masking”.

Parameters:

  • base_model_creator (BaseModelCreator) – Instance of BaseModelCreator that can be used to instantiate a new model.

  • train_predictors (ndarray) – A 2D numpy array containing the (normalized) predictors. Shape: (n_time_steps, n_features).

  • train_targets (ndarray) – A 1D numpy array containing the (normalized) targets. Shape: (n_time_steps, ).

  • expected_signal (ndarray) – A 1D numpy array containing the expected signal for the hypothesis under which we are training the current model. Shape: (n_time_steps, ).

Returns:

The trained model instance, or None, if the training failed with a numpy.linalg.LinAlgError.

Return type:

Optional[RegressorModel]

hsr4hci.training.add_obscon_as_predictors(predictors, obscon_array, expected_signal, max_correlation=0.5)[source]#

Merge the predictors and the obscon_array and take into account the desired maximum correlation with the expected signal.

Parameters:

  • predictors (ndarray) – A 2D numpy array with shape (n_frames, n_pixels) that contains the predictors for a target pixel.

  • obscon_array (ndarray) – A 2D numpy array with shape (n_frames, n_obscon) that contains the (global) observing conditions.

  • expected_signal (ndarray) – A 1D numpy array with shape (n_frames,) that contains the expected signal for a target pixel. In case we are training a default model, the expected_signal will be all NaN.

  • max_correlation (float) – Maximum value for the correlation between the expected signal and the observing conditions. OCs with a higher correlation will not be added as predictors.

Returns:

A 2D numpy array with shape (n_frames, n_predictors) that contains the (full) predictors: both pixels and admissible OC. This means: n_predictors <= n_pixels + n_obscon.

Return type:

ndarray

hsr4hci.training.get_signal_times(n_frames, n_signal_times)[source]#

Generate a temporal grid of signal times. This function is just a wrapper around numpy.linspace() and exists mainly to ensure consistency everywhere.

Parameters:

  • n_frames (int) – The total number of frames in the stack.

  • n_signal_times (int) – The number of positions on the temporal grid that we create.

Returns:

A 1D numpy array of shape (n_signal_times, ) containing the temporal grid (i.e., signal times) as integers.

Return type:

ndarray

hsr4hci.training.train_all_models(roi_mask, stack, parang, obscon_array, selection_mask_config, base_model_creator, psf_template, train_mode, max_oc_correlation=0.5, n_train_splits=3, n_signal_times=30, n_roi_splits=1, roi_split=0, return_format='full')[source]#

Loop over all positions selected by the roi_mask (or rather, the subset given by roi_split and n_roi_splits), train a model for each pixel (using train_model_for_position()) and each potential signal time from the temporal grid, and return the results formatted according to the requested return_format.

Parameters:

  • roi_mask (ndarray) – A 2D numpy array of shape (x_size, y_size) that contains a binary mask to select the region of interest, that is, the pixels for which to train noise models.

  • stack (ndarray) – A 3D numpy array of shape (n_frames, x_size, y_size) containing the data on which to train the noise models.

  • parang (ndarray) – A 1D numpy array of shape (n_frames, ) containing the corresponding parallactic angle for each frame.

  • obscon_array (ndarray) – A 2D numpy array of shape (n_frames, n_features) containing the observing conditions that should be used as additional predictors.

  • selection_mask_config (Dict[str, Any]) – A dictionary containing two keys (namely “radius_position” and “radius_opposite”) that define the mask that is used to select the predictor pixels. The values of the dict should be tuples of the form (value, “unit”).

  • base_model_creator (BaseModelCreator) – An instance of BaseModelCreator that can be used to instantiate new base models.

  • psf_template (ndarray) – A 2D numpy array containing the unsaturated PSF template.

  • train_mode (str) – The mode to use for training; must be one of the following: “default” (for the vanilla HSR model), “signal_fitting” or “signal_masking”.

  • max_oc_correlation (float) – Maximum value for the correlation between the expected_signal and an observing conditions (OC) for this OC to be used as a predictor. (Basically, we do not want to use OC that are “accidentally” strongly correlated with a potential signal.)

  • n_train_splits (int) – The number of training / test splits to use.

  • n_signal_times (int) – The size of the temporal grid, that is, the number of different (temporal) signal positions that are assumed for each pixel.

  • n_roi_splits (int) – The (total) number of splits into which the ROI should be divided.

  • roi_split (int) – The index of the split for which to return the mask.

  • return_format (str) – The format in which the residuals are returned. If “full”, the residuals are 3D arrays that have the same size as the stack. If “partial”, the residuals are 2D arrays that have the shape (n_frames, n_pixels_in_split). The latter is recommended when training in parallel, because otherwise we waste a lot of storage for storing NaN in the intermediate result files.

Returns:

A dictionary containing three keys

  1. ”stack_shape”: the shape of the original stack; required when merging partial result files.

  2. ”roi_mask”: the partial ROI mask that was used for training; also required when merging partial result files.

  3. ”residuals”: a dictionary with keys “default”, “0”, …, “N”, where the latter are the signal times for which we have trained models. Each key maps onto a numpy array containing the residuals for the respective model.

Return type:

Dict[str, Union[ndarray, Dict[str, ndarray]]]

hsr4hci.training.train_model_for_position(stack, parang, obscon_array, position, train_mode, signal_time, selection_mask_config, psf_template, n_train_splits, base_model_creator, expected_signal=None, max_oc_correlation=0.5)[source]#

Train a model for a given position (or rather: a set of models, because of the train / test splitting scheme).

Parameters:

  • stack (ndarray) – A 3D numpy array of shape (n_frames, x_size, y_size) containing the data on which to train the noise models.

  • parang (ndarray) – A 1D numpy array of shape (n_frames, ) containing the corresponding parallactic angle for each frame.

  • obscon_array (ndarray) – A 2D numpy array of shape (n_frames, n_features) containing the observing conditions that should be used as additional predictors.

  • position (Tuple[int, int]) – A tuple (x, y) of integers containing the position for which to train the model(s).

  • train_mode (str) – The mode to use for training; must be one of the following: “default” (for the vanilla HSR model), “signal_fitting” or “signal_masking”.

  • signal_time (Optional[int]) – If train_mode is “default”, this should be None. Otherwise, this should contain the time at which the planet signal is assumed to peak at the given position (we need this value to be able to compute the forward model for signal fitting / masking).

  • selection_mask_config (Dict[str, Any]) – A dictionary containing two keys (namely “radius_position” and “radius_opposite”) that define the mask that is used to select the predictor pixels. The values of the dict should be tuples of the form (value, “unit”).

  • psf_template (ndarray) – A 2D numpy array containing the unsaturated PSF template.

  • n_train_splits (int) – The number of training / test splits to use.

  • base_model_creator (BaseModelCreator) – An instance of BaseModelCreator that can be used to instantiate new base models.

  • expected_signal (Optional[ndarray]) – If the train_mode is “signal_fitting” or “signal_masking”, you can optionally also pass the expected signal explicitly to this function to avoid computing it here. This option may be useful when the HSR is used “hypothesis- based” instead of for a blind search, that is, we already have a hypothesis about the planet position from which we can compute the expected signal stack, meaning we do not need to loop over a temporal grid but only train a single model per pixel (either “default” or “signal_fitting” / “signal_masking”). Note that the expected_signal should be consistent with the given signal_time; otherwise the mask that is used for the pixel predictor selection will be wrong.

  • max_oc_correlation (float) – Maximum value for the correlation between the expected_signal and an observing conditions (OC) for this OC to be used as a predictor. (Basically, we do not want to use OC that are “accidentally” strongly correlated with a potential signal.)

Returns:

A 2-tuple consisting of

  1. the residual time series for the given position,

  2. a dictionary containing additional debugging information about the model(s) that we have trained; for example, the values of the coefficients or regularization coefficients.

Return type:

Tuple[ndarray, Dict[str, ndarray]]