twocan.RegEstimator

class twocan.RegEstimator(registration_max_features: int = 10000, registration_percentile: float = 0.9)[source]

Bases: TransformerMixin, BaseEstimator, ABC

A scikit-learn compatible estimator for multimodal image registration.

This class implements feature-based image registration using OpenCV’s ORB (Oriented FAST and Rotated BRIEF) detector and a partial affine transformation model. It follows scikit-learn’s estimator API with fit, transform, and fit_transform methods, making it easy to integrate into machine learning pipelines.

The registration process consists of: 1. Feature detection using ORB on both images 2. Feature matching using brute-force Hamming distance 3. Affine transformation estimation using RANSAC 4. Image transformation using the estimated parameters

Parameters:

  • registration_max_features (int, default=10000) – Maximum number of features to detect in each image using ORB. Higher values can improve registration accuracy but increase computation time.

  • registration_percentile (float, default=0.9) – Percentile of features to keep after sorting by match quality (0-1). Only the top percentile of matches by distance are used for transformation estimation, which helps remove outliers.

M_

The estimated 2x3 affine transformation matrix after fitting. Shape is (2, 3) representing the transformation [R|t] where R is rotation/scaling and t is translation.

Type:

np.ndarray

y_shape_

Shape (height, width) of the target image used during fitting. Used as the default output shape for transformations.

Type:

Tuple[int, int]

Notes

The estimator automatically converts input images to 8-bit grayscale for feature detection using the stretch_255 utility function. This ensures consistent feature detection regardless of input image dynamic range.

The partial affine transformation model allows rotation, scaling, and translation but not shearing, which is appropriate for most microscopy registration tasks where imaging geometry is approximately preserved.

For best results: - Ensure sufficient overlap between images - Use images with distinct features (not uniform regions) - Consider preprocessing to enhance relevant structures - Adjust max_features based on image complexity and computational budget

fit(X: ndarray, y: ndarray) → RegEstimator[source]

Estimate the affine transformation matrix between source (X) and target (y) images.

This method detects features in both images using ORB, matches them, and estimates the best affine transformation that maps source features to target features using OpenCV’s robust estimation.

Parameters:

  • X (np.ndarray) – Source image to be registered. Shape can be (H, W) for single-channel or (C, H, W) for multi-channel. If multi-channel, all channels are summed for feature detection.

  • y (np.ndarray) – Target (reference) image to register to. Shape can be (H, W) for single-channel or (C, H, W) for multi-channel. If multi-channel, all channels are summed for feature detection.

Returns:

self – The fitted estimator with estimated transformation matrix in self.M_.

Return type:

RegEstimator

Raises:

cv2.error – If affine transformation cannot be estimated, typically due to insufficient or poorly matched features.

Notes

The fitting process: 1. Convert images to 8-bit for ORB compatibility 2. Detect up to max_features keypoints in each image 3. Compute ORB descriptors for each keypoint 4. Match descriptors using brute-force Hamming distance 5. Keep top percentile of matches by distance 6. Estimate partial affine transformation using RANSAC

The method uses OpenCV’s estimateAffinePartial2D which finds the optimal similarity transformation (rotation, scaling, translation) rather than a full affine transformation. This is more robust for most registration scenarios.

fit_transform(X: ndarray, y: ndarray) → ndarray[source]

Fit to data, then transform it.

This convenience method combines fitting and transformation in a single call. It’s equivalent to calling fit(X, y).transform(X, y) but slightly more efficient.

Parameters:

  • X (np.ndarray) – Source image to fit the transformation to and then transform.

  • y (np.ndarray) – Target image to fit the transformation against. This image will also be included in the output stack.

Returns:

Stacked array containing the transformed X channels followed by the original y channels. Shape is (C_x + C_y, H_y, W_y) where C_x, C_y are the channel counts and H_y, W_y are target dimensions.

Return type:

np.ndarray

score(source: ndarray, target: ndarray) → Dict[str, float][source]

Calculate registration quality metrics between source and target images.

This method computes various metrics to assess the quality of registration between binary or continuous-valued images. Metrics are calculated only in regions where both images have valid data after transformation.

Parameters:

  • source (np.ndarray) – Source image, shape (H, W). Should be the same image used for fitting or a similar image from the same modality.

  • target (np.ndarray) – Target image, shape (H, W). Should be the same image used for fitting or a similar image from the same modality.

Returns:

Dictionary containing registration quality metrics:

  • ’and’float

    Count of pixels where both source and target are positive (logical AND operation). Higher values indicate better overlap.

  • ’or’float

    Count of pixels where either source or target is positive (logical OR operation).

  • ’xor’float

    Count of pixels where source and target disagree (logical XOR operation). Lower values indicate better agreement.

  • ’iou’float

    Intersection over Union ratio (and/or). Values range from 0-1 with 1 indicating perfect overlap. Returns 0.0 if no positive pixels exist.

  • ’source_sum’float

    Sum of all source pixel intensities in the overlap region.

  • ’target_sum’float

    Sum of all target pixel intensities in the overlap region.

Return type:

Dict[str, float]

Notes

Metrics are computed only in the intersection region where both images have valid data after transformation. This ensures fair comparison and avoids edge effects from the transformation.

For binary images, the metrics have intuitive interpretations: - IoU is the standard Jaccard index - ‘and’ counts overlapping positive pixels - ‘xor’ counts disagreement pixels

For continuous images, the logical operations are applied after implicit conversion to boolean (non-zero values are True).

set_score_request(*, source: bool | None | str = '$UNCHANGED$', target: bool | None | str = '$UNCHANGED$') → RegEstimator

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to score.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:

  • source (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for source parameter in score.

  • target (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for target parameter in score.

Returns:

self – The updated object.

Return type:

object

transform(X: ndarray, y: ndarray | None = None) → ndarray[source]

Apply the estimated transformation to the source image(s).

This method transforms the source image(s) using the affine transformation estimated during fitting. Optionally, a target image can be provided which will be stacked with the transformed source without transformation.

Parameters:

  • X (np.ndarray) – Source image(s) to transform. Shape can be (H, W) for single-channel or (C, H, W) for multi-channel. All channels are transformed using the same transformation matrix.

  • y (Optional[np.ndarray], default=None) – Target image to stack with transformed source. If provided, this image is NOT transformed but is included in the output for direct comparison. Shape should be (H, W) or (C, H, W).

Returns:

Transformed image(s). If y is None, returns transformed X with shape (C, H_out, W_out) where H_out, W_out match the target image from fitting. If y is provided, returns stacked array with transformed X channels followed by untransformed y channels.

Return type:

np.ndarray

Raises:

  • NotFittedError – If transform is called before fitting the estimator.

  • AssertionError – If the stored transformation matrix has invalid shape.

Notes

The transformation uses scikit-image’s warp function with the inverse transformation matrix. This ensures proper interpolation and handles edge cases automatically.

Output image dimensions match the target image used during fitting unless a different y image is provided during transformation.