CNN -> Data processing

Image patching - Image trimming

tardis_em.cnn.data_processing.trim.trim_with_stride(image: ndarray, trim_size_xy: int, trim_size_z: int, output: str, image_counter: int, scale: list, clean_empty=True, keep_if=0.01, stride=25, mask: ndarray | None = None, log=True, pixel_size=None, device=device(type='cpu'))

Trim a 2D/3D image with optional masking into smaller patches with a specific stride and size. This function is designed for preprocessing image data for machine learning tasks, ensuring that the image dimensions meet specified requirements or adjusting them when necessary.

Parameters:

  • image – The input image array, which can be 2D or 3D, and include grayscale or RGB channels. The data type and shape may vary depending on the input.

  • trim_size_xy – Desired size for trimming in the X and Y dimensions. The input value determines the width and height of each patch. If not provided, default values will be calculated.

  • trim_size_z – Desired size for trimming in the Z dimension. The input value determines the depth of each patch in case of 3D data. If not provided, default values will be calculated.

  • output – Directory path where the trimmed patches will be saved. Output images and masks are stored in subdirectories.

  • image_counter – Counter used for naming the output files. This ensures unique filenames during the trimming process.

  • scale – A scaling factor list that adjusts the image size before trimming is performed.

  • clean_empty – When set to True, automatic filtering is applied to exclude patches considered empty based on a specific condition.

  • keep_if – Minimum threshold for keeping patches. The condition ensures that patches with non-empty values above this percentage are retained.

  • stride – The stride size for trimming patches. Stride determines the overlap between adjacent patches when slicing the image.

  • mask – Optional mask array corresponding to the input image. Used for segmentation or region-specific analyses. The function scales and trims the mask alongside the image.

  • log – Flag to enable or disable logging of the process; logs errors and important actions during execution when set to True.

  • pixel_size – Indicates whether the output should be saved in ‘.mrc’ (specific file format) or ‘.tif’ (common image format). Leave None for default behavior.

  • device – Specifies the computational device (CPU, GPU, etc.) to be used if scaling operations employ device-specific optimizations.

Returns:

Returns None. Side effect: Saves the image patches and corresponding mask patches (when provided) to the specified output directory. Each patch is named according to its positional indices to maintain unique identification.

tardis_em.cnn.data_processing.trim.trim_label_mask(points: ndarray, image: ndarray, label_mask: ndarray) → Tuple[ndarray, ndarray, ndarray]

Trim the label mask, image, and adjust the point coordinates based on the minimum and maximum boundaries in the provided points. The function calculates the bounding box using the points array, crops the image and label mask within this bounding box, and shifts the points coordinates accordingly.

Parameters:

  • points – A numpy array of shape (N, 3) representing the coordinates of points.

  • image – A 3D numpy array representing the volumetric image.

  • label_mask – A 3D numpy array representing the volumetric label mask.

Returns:

A tuple containing the trimmed image array, the trimmed label_mask array, and the adjusted points array.

Return type:

Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray]

Image patching - Image stitching

class tardis_em.cnn.data_processing.stitch.StitchImages

A class for stitching multiple image patches into a single image.

This class is utilized to process and combine multiple small image patches into a large cohesive image. The images are expected to follow a specific naming convention that includes indices and dimensions, which the class uses for computation. This functionality is useful in domains such as image processing and machine learning, where use of segmented image parts during preprocessing and their subsequent recombination is common.

tardis_em.cnn.data_processing.stitch.generate_grid(image_size: tuple, patch_size: list, grid_size: list, stride: int)

Generates grid coordinates for a given image, with support for both 2D and 3D images. The method computes overlapping patch regions for an image based on the specified patch size, grid size, and stride. For 2D images, the function generates grid points for rows and columns, and for 3D images, it extends the functionality by incorporating depth information. Output grid coordinates serve as indexing patterns for segmenting images into smaller patches or for analyzing specific grid areas.

Parameters:

  • image_size – Tuple representing dimensions of the image. For a 2D image, it takes the form (height, width). For a 3D image, it takes the form (depth, height, width).

  • patch_size – List representing the size of the patches for either a 2D or 3D image. For a 2D image, it should be [patch_height, patch_width]. For a 3D image, [patch_depth, patch_height, patch_width].

  • grid_size – List defining the number of grid steps along each dimension. For a 2D image, it is [grid_rows, grid_columns]. For a 3D image, it is [grid_depth, grid_rows, grid_columns].

  • stride – Integer specifying the step size or overlap between the patch regions.

Returns:

If the input image is 2D, returns two arrays containing the y-coordinates and x-coordinates of the grid. If the input image is 3D, returns three arrays: (1) a 2D array of coordinates for depth and rows, (2) a 2D array of coordinates for depth and columns, and (3) a 1D array of z-coordinates grid.

Utils

tardis_em.cnn.data_processing.interpolation.interpolate_generator(points: ndarray) → Iterable

Generates interpolated points between two given 2D or 3D points. The function determines the interpolation path in a pixel-grid-like manner between the start and end points, based on Bresenham-like algorithms. Interpolation supports only 2D and 3D coordinate systems, and exactly two points must be provided for interpolation.

Parameters:

points – An array of shape (2, 2) for 2D points or (2, 3) for 3D points. Defines the two points between which interpolation is performed.

Returns:

An iterable generator that yields tuples representing (x, y) in 2D space and (x, y, z) in 3D space.

tardis_em.cnn.data_processing.interpolation.interpolation(points: ndarray) → ndarray

Performs 3D interpolation for XYZ coordinates using a given interpolation generator over sequential point pairs, and appends the last point to the output in integer format. The function creates a continuous series of interpolated points connecting all input coordinates.

Parameters:

points – An array of shape (N, 3) representing a sequence of 3D points.

Returns:

An array of shape (M, 3) containing the interpolated 3D coordinates, where M >= N due to newly generated intermediate points.

tardis_em.cnn.data_processing.draw_mask.draw_instances(mask_size: list | tuple, coordinate: ndarray, pixel_size: float, circle_size=250, label=True, dtype=None) → ndarray

Draws labeled or binary masks based on the input coordinates, mask size, and additional parameters.

This function generates either 2D or 3D masks, depending on the shape of the provided coordinates. If label generation is enabled, unique segment labels are created to distinguish between different segments. Circles centered around specific points are drawn in the masks, with sizes determined by the input circle size and pixel size. Input parameters related to the shape, size, and type of the mask, as well as the labeling behavior, are fully customizable. This function is suitable for constructing semantic or instance masks.

Parameters:

  • mask_size (list | tuple) – The dimensions of the mask to be created.

  • coordinate (np.ndarray) – An array of coordinates specifying the locations to draw the mask.

  • pixel_size (float) – Size of a pixel in the mask, used to scale the mask appropriately.

  • circle_size (int, optional) – Diameter of the circle to be drawn at each coordinate point. Defaults to 250.

  • label (bool, optional) – Flag indicating whether a labeled mask or a binary mask should be created. Defaults to True.

  • dtype (str | None, optional) – Data type for the output mask. If not provided, defaults to np.uint16 or np.uint8.

Returns:

A 2D or 3D mask generated based on the input parameters.

Return type:

np.ndarray

tardis_em.cnn.data_processing.draw_mask.draw_semantic_membrane(mask_size: tuple, coordinate: ndarray, pixel_size: float, spline_size=70) → ndarray

Draws a semantic membrane mask based on given coordinates, pixel size, and spline size. The function generates a mask of the specified size and marks regions centered around the provided coordinates. The region size is determined by the given spline size, adjusted by the pixel size.

Parameters:

  • mask_size (tuple) – The dimensions of the output mask, specified as a tuple of integers (z, y, x).

  • coordinate (np.ndarray) – A numpy array of coordinates specifying the center points of each region to be drawn in the mask.

  • pixel_size (float) – Resolution of the mask, defining the relationship between spline size and the mask grid.

  • spline_size (int, optional) – The diameter of the spline in real-world size, measured in the same unit as pixel_size. Default is 70.

Returns:

A numpy ndarray representing the mask, where regions centered at the provided coordinates are filled.

Return type:

np.ndarray

tardis_em.cnn.data_processing.draw_mask.draw_instances_membrane(mask_size: tuple, coordinate: ndarray, pixel_size: float, spline_size=70) → ndarray

Draws the instances of membranes on a specified mask by iterating through given coordinates, creating circular “stamp”-like regions based on the spline size and pixel size, and assigning unique identifiers to each region in the mask.

The mask is a 3D array where each element corresponds to a region assigned to a specific identifier. The process converts coordinates to integers, calculates a radius from the given spline and pixel sizes, and applies the circular regions to the mask for each set of input coordinates.

Parameters:

  • mask_size – A tuple representing the size of the mask, typically in 3D, that specifies its spatial dimensions.

  • coordinate – A 2D numpy array where each row corresponds to an instance, containing the identifier in the first column and the coordinates in the remaining columns.

  • pixel_size – A float representing the physical distance represented by a single pixel in the spatial dimensions of the mask.

  • spline_size – An integer specifying the base size of the “stamped” mask regions. Optional, default value is 70.

Returns:

A 3D numpy array with the same dimensions as the input mask, containing unique identifiers based on the given coordinate for each “stamped” membrane region.

Return type:

numpy.ndarray

tardis_em.cnn.data_processing.draw_mask.draw_mask(r: int, c: ndarray, label_mask: ndarray, segment_shape: str) → Tuple[ndarray, ndarray] | Tuple[ndarray, ndarray, ndarray]

Draws a mask on the given label array by creating a circle or sphere depending on the specified shape. The method can handle both 2D and 3D label masks.

Parameters:

  • r – The radius of the circle (2D) or sphere (3D) to be drawn.

  • c – A numpy array containing the center coordinates of the shape, where the length of the input determines if it is 2D or 3D.

  • label_mask – A numpy array representing the label mask where the circle or sphere will be drawn. Must have 2 or 3 dimensions.

  • segment_shape – A string specifying the shape to draw; “c” for circle (default) or “s” for sphere.

Returns:

A tuple of numpy arrays representing the coordinates of the drawn mask shape. For 3D shapes, the tuple will include z, y, and x arrays. For 2D shapes, the tuple will include y and x arrays, omitting the z array.

tardis_em.cnn.data_processing.draw_mask.draw_circle(r: int, c: tuple, shape: tuple) → Tuple[ndarray, ndarray] | Tuple[ndarray, ndarray, ndarray]

Generates the coordinates of pixels to draw a filled circle in a given array shape. Depending on the input c (2D or 3D), it generates either 2D (y, x) or 3D (z, y, x) coordinates restricted within the specified shape.

The method uses the skimage.draw.disk function to calculate the circular region and adjusts for its position in the provided coordinate frame c. It ensures that all pixels remain within the bounds of the array defined by shape.

Parameters:

  • r – Radius of the circle to be drawn.

  • c – Tuple representing the center coordinates of the circle. In 2D, it requires (y, x), and for 3D, it requires (z, y, x).

  • shape – Tuple representing the shape of the frame or array where the circle needs to be drawn. For 2D: (rows, columns). For 3D: (depth, rows, columns).

Returns:

Coordinates of pixels forming the circle. A tuple of arrays (y, x) for 2D or (z, y, x) for 3D representing the indices of the circle’s pixels.

tardis_em.cnn.data_processing.draw_mask.draw_sphere(r: int, c: tuple, shape: tuple) → Tuple[ndarray, ndarray, ndarray]

Generates the coordinates of a 3D spherical structure within a volume of specified shape. This function simulates a sphere using a 3D binary array, applies trimming to the sphere, and shifts its position within a given coordinate frame. The output consists of the adjusted coordinates of the sphere within the specified 3D volume.

Parameters:

  • r (int) – Radius of the sphere

  • c (tuple) – Center coordinates of the sphere in the 3D volume

  • shape (tuple) – Shape of the 3D volume to contain the sphere

Returns:

Coordinates of the sphere within the 3D volume along z, y, and x axes

Return type:

tuple of numpy.ndarray

tardis_em.cnn.data_processing.scaling.scale_image(scale: tuple, image: ndarray | None = None, mask: ndarray | None = None, device=device(type='cpu')) → Tuple[ndarray, ndarray, int] | Tuple[ndarray, int] | Tuple[None, int]

Scales the input image and/or mask to the specified dimensions. Depending on the sizes given in scale compared to the original dimensions of the image or mask, down-sampling or up-sampling is applied. When both image and mask are provided, both are processed and returned along with the dimensional value. If only one of them (image or mask) is provided, it is scaled and returned with the dimensional value. If neither is provided, a None result is returned with the dimensional value.

Parameters:

  • scale – A tuple of integers specifying the desired dimensions for scaling. Can be of length 2 (height and width) or 3 (height, width, and depth).

  • image – Optional. A NumPy array representing the image to be scaled. If provided, it is processed and scaled according to the provided scale.

  • mask – Optional. A NumPy array representing the mask to be scaled. If provided, it is processed and scaled according to the provided scale.

  • device – An optional argument specifying the device to perform scaling operations (e.g., torch.device(‘cpu’) or torch.device(‘cuda’)).

Returns:

A tuple containing either: - (image, mask, dim) if both image and mask are provided and processed. - (image, dim) if only image is provided and processed. - (mask, dim) if only mask is provided and processed. - (None, dim) if neither image nor mask are provided.

tardis_em.cnn.data_processing.scaling.nn_scaling(img: ndarray, scale: tuple, device=device(type='cpu'), gauss=False) → ndarray

Scale an image using nearest-neighbor interpolation, with an optional Gaussian filter applied before scaling.

This function performs image resizing using PyTorch’s interpolation method with mode set to “nearest”. If the gauss parameter is set to True, a Gaussian filter is applied before resizing to smoothen the image.

Parameters:

  • img – Input image as a NumPy array.

  • scale – New dimensions for the image as a tuple (height, width).

  • device – PyTorch device to perform operations on. Defaults to “cpu”.

  • gauss – Optional boolean flag to apply Gaussian filtering before scaling. Defaults to False.

Returns:

Rescaled image as a NumPy array.

tardis_em.cnn.data_processing.scaling.linear_scaling(img: ndarray, scale: tuple, device=device(type='cpu')) → ndarray

Scales a 2D or 3D image to a desired resolution using trilinear or bilinear interpolation. The function performs the scaling in multiple steps, doubling or halving dimensions at each step, until the desired resolution is achieved or exceeded.

Parameters:

  • img – Input image to be scaled. Can be 2D or 3D. Should be provided as a NumPy array.

  • scale – Target resolution as a tuple of integers representing the desired dimensions. Must correspond to either (depth, height, width) for 3D images or (height, width) for 2D images.

  • device – Device to use for the tensor operations (default is torch.device(“cpu”)). It accepts GPU devices (e.g., torch.device(“cuda”)).

Returns:

A NumPy array representing the resized image, scaled to the target resolution.

tardis_em.cnn.data_processing.scaling.area_scaling(img: ndarray, scale: tuple, device=device(type='cpu')) → ndarray

Scales a 3D image array to a specified size along all axes using area-based interpolation. The function uses PyTorch’s interpolation functionality to rescale the image accurately. This process first adjusts the Z-axis size, then scales along the XY plane to match the target scale. The function assumes input and output images to be 3D arrays. It takes a device parameter that leverages GPU acceleration if specified.

Parameters:

  • img (np.ndarray) – Input 3D numpy array representing the image data to be scaled.

  • scale (tuple) – A tuple specifying the desired size of the scaled image (depth, height, width).

  • device (torch.device) – The computation device to perform scaling. Defaults to CPU.

Returns:

A scaled 3D numpy array.

Return type:

np.ndarray

tardis_em.cnn.data_processing.scaling.fourier_scaling(img: ndarray, scale: tuple, device=device(type='cpu')) → ndarray

Rescales the given image in the frequency domain to the specified spatial dimensions using Fourier Transform. The function accepts an image as a NumPy array, performs Fourier Transform, adjusts the frequency components to fit the target scale, and reconstructs the resized image via inverse Fourier Transform. The scaling is performed directly in the frequency domain, which helps preserve spatial details in the resized image.

Parameters:

  • img (np.ndarray) – 2D or 3D image array to be processed.

  • scale (tuple) – Target dimensions for scaling the image. Must be a tuple.

  • device (torch.device) – Computing device to perform the Fourier Transform operations.

Returns:

Resized image array with dimensions matching the specified scale.

Return type:

np.ndarray