Welcome to elasticdeform’s documentation!¶

Elastic deformations for N-dimensional images (Python, SciPy, NumPy, TensorFlow, PyTorch)¶

This library implements elastic grid-based deformations for N-dimensional images.

The elastic deformation approach is described in

Ronneberger, Fischer, and Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation” (https://arxiv.org/abs/1505.04597)
Çiçek et al., “3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation” (https://arxiv.org/abs/1606.06650)

The procedure generates a coarse displacement grid with a random displacement for each grid point. This grid is then interpolated to compute a displacement for each pixel in the input image. The input image is then deformed using the displacement vectors and a spline interpolation.

In addition to the normal, forward deformation, this package also provides a function that can backpropagate the gradient through the deformation. This makes it possible to use the deformation as a layer in a convolutional neural network. For convenience, TensorFlow and PyTorch wrappers are provided in elasticdeform.tf and elasticdeform.torch.

Installation¶

pip install elasticdeform
or
pip install git+https://github.com/gvtulder/elasticdeform

This library requires Python 3 and NumPy development headers.

On Windows, try to install the precompiled binaries directly using pip install elasticdeform. If that does not work, these precompiled packages might be an alternative option.

Examples¶

This basic example deforms an image with a random 3 x 3 deformation grid:

import numpy, imageio, elasticdeform
X = numpy.zeros((200, 300))
X[::10, ::10] = 1

# apply deformation with a random 3 x 3 grid
X_deformed = elasticdeform.deform_random_grid(X, sigma=25, points=3)

imageio.imsave('test_X.png', X)
imageio.imsave('test_X_deformed.png', X_deformed)

Multiple inputs¶

If you have multiple images, e.g., an image and a segmentation image, you can deform both simultaneously by providing a list of inputs. You can specify a different spline order for each input.

# apply deformation to inputs X and Y
[X_deformed, Y_deformed] = elasticdeform.deform_random_grid([X, Y])

# apply deformation to inputs X and Y,
# with a different interpolation for each input
[X_deformed, Y_deformed] = elasticdeform.deform_random_grid([X, Y], order=[3, 0])

Multi-channel images¶

By default, a deformation will be applied to every dimension of the input. If you have multi-channel images, you can use the axis parameter to specify which axes should be deformed. The same deformation will be applied for each channel.

For example, to deform an RGB image across the first two dimensions, run:

X_deformed = elasticdeform.deform_random_grid(X, axis=(0, 1))

When deforming multiple inputs, you can provide a tuple of axes for each input:

X = numpy.random.rand(3, 200, 300)
Y = numpy.random.rand(200, 300)
[X_deformed, Y_deformed] = elasticdeform.deform_random_grid([X, Y], axis=[(1, 2), (0, 1)])

Cropping¶

If you intend to crop a small subpatch from the deformed image, you can provide the crop dimensions to the deform function. It will then compute only the cropped output pixels, while still computing the deformation grid based on the full image dimensions. This saves computation time.

X = numpy.random.rand(200, 300)

# define a crop region
crop = (slice(50, 150), slice(0, 100))

# generate a deformation grid
displacement = numpy.random.randn(2, 3, 3) * 25

# deform full image
X_deformed = elasticdeform.deform_grid(X, displacement)
# compute only the cropped region
X_deformed_crop = elasticdeform.deform_grid(X, displacement, crop=crop)

# the deformation is the same
numpy.testing.assert_equal(X_deformed[crop], X_deformed_crop)

Rotate and zoom¶

The deformation functions accept rotate and zoom parameters, which allows you to combine the elastic deformation with rotation and scaling. This can be useful as data augmentation step. The rotation and zoom are applied to the output coordinates, using the center pixel of the output patch as the origin.

# apply deformation with a random 3 x 3 grid,
# rotate by 30 degrees and rescale with a factor 1.5
X_deformed = elasticdeform.deform_random_grid(X, sigma=25, points=3,
                                              rotate=30, zoom=1.5)

Note that the output shape remains the same. The mapping of the input to the output is rotated within the given output frame.

Rotate and zoom can be combined with the crop argument. In that case, the scaling and rotation is performed relative to the center of the cropped output.

For more advanced transformations, it is also possible to provide an affine transformation matrix directly.

Gradient¶

The deform_grid_gradient function can be used to backpropagate the gradient of the output with respect to the input. Call deform_grid_gradient with the parameters that were used for the forward step.

X = numpy.random.rand(200, 300)

# generate a deformation grid
displacement = numpy.random.randn(2, 3, 3) * 25

# perform forward deformation
X_deformed = elasticdeform.deform_grid(X, displacement)

# obtain the gradient w.r.t. X_deformed (e.g., with backpropagation)
dX_deformed = numpy.random.randn(*X_deformed.shape)

# compute the gradient w.r.t. X
dX = elasticdeform.deform_grid_gradient(dX_deformed, displacement)

Note: The gradient function will assume that the input has the same size as the output. If you used the crop parameter in the forward phase, it is necessary to provide the gradient function with the original, uncropped input shape in the X_shape parameter.

TensorFlow wrapper¶

The elasticdeform.tf module provides a wrapper for deform_grid in TensorFlow. The function uses TensorFlow Tensors as input and output, but otherwise uses the same parameters.

import numpy
import elasticdeform.tf as etf

displacement_val = numpy.random.randn(2, 3, 3) * 5
X_val = numpy.random.rand(200, 300)
dY_val = numpy.random.rand(200, 300)

# construct TensorFlow input and top gradient
displacement = tf.Variable(displacement_val)
X = tf.Variable(X_val)
dY = tf.Variable(dY_val)

# the deform_grid function is similar to the plain Python equivalent,
# but it accepts and returns TensorFlow Tensors
X_deformed = etf.deform_grid(X, displacement, order=3)

# the gradient w.r.t. X can be computed in the normal TensorFlow manner
[dX] = tf.gradients(X_deformed, X, dY)

PyTorch wrapper¶

The elasticdeform.torch module provides a wrapper for deform_grid in PyTorch. The function uses PyTorch Tensors as input and output, but otherwise uses the same parameters.

import numpy
import elasticdeform.torch as etorch

displacement_val = numpy.random.randn(2, 3, 3) * 5
X_val = numpy.random.rand(200, 300)
dY_val = numpy.random.rand(200, 300)

# construct PyTorch input and top gradient
displacement = torch.tensor(displacement_val)
X = torch.tensor(X_val, requires_grad=True)
dY = torch.tensor(dY_val)

# the deform_grid function is similar to the plain Python equivalent,
# but it accepts and returns PyTorch Tensors
X_deformed = etorch.deform_grid(X, displacement, order=3)

# the gradient w.r.t. X can be computed in the normal PyTorch manner
X_deformed.backward(dY)
print(X.grad)

License information¶

This library was written by Gijs van Tulder at the Biomedical Imaging Group Rotterdam, Erasmus MC, Rotterdam, the Netherlands

It is inspired by a similar, Python-based implementation by Florian Calvet. This C-based implementation gives the same results, but is faster and has a gradient implementation.

This C implementation includes a modified version of the NI_GeometricTransform from SciPy’s ndimage library.

This code is made available under the BSD license. See LICENSE.txt for details.

If you want to cite this library, please see DOI.

elasticdeform¶

Deformation¶

elasticdeform.deform_random_grid(X, sigma=25, points=3, order=3, mode='constant', cval=0.0, crop=None, prefilter=True, axis=None, affine=None, rotate=None, zoom=None)[source]¶

Elastic deformation with a random deformation grid

This generates a random, square deformation grid with displacements sampled from from a normal distribution with standard deviation sigma. The deformation is then applied to the image or list of images,

See deform_grid for a full description of the parameters.

Parameters:

X (numpy array or list of arrays) – image, or list of images of the same size
sigma (float) – standard deviation of the normal distribution
points (array) – number of points of the deformation grid
rotate (float or None) – angle in degrees to rotate the output

This only works for 2D images and rotates the image around the center of the output.
zoom (float or None) – scale factor to zoom the output

This only works for 2D images and scales the image around the center of the output.

See also

deform_grid(): for a full description of the parameters.

elasticdeform.deform_grid(X, displacement, order=3, mode='constant', cval=0.0, crop=None, prefilter=True, axis=None, affine=None, rotate=None, zoom=None)[source]¶

Elastic deformation with a deformation grid

The procedure generates a coarse displacement grid with a random displacement for each grid point. This grid is then interpolated to compute a displacement for each pixel in the input image. The input image is then deformed using the displacement vectors and a spline interpolation.

Parameters:	X (numpy array or list of arrays) – image, or list of images of the same size If X is a list of images, the values for order, mode and cval can be lists to specify a different value for every image in X. displacement (numpy array) – displacement vectors for each control point displacement is a NumPy array with displacement vectors for each control points. For example, to deform a 2D image with 3 x 5 control points, provide a displacement matrix of shape 2 x 3 x 5. order ({0, 1, 2, 3, 4}) – interpolation order mode (({nearest, wrap, reflect, mirror, constant})) – border mode cval (float) – constant value to be used if mode == ‘constant’ crop (None or list) – None, or a list of slice() objects to crop the output crop can be a list of slice() objects to crop the output with. Only very simple slicing is supported: the slice start and stop values must be positive and should not be larger than the output. Note that this parameter is dependent of the axis parameter: if an axis list is given, crop must only contain slice() objects for the dimensions in axis. prefilter (bool) – if True the input X will be pre-filtered with a spline filter axis (None, int, a list of ints, or a list of lists of ints) – the axes to deform over axis indicates the axes on which the deformation should be applied. The default (None) is to apply a deformation to all dimensions of the input. Giving a single axis (int) or a tuple of axes will apply the deformation only to those axes. The shape of the displacement must match this number of axes. If multiple inputs are given, axis should be None or a list of tuples with the axes for each input. affine (None, numpy array of shape (ndim, ndim + 1)) – affine transformation to apply to the output The affine transformation is applied to the output during interpolation. rotate (float or None) – angle in degrees to rotate the output This only works for 2D images and rotates the image around the center of the output. zoom (float or None) – scale factor to zoom the output This only works for 2D images and scales the image around the center of the output.
Returns:	The deformed image, or a list of deformed images if a list of inputs is given.
Return type:	numpy array or list of arrays

Notes

See the SciPy documentation for scipy.ndimage.interpolation.map_coordinates for more details on some of the parameters.

The elastic deformation approach is found in

Ronneberger, Fischer, and Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation” https://arxiv.org/abs/1505.04597
Cicek et al., “3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation” https://arxiv.org/abs/1606.06650

Based on a Python implementation by Florian Calvet.

Gradient¶

elasticdeform.deform_grid_gradient(dY, displacement, order=3, mode='constant', cval=0.0, crop=None, prefilter=True, axis=None, X_shape=None, affine=None, rotate=None, zoom=None)[source]¶

Gradient for deform_grid.

This method performs the backward operation that returns the gradient of deform_grid with respect to the input. This is similar to performing an inverse deformation on the gradient, but not exactly the same: this function gives an exact gradient that also takes the interpolation into account.

The X_shape parameter specifices the shape of the original inputs, and is only necessary if the crop parameter is used. Otherwise, the input shape is the same as the shape of the gradient dY.

See the documentation for deform_grid.

Parameters:	dY (numpy array) – the input gradient, or list of gradients of the same size displacement (numpy array) – displacement vectors for each control point order ({0, 1, 2, 3, 4}) – interpolation order mode (({nearest, wrap, reflect, mirror, constant})) – border mode cval (float) – constant value to be used if mode == ‘constant’ crop (None or list) – None, or a list of slice() objects to crop the output prefilter (bool) – if True the input X will be pre-filtered with a spline filter axis (None, int, a list of ints, or a list of lists of ints) – the axes to deform over X_shape (tuple with the shape of the input, or a list of tuples) affine (None, numpy array of shape (ndim, ndim + 1)) – affine transformation to apply to the output rotate (float or None) – angle in degrees to rotate the output zoom (float or None) – scale factor to zoom the output
Returns:	Returns the gradient with respect to X.
Return type:	numpy array

elasticdeform.tf¶

TensorFlow wrapper function¶

elasticdeform.tf.deform_grid(X, displacement, *args, **kwargs)[source]¶

Elastic deformation with a deformation grid, wrapped in a TensorFlow Op.

This function wraps the elasticdeform.deform_grid function in a TensorFlow Op with a custom gradient.

Parameters:	X (Tensor or list of Tensors) – input image or list of input images displacement (Tensor or numpy array) – displacement vectors for each control point
Returns:	the deformed image, or a list of deformed images
Return type:	Tensor

elasticdeform.torch¶

PyTorch wrapper function¶

elasticdeform.torch.deform_grid(X, displacement, *args, **kwargs)[source]¶

Elastic deformation with a deformation grid, wrapped for PyTorch.

This function wraps the elasticdeform.deform_grid function in a PyTorch function with a custom gradient.

Parameters:	X (torch.Tensor or list of torch.Tensors) – input image or list of input images displacement (torch.Tensor) – displacement vectors for each control point
Returns:	the deformed image, or a list of deformed images
Return type:	torch.Tensor

License¶

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

Neither the name of Enthought nor the names of the SciPy Developers may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Welcome to elasticdeform’s documentation!¶

Elastic deformations for N-dimensional images (Python, SciPy, NumPy, TensorFlow, PyTorch)¶

Installation¶

Examples¶

Multiple inputs¶

Multi-channel images¶

Cropping¶

Rotate and zoom¶

Gradient¶

TensorFlow wrapper¶

PyTorch wrapper¶

License information¶

elasticdeform¶

Deformation¶

Gradient¶

elasticdeform.tf¶

TensorFlow wrapper function¶

elasticdeform.torch¶

PyTorch wrapper function¶

License¶

Indices and tables¶