Pipelines

Videos loaded by pims (FramesSequence objects) are like lists of numpy arrays. Unlike Python lists of arrays they are “lazy”, they only load the data from the harddrive when it is necessary.

In order to modify a FramesSequence, for instance to convert RGB color videos to grayscale, one could load all the video frames in memory and do the conversion. This however costs a lot of time and memory, and for very large videos this is just not feasible. To solve this problem, PIMS uses so-called pipeline decorators from a sister project called slicerator. A pipeline-decorated function is only evaluated when needed, so that the underlying video data is only accessed one element at a time.

Note

This supersedes the process_func and as_grey reader keyword arguments starting from PIMS v0.4

Conversion to greyscale

Say we want to convert an RGB video to greyscale. A pipeline to do this is already provided as pims.as_grey, but it is also easy to make our own. We define a function as follows and decorate it with @pipeline to turn it into a pipeline:

@pims.pipeline
def as_grey(frame):
    red = frame[:, :, 0]
    green = frame[:, :, 1]
    blue = frame[:, :, 2]
    return 0.2125 * red + 0.7154 * green + 0.0721 * blue

The behavior of as_grey is unchanged if it is used on a single frame:

In [1]: frame = video[0]

In [2]: print(frame.shape)   # the shape of the example video is RGB
(256, 256, 3)

In [3]: processed_frame = as_grey(frame)

In [4]: print(processed_frame.shape)  # the converted frame is indeed greyscale
(256, 256)

However, the @pipeline decorator enables lazy evaluation of full videos:

In [5]: processed_video = as_grey(video)  # this would not be possible without @pipeline

# nothing has been converted yet!
In [6]: processed_frame = processed_video[0]  # now the conversion takes place

In [7]: print(processed_frame.shape)
(256, 256)

This means that the modified video can be used exactly as you would use the original one. In most cases, it will look as though you are accessing a grayscale video file, even though the file on disk is still in color. Please keep in mind that these simple pipelines do not change the reader properties, such as video.frame_shape.

Propagating metadata properly through pipelines is partly implemented, but currently still experimental. For a detailed description of this tricky point, please consult this discussion on GitHub.

Cropping

Along with the built-in pims.as_grey pipeline that saves you from typing out the previous example, there’s also a pims.process.crop pipeline that _does_ change frame_shape. This example takes the video we had converted to grayscale in the previous example, and removes 15 pixels from the left side of each image:

In [8]: print(video.frame_shape)
(256, 256, 3)

# Because this is a color video, we need 3 pairs of cropping parameters
In [9]: cropped_video = pims.process.crop(video, ((0, 0), (15, 0), (0, 0)) )

In [10]: print(cropped_video.frame_shape)
(256, 241, 3)

In [11]: cropped_frame = cropped_video[0]  # now the cropping happens

In [12]: print(cropped_frame.shape)
(256, 241, 3)

Converting existing functions to a pipeline

We are now going to do the same greyscale conversion as above, but using an existing function from skimage:

In [13]: from skimage.color import rgb2gray

In [14]: rgb2gray_pipeline = pims.pipeline(rgb2gray)

In [15]: processed_video = rgb2gray_pipeline(video)

In [16]: processed_frame = processed_video[0]

In [17]: print(processed_frame.shape)
(256, 256)

Any function that takes a single frame and returns a single frame can be converted into a pipeline in this way.

Dtype conversion using lambda functions

Note

This supersedes the dtype reader keyword argument starting from PIMS v0.4

We are now going to convert the data type of a video to float using an unnamed lambda function in a single line:

In [18]: processed_video = pims.pipeline(lambda x: x.astype(np.float))(video)

In [19]: processed_frame = processed_video[0]

In [20]: print(processed_frame.shape)
(256, 256, 3)