mir.ndslice

Multidimensional Random Access Ranges

The package provides a multidimensional array implementation. It would be well suited to creating machine learning and image processing algorithms, but should also be general enough for use anywhere with homogeneously-typed multidimensional data. In addition, it includes various functions for iteration, accessing, and manipulation.

Modules

allocation
module mir.ndslice.allocation

This is a submodule of mir.ndslice.

chunks
module mir.ndslice.chunks

This is a submodule of mir.ndslice.

concatenation
module mir.ndslice.concatenation

This is a submodule of mir.ndslice.

connect
module mir.ndslice.connect
dynamic
module mir.ndslice.dynamic
field
module mir.ndslice.field

This is a submodule of mir.ndslice.

filling
module mir.ndslice.filling

This is a submodule of mir.ndslice.

fuse
module mir.ndslice.fuse

This is a submodule of mir.ndslice.

iterator
module mir.ndslice.iterator

This is a submodule of mir.ndslice.

mutation
module mir.ndslice.mutation
Multidimensional mutation algorithms
ndfield
module mir.ndslice.ndfield

This is a submodule of mir.ndslice.

slice
module mir.ndslice.slice

This is a submodule of mir.ndslice.

sorting
module mir.ndslice.sorting

This is a submodule of mir.ndslice.

topology
module mir.ndslice.topology

This is a submodule of mir.ndslice.

traits
module mir.ndslice.traits
Multidimensional traits

Public Imports

mir.algorithm.iteration
public import mir.algorithm.iteration;
Undocumented in source.
mir.ndslice.allocation
public import mir.ndslice.allocation;
Undocumented in source.
mir.ndslice.chunks
Chunks!([0], Iterator, N, kind) chunks(Slice!(Iterator, N, kind) slice, size_t chunkLength) via public import mir.ndslice.chunks;

Creates Chunks.

mir.ndslice.concatenation
auto concatenation(Slices slices) via public import mir.ndslice.concatenation;

Creates a Concatenation view of multiple slices.

mir.ndslice.dynamic
public import mir.ndslice.dynamic;
Undocumented in source.
mir.ndslice.field
public import mir.ndslice.field;
Undocumented in source.
mir.ndslice.filling
public import mir.ndslice.filling;
Undocumented in source.
mir.ndslice.fuse
alias fuse(Dimensions...) = fuseImpl!(false, void, Dimensions) via public import mir.ndslice.fuse;

Fuses ndrange r into GC-allocated (fuse) or RC-allocated (rcfuse) ndslice. Can be used to join rows or columns into a matrix.

mir.ndslice.iterator
public import mir.ndslice.iterator;
Undocumented in source.
mir.ndslice.mutation
public import mir.ndslice.mutation;
Undocumented in source.
mir.ndslice.ndfield
public import mir.ndslice.ndfield;
Undocumented in source.
mir.ndslice.slice
auto slice(Concatenation!(dim, Slices) concatenation) via public import mir.ndslice.slice;

GC-Allocates an n-dimensional slice.

mir.ndslice.topology
public import mir.ndslice.topology;
Undocumented in source.
mir.ndslice.traits
public import mir.ndslice.traits;
Undocumented in source.

Detailed Description

Quick Start

sliced is a function designed to create a multidimensional view over a range. Multidimensional view is presented by Slice type.

import mir.ndslice;

auto matrix = slice!double(3, 4);
matrix[] = 0;
matrix.diagonal[] = 1;

auto row = matrix[2];
row[3] = 6;
assert(matrix[2, 3] == 6); // D & C index order

Note: In many examples mir._ndslice.topology.iota is used instead of a regular array, which makes it possible to carry out tests without memory allocation.

SubmoduleDeclarations
slice
slice, Slice structure Basic constructors
Canonical Contiguous DeepElementType isSlice kindOf Slice sliced slicedField slicedNdField SliceKind Structure Universal
allocation
Allocation utilities
bitRcslice bitSlice makeNdarray makeSlice makeUninitSlice mininitRcslice ndarray rcslice shape slice stdcFreeAlignedSlice stdcFreeSlice stdcSlice stdcUninitAlignedSlice stdcUninitSlice uninitAlignedSlice uninitSlice
topology
Subspace manipulations Advanced constructors SliceKind conversion utilities
alongDim as asKindOf assumeCanonical assumeContiguous assumeHypercube assumeSameShape bitpack bitwise blocks byDim bytegroup cached cachedGC canonical cartesian chopped cycle diagonal diff dropBorders evertPack flattened indexed iota ipack kronecker linspace magic map member ndiota orthogonalReduceField pack pairwise repeat reshape ReshapeError retro slide slideAlong squeeze stairs stride subSlices triplets universal unsqueeze unzip vmap windows zip
filling
Specialized initialisation routines
fillVandermonde
fuse
Data fusing (stacking) See also concatenation submodule.
fuse fuseAs rcfuse rcfuseAs fuseCells
concatenation
Concatenation, padding, and algorithms See also fuse submodule.
forEachFragment isConcatenation pad padEdge padWrap padSymmetric concatenation Concatenation concatenationDimension until
dynamic
Dynamic dimension manipulators
allReversed dropToHypercube everted normalizeStructure reversed rotated strided swapped transposed
sorting
Sorting utilities
sort Examples for isSorted, isStrictlyMonotonic, makeIndex, and schwartzSort.
mutation
Mutation utilities
copyMinor reverseInPlace
iterator
Declarations
BytegroupIterator CachedIterator ChopIterator FieldIterator FlattenedIterator IndexIterator IotaIterator MapIterator MemberIterator RetroIterator SliceIterator SlideIterator StairsIterator StrideIterator SubSliceIterator Triplet TripletIterator ZipIterator
field
Declarations
BitField BitpackField CycleField LinspaceField MagicField MapField ndIotaField OrthogonalReduceField RepeatField
ndfield
Declarations
Cartesian Kronecker
chunks
Declarations
chunks Chunks isChunks popFrontTuple
traits
Declarations
isIterator isVector isMatrix isContiguousSlice isCanonicalSlice isUniversalSlice isContiguousVector isUniversalVector isContiguousMatrix isCanonicalMatrix isUniversalMatrix

Example: Image Processing

A median filter is implemented as an example. The function movingWindowByChannel can also be used with other filters that use a sliding window as the argument, in particular with convolution matrices such as the Sobel operator.

movingWindowByChannel iterates over an image in sliding window mode. Each window is transferred to a filter, which calculates the value of the pixel that corresponds to the given window.

This function does not calculate border cases in which a window overlaps the image partially. However, the function can still be used to carry out such calculations. That can be done by creating an amplified image, with the edges reflected from the original image, and then applying the given function to the new file.

Note: You can find the example at GitHub.

/++
Params:
    filter = unary function. Dimension window 2D is the argument.
    image = image dimensions `(h, w, c)`,
        where с is the number of channels in the image
    nr = number of rows in the window
    nс = number of columns in the window

Returns:
    image dimensions `(h - nr + 1, w - nc + 1, c)`,
        where с is the number of channels in the image.
        Dense data layout is guaranteed.
+/
Slice!(C*, 3) movingWindowByChannel(alias filter, C)
(Slice!(C*, 3, Universal) image, size_t nr, size_t nc)
{
        // 0. 3D
        // The last dimension represents the color channel.
    return image
        // 1. 2D composed of 1D
        // Packs the last dimension.
        .pack!1
        // 2. 2D composed of 2D composed of 1D
        // Splits image into overlapping windows.
        .windows(nr, nc)
        // 3. 5D
        // Unpacks the windows.
        .unpack.unpack
        // 4. 5D
        // Brings the color channel dimension to the third position.
        .transposed!(0, 1, 4)
        // 5. 3D Composed of 2D
        // Packs the last two dimensions.
        .pack!2
        // 2D to pixel lazy conversion.
        .map!filter
        // Creates the new image. The only memory allocation in this function.
        .slice;
}

A function that calculates the value of iterator median is also necessary.

/++

Params:
    r = input range
    buf = buffer with length no less than the number of elements in `r`
Returns:
    median value over the range `r`
+/
T median(Iterator, SliceKind kind, T)(Slice!(Iterator, 2, kind) sl, T[] buf)
{
    import std.algorithm.sorting : topN;
    // copy sl to the buffer
    auto retPtr = reduce!(
        (ptr, elem) {
            *ptr = elem;
            return ptr + 1;
        } )(buf.ptr, sl);
    auto n = retPtr - buf.ptr;
    buf[0 .. n].topN(n / 2);
    return buf[n / 2];
}

The main function:

void main(string[] args)
{
    import std.conv: to;
    import std.getopt: getopt, defaultGetoptPrinter;
    import std.path: stripExtension;

    uint nr, nc, def = 3;
    auto helpInformation = args.getopt(
        "nr", "number of rows in window, default value is " ~ def.to!string, &nr,
        "nc", "number of columns in window, default value is equal to nr", &nc);
    if (helpInformation.helpWanted)
    {
        defaultGetoptPrinter(
            "Usage: median-filter [<options...>] [<file_names...>]\noptions:",
            helpInformation.options);
        return;
    }
    if (!nr) nr = def;
    if (!nc) nc = nr;

    auto buf = new ubyte[nr * nc];

    if (args.length == 1)
    {
        import std.stdio: writeln;
        writeln("No input file given");
    }

    foreach (name; args[1 .. $])
    {
        import imageformats; // can be found at code.dlang.org

        IFImage image = read_image(name);

        auto ret = image.pixels
            .sliced(cast(size_t)image.h, cast(size_t)image.w, cast(size_t)image.c)
            .universal
            .movingWindowByChannel
                !(window => median(window, buf))
                 (nr, nc);

        write_image(
            name.stripExtension ~ "_filtered.png",
            ret.length!1,
            ret.length!0,
            ret.field);
    }
}

This program works both with color and grayscale images.

$ median-filter --help
Usage: median-filter [<options...>] [<file_names...>]
options:

--nc number of columns in window default value equals to nr -h --help This help information.

Compared with numpy.ndarray

numpy is undoubtedly one of the most effective software packages that has facilitated the work of many engineers and scientists. However, due to the specifics of implementation of Python, a programmer who wishes to use the functions not represented in numpy may find that the built-in functions implemented specifically for numpy are not enough, and their Python implementations work at a very low speed. Extending numpy can be done, but is somewhat laborious as even the most basic numpy functions that refer directly to ndarray data must be implemented in C for reasonable performance.

At the same time, while working with ndslice, an engineer has access to the whole set of standard D library, so the functions he creates will be as efficient as if they were written in C.

Meta

Authors

Ilia Ki Acknowledgements: John Loughran Colvin