Layout API overview¶

Introduction¶

In few words, with layout NCHW it is easier to understand what model’s shape {8, 3, 224, 224} means. Without layout it is just a 4-dimensional tensor.

Concept of layout helps you (and your application) to understand what does each particular dimension of input/output tensor mean. For example, if your input has shape {1, 3, 720, 1280} and layout “NCHW” - it is clear that N(batch) = 1, C(channels) = 3, H(height) = 720 and W(width) = 1280. Without layout information {1, 3, 720, 1280} doesn’t give any idea to your application what these number mean and how to resize input image to fit model’s expectations.

Reasons when you may want to care about input/output layout:

Perform model modification:
- Apply preprocessing steps, like subtract means, divide by scales, resize image, convert RGB<->BGR
- Set/get batch for a model
Same operations, used during model conversion phase, see Model Optimizer Embedding Preprocessing Computation
Improve readability of a model’s input and output

Layout syntax¶

Short¶

The easiest way is to fully specify each dimension with one alphabetical letter

layout = ov::Layout("NHWC");

from openvino.runtime import Layout
layout = Layout('NCHW')

This assigns ‘N’ to first dimension, ‘C’ to second, ‘H’ to 3rd and ‘W’ to 4th

Advanced¶

Advanced syntax allows assigning a word to a dimension. To do this, wrap layout with square brackets [] and specify each name separated by comma ,

// Each dimension has name separated by comma, layout is wrapped with square brackets
layout = ov::Layout("[time,temperature,humidity]");

# Each dimension has name separated by comma
# Layout is wrapped with square brackets
layout = Layout('[time,temperature,humidity]')

Partially defined layout¶

If some dimension is not important, it’s name can be set to ?

// First dimension is batch, 4th is 'channels'. Others are not important for us
layout = ov::Layout("N??C");
// Or the same using advanced syntax
layout = ov::Layout("[n,?,?,c]");

# First dimension is batch, 4th is 'channels'.
# Others are not important for us
layout = Layout('N??C')

# Or the same using advanced syntax
layout = Layout('[n,?,?,c]')

Dynamic layout¶

If number of dimensions is not important, ellipsis ... can be used to specify variadic number of dimensions.

// First dimension is 'batch' others are whatever
layout = ov::Layout("N...");

// Second dimension is 'channels' others are whatever
layout = ov::Layout("?C...");

// Last dimension is 'channels' others are whatever
layout = ov::Layout("...C");

# First dimension is 'batch' others are whatever
layout = Layout('N...')

# Second dimension is 'channels' others are whatever
layout = Layout('?C...')

# Last dimension is 'channels' others are whatever
layout = Layout('...C')

Predefined names¶

Layout has pre-defined some widely used in computer vision dimension names:

N/Batch - batch size
C/Channels - channels dimension
D/Depth - depth
H/Height - height
W/Width - width

These names are used in PreProcessing API and there is a set of helper functions to get appropriate dimension index from layout

// returns 0 for batch
ov::layout::batch_idx("NCDHW");

// returns 1 for channels
ov::layout::channels_idx("NCDHW");

// returns 2 for depth
ov::layout::depth_idx("NCDHW");

// returns -2 for height
ov::layout::height_idx("...HW");

// returns -1 for width
ov::layout::width_idx("...HW");

from openvino.runtime import layout_helpers
# returns 0 for batch
layout_helpers.batch_idx(Layout('NCDHW'))

# returns 1 for channels
layout_helpers.channels_idx(Layout('NCDHW'))

# returns 2 for depth
layout_helpers.depth_idx(Layout('NCDHW'))

# returns -2 for height
layout_helpers.height_idx(Layout('...HW'))

# returns -1 for width
layout_helpers.width_idx(Layout('...HW'))

Equality¶

Layout names are case-insensitive, which means that Layout("NCHW") == Layout("nChW") == Layout("[N,c,H,w]")

Dump layout¶

Layout can be converted to string in advanced syntax format. Can be useful for debugging and serialization purposes

layout = ov::Layout("NCHW");
std::cout << layout.to_string(); // prints [N,C,H,W]

layout = Layout('NCHW')
print(layout)    # prints [N,C,H,W]