Preprocessing¶

Introduction¶

Inference Engine API contains preprocessing capabilities in the InferenceEngine::CNNNetwork class. Such preprocessing information is not a part of the main inference graph executed by OpenVINO devices. Therefore, it is stored and executed separately before the inference stage.

Preprocessing operations are executed on the CPU for most OpenVINO inference plugins. Thus, instead of occupying accelerators, they keep the CPU busy with computational tasks.
Preprocessing information stored in InferenceEngine::CNNNetwork is lost when saving back to the IR file format.

OpenVINO Runtime API 2.0 introduces a new way of adding preprocessing operations to the model - each preprocessing or postprocessing operation is integrated directly into the model and compiled together with the inference graph.

Add preprocessing operations first using ov::preprocess::PrePostProcessor
Then, compile the model on the target, using ov::Core::compile_model

Having preprocessing operations as a part of an OpenVINO opset makes it possible to read and serialize a preprocessed model as the IR file format.

More importantly, OpenVINO Runtime API 2.0 does not assume any default layouts, as Inference Engine did. For example, both { 1, 224, 224, 3 } and { 1, 3, 224, 224 } shapes are supposed to be in the NCHW layout, while only the latter is. Therefore, some preprocessing capabilities in the API require layouts to be set explicitly. To learn how to do it, refer to the Layout overview. For example, to perform image scaling by partial dimensions H and W, preprocessing needs to know what dimensions H and W are.

Note

Use Model Optimizer preprocessing capabilities to insert preprocessing operations in your model for optimization. Thus, the application does not need to read the model and set preprocessing repeatedly. You can use the model caching feature to improve the time-to-inference.

The steps below demonstrate how to migrate preprocessing scenarios from Inference Engine API to OpenVINO Runtime API 2.0. The snippets assume we need to preprocess a model input with the tensor_name in Inference Engine API, using operation_name to address the data.

Importing Preprocessing in Python¶

In order to utilize preprocessing, the following imports must be added.

Inference Engine API:

import openvino.inference_engine as ie

OpenVINO Runtime API 2.0:

from openvino.runtime import Core, Layout, Type
from openvino.preprocess import ColorFormat, PrePostProcessor, ResizeAlgorithm

There are two different namespaces: runtime, which contains OpenVINO Runtime API classes; and preprocess, which provides Preprocessing API.

Mean and Scale Values¶

Inference Engine API:

auto preProcess = network.getInputsInfo()[operation_name]->getPreProcess();
preProcess.init(3);
preProcess[0]->meanValue = 116.78f;
preProcess[1]->meanValue = 116.78f;
preProcess[2]->meanValue = 116.78f;
preProcess[0]->stdScale = 57.21f;
preProcess[1]->stdScale = 57.45f;
preProcess[2]->stdScale = 57.73f;
preProcess.setVariant(InferenceEngine::MEAN_VALUE);

preProcess = network.getInputsInfo()[operation_name].getPreProcess()
preProcess.init(3)
preProcess[0].meanValue = 116.78
preProcess[1].meanValue = 116.78
preProcess[2].meanValue = 116.78
preProcess[0].stdScale = 57.21
preProcess[1].stdScale = 57.45
preProcess[2].stdScale = 57.73
preProcess.setVariant(ie.MEAN_VALUE)

OpenVINO Runtime API 2.0:

ov::preprocess::PrePostProcessor ppp(model);
ov::preprocess::InputInfo& input = ppp.input(tensor_name);
// we only need to know where is C dimension
input.model().set_layout("...C");
// specify scale and mean values, order of operations is important
input.preprocess().mean(116.78f).scale({ 57.21f, 57.45f, 57.73f });
// insert preprocessing operations to the 'model'
model = ppp.build();

ppp = PrePostProcessor(model)
input = ppp.input(tensor_name)
# we only need to know where is C dimension
input.model().set_layout(Layout('...C'))
# specify scale and mean values, order of operations is important
input.preprocess().mean([116.78]).scale([57.21, 57.45, 57.73])
# insert preprocessing operations to the 'model'
model = ppp.build()

Precision and Layout Conversions¶

Inference Engine API:

auto inputInfo = network.getInputsInfo()[operation_name];
inputInfo->setPrecision(InferenceEngine::Precision::U8);
inputInfo->setLayout(InferenceEngine::Layout::NHWC);
// model input layout is always NCHW in Inference Engine
// for shapes with 4 dimensions

inputInfo = network.getInputsInfo()[operation_name]
inputInfo.setPrecision(ie.Precision.U8)
inputInfo.setLayout(ie.Layout.NHWC)
# model input layout is always NCHW in Inference Engine
# for shapes with 4 dimensions

OpenVINO Runtime API 2.0:

ov::preprocess::PrePostProcessor ppp(model);
ov::preprocess::InputInfo& input = ppp.input(tensor_name);
input.tensor().set_layout("NHWC").set_element_type(ov::element::u8);
input.model().set_layout("NCHW");
// layout and precision conversion is inserted automatically,
// because tensor format != model input format
model = ppp.build();

ppp = PrePostProcessor(model)
input = ppp.input(tensor_name)
input.tensor().set_layout(Layout('NCHW')).set_element_type(Type.u8)
input.model().set_layout(Layout('NCHW'))
# layout and precision conversion is inserted automatically,
# because tensor format != model input format
model = ppp.build()

Image Scaling¶

Inference Engine API:

auto preProcess = network.getInputsInfo()[operation_name]->getPreProcess();
// Inference Engine supposes input for resize is always in NCHW layout
// while for OpenVINO Runtime API 2.0 `H` and `W` dimensions must be specified
// Also, current code snippet supposed resize from dynamic shapes
preProcess.setResizeAlgorithm(InferenceEngine::ResizeAlgorithm::RESIZE_BILINEAR);

preProcess = network.getInputsInfo()[operation_name].getPreProcess()
# Inference Engine supposes input for resize is always in NCHW layout
# while for OpenVINO Runtime API 2.0 `H` and `W` dimensions must be specified
# Also, current code snippet supposed resize from dynamic shapes
preProcess.setResizeAlgorithm(ie.ResizeAlgorithm.RESIZE_BILINEAR)

OpenVINO Runtime API 2.0:

ov::preprocess::PrePostProcessor ppp(model);
ov::preprocess::InputInfo& input = ppp.input(tensor_name);
// scale from the specified tensor size
input.tensor().set_spatial_static_shape(448, 448);
// need to specify H and W dimensions in model, others are not important
input.model().set_layout("??HW");
// scale to model shape
input.preprocess().resize(ov::preprocess::ResizeAlgorithm::RESIZE_LINEAR);
// and insert operations to the model
model = ppp.build();

ppp = PrePostProcessor(model)
input = ppp.input(tensor_name)
# need to specify H and W dimensions in model, others are not important
input.model().set_layout(Layout('??HW'))
# scale to model shape
input.preprocess().resize(ResizeAlgorithm.RESIZE_LINEAR, 448, 448)
# and insert operations to the model
model = ppp.build()

Color Space Conversions¶

Inference Engine API:

auto preProcess = network.getInputsInfo()[operation_name]->getPreProcess();
// Inference Engine supposes NV12 as two inputs which need to be passed
// as InferenceEngine::NV12Blob composed of two Y and UV planes
preProcess.setColorFormat(InferenceEngine::NV12);

preProcess = network.getInputsInfo()[operation_name].getPreProcess()
# Inference Engine supposes NV12 as two inputs which need to be passed
# as InferenceEngine::NV12Blob composed of two Y and UV planes
preProcess.setColorFormat(ie.NV12)

OpenVINO Runtime API 2.0:

ov::preprocess::PrePostProcessor ppp(model);
ov::preprocess::InputInfo& input = ppp.input(tensor_name);
input.tensor().set_color_format(ov::preprocess::ColorFormat::NV12_TWO_PLANES);
// add NV12 to BGR conversion
input.preprocess().convert_color(ov::preprocess::ColorFormat::BGR);
// and insert operations to the model
model = ppp.build();

ppp = PrePostProcessor(model)
input = ppp.input(tensor_name)
input.tensor().set_color_format(ColorFormat.NV12_TWO_PLANES)
# add NV12 to BGR conversion
input.preprocess().convert_color(ColorFormat.BGR)
# and insert operations to the model
model = ppp.build()

See also:

Prev Next