Deploy with OpenVINO¶

Once the OpenVINO application development has been finished, usually application developers need to deploy their applications to end users. There are several ways how to achieve that:

Set a dependency on existing prebuilt packages (so called centralized distribution):
- Using Debian / RPM packages, a recommended way for a family of Linux operation systems
- Using pip package manager on PyPi, default approach for Python-based applications
- Using Docker images. If the application should be deployed as a Docker image, developer can use a pre-built runtime OpenVINO Docker image as a base image in the Dockerfile for the application container image. You can find more info about available OpenVINO Docker images in the Install Guides for Linux and Windows. Also, if you need to customize OpenVINO Docker image, you can use Docker CI Framework to generate a Dockerfile and built it.
Grab a necessary functionality of OpenVINO together with your application (so-called local distribution):
- Using OpenVINO Deployment manager providing a convinient way create a distribution package
- Using advanced Local distribution approach
- Using static version of OpenVINO Runtime linked into the final app

The table below shows which distribution type can be used depending on target operation system:

Distribution type	Operation systems
Debian packages	Ubuntu 18.04 long-term support (LTS), 64-bit; Ubuntu 20.04 long-term support (LTS), 64-bit
RMP packages	Red Hat Enterprise Linux 8, 64-bit
Docker images	Ubuntu 18.04 long-term support (LTS), 64-bit; Ubuntu 20.04 long-term support (LTS), 64-bit; Red Hat Enterprise Linux 8, 64-bit; Windows Server Core base LTSC 2019, 64-bit; Windows 10, version 20H2, 64-bit
PyPi (pip package manager)	See https://pypi.org/project/openvino/
OpenVINO Deployment Manager	All operation systems
Local distribution	All operation systems
Build OpenVINO statically and link into the final app	All operation systems

Depending on the distribution type, the granularity of OpenVINO packages may vary: PyPi distribution OpenVINO has a single package ‘openvino’ containing all the runtime libraries and plugins, while more configurable ways like Local distribution provide higher granularity, so it is important to now some details about the set of libraries which are part of OpenVINO Runtime package:

The main library openvino is used by C++ user’s applications to link against with. The library provides all OpenVINO Runtime public API for both OpenVINO API 2.0 and Inference Engine, nGraph APIs. For C language applications openvino_c is additionally required for distribution.
The optional plugin libraries like openvino_intel_cpu_plugin (matching openvino_.+_plugin pattern) are used to provide inference capabilities on specific devices or additional capabitilies like Hetero execution or Multi-Device execution.
The optional plugin libraries like openvino_ir_frontnend (matching openvino_.+_frontend) are used to provide capabilities to read models of different file formats like OpenVINO IR, ONNX or Paddle.

The optional means that if the application does not use the capability enabled by the plugin, the plugin’s library or package with the plugin is not needed in the final distribution.

The information above covers granularity aspects of majority distribution types, more detailed information is only needed and provided in Local Distribution.

Note

Depending on target OpenVINO devices, you also have to use Configurations for GPU, Configurations for GNA, Configurations for NCS2 or Configurations for VPU for proper configuration of deployed machines.

Prev Next