class ov::CompiledModel¶

Overview¶

This class represents a compiled model.

A model is compiled by a specific device by applying multiple optimization transformations, then mapping to compute kernels. More…

#include <compiled_model.hpp>

class CompiledModel
{
public:
    // methods

    std::shared_ptr<const Model> get_runtime_model() const;
    std::vector<ov::Output<const ov::Node>> inputs() const;
    ov::Output<const ov::Node> input() const;
    ov::Output<const ov::Node> input(size_t i) const;
    ov::Output<const ov::Node> input(const std::string& tensor_name) const;
    std::vector<ov::Output<const ov::Node>> outputs() const;
    ov::Output<const ov::Node> output() const;
    ov::Output<const ov::Node> output(size_t i) const;
    ov::Output<const ov::Node> output(const std::string& tensor_name) const;
    InferRequest create_infer_request();
    void export_model(std::ostream& model_stream);
    void set_property(const AnyMap& properties);

    template <typename... Properties>
    util::EnableIfAllStringAny<void, Properties...> set_property(Properties&&... properties);

    Any get_property(const std::string& name) const;

    template <typename T, PropertyMutability mutability>
    T get_property(const ov::Property<T, mutability>& property) const;

    RemoteContext get_context() const;
    bool operator ! () const;
    operator bool () const;
};

Detailed Documentation¶

This class represents a compiled model.

A model is compiled by a specific device by applying multiple optimization transformations, then mapping to compute kernels.

Methods¶

std::shared_ptr<const Model> get_runtime_model() const

Gets runtime model information from a device. This object represents an internal device-specific model that is optimized for a particular accelerator. It contains device-specific nodes, runtime information and can be used only to understand how the source model is optimized and which kernels, element types, and layouts are selected for optimal inference.

Returns:

A model containing Executable Graph Info.

std::vector<ov::Output<const ov::Node>> inputs() const

Gets all inputs of a compiled model. Inputs are represented as a vector of outputs of the ov::op::v0::Parameter operations. They contain information about input tensors such as tensor shape, names, and element type.

Returns:

std::vector of model inputs.

ov::Output<const ov::Node> input() const

Gets a single input of a compiled model. The input is represented as an output of the ov::op::v0::Parameter operation. The input contains information about input tensor such as tensor shape, names, and element type.

If a model has more than one input, this method throws ov::Exception.

Returns:

Compiled model input.

ov::Output<const ov::Node> input(size_t i) const

Gets input of a compiled model identified by i. The input contains information about input tensor such as tensor shape, names, and element type.

The method throws ov::Exception if input with the specified index i is not found.

Parameters:

i	Index of input.

Returns:

Compiled model input.

ov::Output<const ov::Node> input(const std::string& tensor_name) const

Gets input of a compiled model identified by tensor_name. The input contains information about input tensor such as tensor shape, names, and element type.

The method throws ov::Exception if input with the specified tensor name tensor_name is not found.

Parameters:

tensor_name

The input tensor name.

Returns:

Compiled model input.

std::vector<ov::Output<const ov::Node>> outputs() const

Get all outputs of a compiled model. Outputs are represented as a vector of output from the ov::op::v0::Result operations. Outputs contain information about output tensors such as tensor shape, names, and element type.

Returns:

std::vector of model outputs.

ov::Output<const ov::Node> output() const

Gets a single output of a compiled model. The output is represented as an output from the ov::op::v0::Result operation. The output contains information about output tensor such as tensor shape, names, and element type.

If a model has more than one output, this method throws ov::Exception.

Returns:

Compiled model output.

ov::Output<const ov::Node> output(size_t i) const

Gets output of a compiled model identified by index. The output contains information about output tensor such as tensor shape, names, and element type.

The method throws ov::Exception if output with the specified index index is not found.

Parameters:

i	Index of input.

Returns:

Compiled model output.

ov::Output<const ov::Node> output(const std::string& tensor_name) const

Gets output of a compiled model identified by tensor_name. The output contains information about output tensor such as tensor shape, names, and element type.

The method throws ov::Exception if output with the specified tensor name tensor_name is not found.

Parameters:

tensor_name

Output tensor name.

Returns:

Compiled model output.

InferRequest create_infer_request()

Creates an inference request object used to infer the compiled model. The created request has allocated input and output tensors (which can be changed later).

Returns:

InferRequest object

void export_model(std::ostream& model_stream)

Exports the current compiled model to an output stream std::ostream. The exported model can also be imported via the ov::Core::import_model method.

Parameters:

model_stream

Output stream to store the model to.

See also:

ov::Core::import_model

void set_property(const AnyMap& properties)

Sets properties for the current compiled model.

Parameters:

properties

Map of pairs: (property name, property value).

template <typename... Properties>
util::EnableIfAllStringAny<void, Properties...> set_property(Properties&&... properties)

Sets properties for the current compiled model.

Parameters:

Properties	Should be the pack of `std::pair<std::string, ov::Any>` types.
properties	Optional pack of pairs: (property name, property value).

Any get_property(const std::string& name) const

Gets properties for current compiled model.

The method is responsible for extracting information that affects compiled model inference. The list of supported configuration values can be extracted via CompiledModel::get_property with the ov::supported_properties key, but some of these keys cannot be changed dynamically, for example, ov::device::id cannot be changed if a compiled model has already been compiled for a particular device.

Parameters:

name	Property key, can be found in `openvino/runtime/properties.hpp`.

Returns:

Property value.

template <typename T, PropertyMutability mutability>
T get_property(const ov::Property<T, mutability>& property) const

Gets properties related to device behaviour.

The method extracts information that can be set via the set_property method.

Parameters:

T	Type of a returned value.
property	Property object.

Returns:

Value of property.

RemoteContext get_context() const

Returns pointer to device-specific shared context on a remote accelerator device that was used to create this CompiledModel.

Returns:

A context.

bool operator ! () const

Checks if the current CompiledModel object is not initialized.

Returns:

true if the current CompiledModel object is not initialized; false, otherwise.

operator bool () const

Checks if the current CompiledModel object is initialized.

Returns:

true if the current CompiledModel object is initialized; false, otherwise.

Prev Next