class InferenceEngine::IInferRequest¶

Overview¶

This is an interface of asynchronous infer request. More…

#include <ie_iinfer_request.hpp>

class IInferRequest: public std::enable_shared_from_this< IInferRequest >
{
public:
    // typedefs

    typedef std::shared_ptr<IInferRequest> Ptr;
    typedef std::weak_ptr<IInferRequest> WeakPtr;

    typedef void(\* CompletionCallback)(
        InferenceEngine::IInferRequest::Ptr context,
        InferenceEngine::StatusCode code
        );

    // enums

    enum WaitMode;

    // methods

    virtual StatusCode SetBlob(
        const char \* name,
        const Blob::Ptr& data,
        ResponseDesc \* resp
        ) = 0;

    virtual StatusCode GetBlob(
        const char \* name,
        Blob::Ptr& data,
        ResponseDesc \* resp
        ) = 0;

    virtual StatusCode SetBlob(
        const char \* name,
        const Blob::Ptr& data,
        const PreProcessInfo& info,
        ResponseDesc \* resp
        ) = 0;

    virtual StatusCode GetPreProcess(
        const char \* name,
        const PreProcessInfo \*\* info,
        ResponseDesc \* resp
        ) const = 0;

    virtual StatusCode Infer(ResponseDesc \* resp) = 0;
    virtual StatusCode Cancel(ResponseDesc \* resp) = 0;

    virtual StatusCode GetPerformanceCounts(
        std::map<std::string, InferenceEngineProfileInfo>& perfMap,
        ResponseDesc \* resp
        ) const = 0;

    virtual InferenceEngine::StatusCode Wait(
        int64_t millis_timeout,
        ResponseDesc \* resp
        ) = 0;

    virtual StatusCode StartAsync(ResponseDesc \* resp) = 0;
    virtual StatusCode SetCompletionCallback(CompletionCallback callback) = 0;
    virtual StatusCode GetUserData(void \*\* data, ResponseDesc \* resp) = 0;
    virtual StatusCode SetUserData(void \* data, ResponseDesc \* resp) = 0;

    virtual InferenceEngine::StatusCode SetBatch(
        int batch_size,
        ResponseDesc \* resp
        ) = 0;

protected:
};

Detailed Documentation¶

This is an interface of asynchronous infer request.

Deprecated Use InferenceEngine::InferRequest C++ wrapper

Typedefs¶

typedef std::shared_ptr<IInferRequest> Ptr

A shared pointer to the IInferRequest object.

typedef std::weak_ptr<IInferRequest> WeakPtr

A smart pointer to the IInferRequest object.

typedef void(\* CompletionCallback)(
    InferenceEngine::IInferRequest::Ptr context,
    InferenceEngine::StatusCode code
    )

Completion callback definition as pointer to a function.

Parameters:

context	Pointer to request for providing context inside callback
code	Completion result status: InferenceEngine::OK (0) for success

Methods¶

virtual StatusCode SetBlob(
    const char \* name,
    const Blob::Ptr& data,
    ResponseDesc \* resp
    ) = 0

Sets input/output data to infer.

Memory allocation does not happen

Parameters:

name	Name of input or output blob.
data	Reference to input or output blob. The type of a blob must match the network input precision and size.
resp	Optional: pointer to an already allocated object to contain information in case of failure

Returns:

Status code of the operation: InferenceEngine::OK (0) for success

virtual StatusCode GetBlob(
    const char \* name,
    Blob::Ptr& data,
    ResponseDesc \* resp
    ) = 0

Gets input/output data for inference.

Memory allocation does not happen

Parameters:

name	Name of input or output blob.
data	Reference to input or output blob. The type of Blob must match the network input precision and size.
resp	Optional: pointer to an already allocated object to contain information in case of failure

Returns:

Status code of the operation: InferenceEngine::OK (0) for success

virtual StatusCode SetBlob(
    const char \* name,
    const Blob::Ptr& data,
    const PreProcessInfo& info,
    ResponseDesc \* resp
    ) = 0

Sets pre-process for input data.

Parameters:

name	Name of input blob.
data	Reference to input or output blob. The type of Blob must match the network input precision and size.
info	Preprocess info for blob.
resp	Optional: pointer to an already allocated object to contain information in case of failure

Returns:

Status code of the operation: OK (0) for success

virtual StatusCode GetPreProcess(
    const char \* name,
    const PreProcessInfo \*\* info,
    ResponseDesc \* resp
    ) const = 0

Gets pre-process for input data.

Parameters:

name	Name of input blob.
info	pointer to a pointer to PreProcessInfo structure
resp	Optional: pointer to an already allocated object to contain information in case of failure

Returns:

Status code of the operation: OK (0) for success

virtual StatusCode Infer(ResponseDesc \* resp) = 0

Infers specified input(s) in synchronous mode.

blocks all methods of IInferRequest while request is ongoing (running or waiting in queue)

Parameters:

resp	Optional: pointer to an already allocated object to contain information in case of failure

Returns:

Status code of the operation: InferenceEngine::OK (0) for success

virtual StatusCode Cancel(ResponseDesc \* resp) = 0

Cancels current async inference request.

Parameters:

resp	Optional: pointer to an already allocated object to contain information in case of failure

Returns:

Status code of the operation: InferenceEngine::OK (0) for success

virtual StatusCode GetPerformanceCounts(
    std::map<std::string, InferenceEngineProfileInfo>& perfMap,
    ResponseDesc \* resp
    ) const = 0

Queries performance measures per layer to get feedback of what is the most time consuming layer.

not all plugins provide meaningful data

Parameters:

perfMap	Map of layer names to profiling information for that layer
resp	Optional: pointer to an already allocated object to contain information in case of failure

Returns:

Status code of the operation: InferenceEngine::OK (0) for success

virtual InferenceEngine::StatusCode Wait(
    int64_t millis_timeout,
    ResponseDesc \* resp
    ) = 0

Waits for the result to become available. Blocks until specified millis_timeout has elapsed or the result becomes available, whichever comes first.

There are special cases when millis_timeout is equal some value of the WaitMode enum:

STATUS_ONLY - immediately returns inference status (IInferRequest::RequestStatus). It does not block or interrupt current thread
RESULT_READY - waits until inference result becomes available

Parameters:

millis_timeout	Maximum duration in milliseconds to block for
resp	Optional: a pointer to an already allocated object to contain extra information of a failure (if occurred)

Returns:

Enumeration of the resulted action: InferenceEngine::OK (0) for success

virtual StatusCode StartAsync(ResponseDesc \* resp) = 0

Starts inference of specified input(s) in asynchronous mode.

It returns immediately. Inference starts also immediately

Parameters:

resp	Optional: a pointer to an already allocated object to contain extra information of a failure (if occurred)

Returns:

Enumeration of the resulted action: InferenceEngine::OK (0) for success

virtual StatusCode SetCompletionCallback(CompletionCallback callback) = 0

Sets a callback function that will be called on success or failure of asynchronous request.

Parameters:

callback

A function to be called

Returns:

Enumeration of the resulted action: InferenceEngine::OK (0) for success

virtual StatusCode GetUserData(void \*\* data, ResponseDesc \* resp) = 0

Gets arbitrary data for the request and stores a pointer to a pointer to the obtained data.

Parameters:

data	Pointer to a pointer to the gotten arbitrary data
resp	Optional: a pointer to an already allocated object to contain extra information of a failure (if occurred)

Returns:

Enumeration of the resulted action: InferenceEngine::OK (0) for success

virtual StatusCode SetUserData(void \* data, ResponseDesc \* resp) = 0

Sets arbitrary data for the request.

Parameters:

data	Pointer to a pointer to arbitrary data to set
resp	Optional: a pointer to an already allocated object to contain extra information of a failure (if occurred)

Returns:

Enumeration of the resulted action: InferenceEngine::OK (0) for success

virtual InferenceEngine::StatusCode SetBatch(
    int batch_size,
    ResponseDesc \* resp
    ) = 0

Sets new batch size when dynamic batching is enabled in executable network that created this request.

Parameters:

batch_size	new batch size to be used by all the following inference calls for this request.
resp	Optional: a pointer to an already allocated object to contain extra information of a failure (if occurred)

Returns:

Enumeration of the resulted action: InferenceEngine::OK (0) for success