class InferenceEngine::InferRequest¶

Overview¶

This is an interface of asynchronous infer request. More…

#include <ie_infer_request.hpp>

class InferRequest
{
public:
    // typedefs

    typedef std::shared_ptr<InferRequest> Ptr;

    // enums

    enum WaitMode;

    // structs

    template <typename T>
    struct SetCallback;
    template <>
    struct SetCallback<IInferRequest::CompletionCallback>;
    template <>
    struct SetCallback<std::function<void(InferRequest, StatusCode)>>;

    // construction

    InferRequest();
    InferRequest(const InferRequest& other);
    InferRequest(InferRequest&& other);

    // methods

    InferRequest& operator = (const InferRequest& other);
    InferRequest& operator = (InferRequest&& other);
    void SetBlob(const std::string& name, const Blob::Ptr& data);
    Blob::Ptr GetBlob(const std::string& name);

    void SetBlob(
        const std::string& name,
        const Blob::Ptr& data,
        const PreProcessInfo& info
        );

    const PreProcessInfo& GetPreProcess(const std::string& name) const;
    void Infer();
    void Cancel();
    std::map<std::string, InferenceEngineProfileInfo> GetPerformanceCounts() const;
    void SetInput(const BlobMap& inputs);
    void SetOutput(const BlobMap& results);
    void SetBatch(const int batch);
    void StartAsync();
    StatusCode Wait(int64_t millis_timeout = RESULT_READY);

    template <typename F>
    void SetCompletionCallback(F callbackToSet);

    std::vector<VariableState> QueryState();
    operator std::shared_ptr< IInferRequest > ();
    bool operator ! () const;
    operator bool () const;
    bool operator != (const InferRequest&) const;
    bool operator == (const InferRequest&) const;
};

Detailed Documentation¶

This is an interface of asynchronous infer request.

Wraps IInferRequest It can throw exceptions safely for the application, where it is properly handled.

Typedefs¶

typedef std::shared_ptr<InferRequest> Ptr

A smart pointer to the InferRequest object.

Construction¶

InferRequest()

Default constructor.

InferRequest(const InferRequest& other)

Default copy constructor.

Parameters:

other

other InferRequest object

InferRequest(InferRequest&& other)

Default move constructor.

Parameters:

other

other InferRequest object

Methods¶

InferRequest& operator = (const InferRequest& other)

Default copy assignment operator.

Parameters:

other

other InferRequest object

Returns:

reference to the current object

InferRequest& operator = (InferRequest&& other)

Default move assignment operator.

Parameters:

other

other InferRequest object

Returns:

reference to the current object

void SetBlob(const std::string& name, const Blob::Ptr& data)

Sets input/output data to infer.

Memory allocation does not happen

Parameters:

name	Name of input or output blob.
data	Reference to input or output blob. The type of a blob must match the network input precision and size.

Blob::Ptr GetBlob(const std::string& name)

Gets input/output data for inference.

Memory allocation does not happen

Parameters:

name	A name of Blob to get

Returns:

A shared pointer to a Blob with a name name. If a blob is not found, an exception is thrown.

void SetBlob(
    const std::string& name,
    const Blob::Ptr& data,
    const PreProcessInfo& info
    )

Sets blob with a pre-process information.

Returns an error in case if data blob is output

Parameters:

name	Name of input blob.
data	A reference to input. The type of Blob must correspond to the network input precision and size.
info	Preprocess info for blob.

const PreProcessInfo& GetPreProcess(const std::string& name) const

Gets pre-process for input data.

Parameters:

name	Name of input blob.

Returns:

pointer to pre-process info of blob with name

void Infer()

Infers specified input(s) in synchronous mode.

blocks all methods of InferRequest while request is ongoing (running or waiting in queue)

void Cancel()

Cancels inference request.

std::map<std::string, InferenceEngineProfileInfo> GetPerformanceCounts() const

Queries performance measures per layer to get feedback of what is the most time consuming layer.

not all plugins provide meaningful data

Returns:

Map of layer names to profiling information for that layer

void SetInput(const BlobMap& inputs)

Sets input data to infer.

Memory allocation doesn’t happen

Parameters:

inputs

A reference to a map of input blobs accessed by input names. The type of Blob must correspond to the network input precision and size.

void SetOutput(const BlobMap& results)

Sets data that will contain result of the inference.

Memory allocation doesn’t happen

Parameters:

results

a reference to a map of result blobs accessed by output names. The type of Blob must correspond to the network output precision and size.

void SetBatch(const int batch)

Sets new batch size when dynamic batching is enabled in executable network that created this request.

Parameters:

batch

new batch size to be used by all the following inference calls for this request.

void StartAsync()

Start inference of specified input(s) in asynchronous mode.

It returns immediately. Inference starts also immediately.

StatusCode Wait(int64_t millis_timeout = RESULT_READY)

Waits for the result to become available. Blocks until specified millis_timeout has elapsed or the result becomes available, whichever comes first.

There are special cases when millis_timeout is equal some value of the WaitMode enum:

STATUS_ONLY - immediately returns inference status (IInferRequest::RequestStatus). It does not block or interrupt current thread
RESULT_READY - waits until inference result becomes available

Parameters:

millis_timeout

Maximum duration in milliseconds to block for

Returns:

A status code of operation

template <typename F>
void SetCompletionCallback(F callbackToSet)

Sets a callback function that will be called on success or failure of asynchronous request.

Parameters:

callbackToSet

callback object which will be called on when inference finish.

std::vector<VariableState> QueryState()

Gets state control interface for given infer request.

State control essential for recurrent networks

Returns:

A vector of Memory State objects

operator std::shared_ptr< IInferRequest > ()

IInferRequest pointer to be used directly in CreateInferRequest functions.

Returns:

A shared pointer to IInferRequest interface

bool operator ! () const

Checks if current InferRequest object is not initialized.

Returns:

true if current InferRequest object is not initialized, false - otherwise

operator bool () const

Checks if current InferRequest object is initialized.

Returns:

true if current InferRequest object is initialized, false - otherwise

bool operator != (const InferRequest&) const

Compares whether this request wraps the same impl underneath.

Returns:

true if current InferRequest object doesn’t wrap the same impl as the operator’s arg

bool operator == (const InferRequest&) const

Compares whether this request wraps the same impl underneath.

Returns:

true if current InferRequest object wraps the same impl as the operator’s arg