Model Inference API¶

cvm.runtime¶

CVM Runtime API

This namespace wraps the python interface of c backend API. Inference methods contain model load, inference, …, and free etc.

We have supply two wrapper format via ctypes and cython (TODO).

cvm.runtime.CVMAPILoadModel(json_str, param_bytes, ctx=None)¶

Ctypes wrapper method: CVMAPILoadModel

Load model from memory binary. The parameters loaded from disk could be generated via cvm.utils.load_model method.

Parameters

json_str (bytes) – The UTF-8 encoded bytes of string type reading from the model json file.
param_bytes (bytes) – The binary reading from the model params file.
ctx (cvm.CVMContext) – The context of model loaded into.

cvm.runtime.CVMAPIFreeModel(net)¶

Ctypes wrapper method: CVMAPIFreeModel

Free model from memory binary.

Parameters: net (ctypes.c_void_p) – The CVM model handle created by the interface cvm.runtime.CVMAPILoadModel.

cvm.runtime.CVMAPIInference(net, input_data)¶

Ctypes wrapper method: CVMAPIInference

CVM interface for model inference. Model output tensor initialization, forward network computing,output tensor disk serialization are successively performed.

Parameters

net (ctypes.c_void_p) – The CVM model handle created by the interface cvm.runtime.CVMAPILoadModel.
input_data (bytes) – The input image bytes.

cvm.runtime.CVMAPIGetInputLength(net)¶

Ctypes wrapper method: CVMAPIGetInputLengthModel

Get the input length of the model, which can be calculated as follows:

\[input\_length = input\_bytes * in\_size\]

Where in_size depends on model input shapes. For models with precision over 8, input_bytes equals to 4, otherwise 1.

Parameters: net (ctypes.c_void_p) – The CVM model handle created by the interface cvm.runtime.CVMAPILoadModel.

cvm.runtime.CVMAPIGetInputTypeSize(net)¶

Ctypes wrapper method: CVMAPIGetInputTypeSize

Get the size of input shape, namely input_bytes.

For models with precision over 8, input_bytes equals to 4, otherwise 1.

Parameters: net (ctypes.c_void_p) – The CVM model handle created by the interface cvm.runtime.CVMAPILoadModel.

cvm.runtime.CVMAPIGetOutputLength(net)¶

Ctypes wrapper method: CVMAPIGetOutputLength

Get the length of the output.

Postprocess method (argmax, detection) can be specified by model definition. If the postprocess method is not specified, the output will be flatten by default.

Parameters: net (ctypes.c_void_p) – The CVM model handle created by the interface cvm.runtime.CVMAPILoadModel.

cvm.runtime.CVMAPIGetOutputTypeSize(net)¶

Ctypes wrapper method: CVMAPIGetOutputTypeSize

Get the output_bytes. For models with ‘postprocess_method’ equals to ‘argmax’, the output_bytes is 1; for models with ‘postprocess_method’ equals to ‘detection’, the output_bytes is 4.

Parameters: net (ctypes.c_void_p) – The CVM model handle created by the interface cvm.runtime.CVMAPILoadModel.

cvm.ndarray¶

class cvm.ndarray.NDArray(handle, is_view=False)¶

asnumpy()¶

Convert this array to numpy array

Returns: np_arr – The corresponding numpy array.
Return type: numpy.ndarray

property context¶: context of this array

copyfrom(source_array)¶

Peform an synchronize copy from the array.

Parameters: source_array (array_like) – The data source we should like to copy from.
Returns: arr – Reference to self.
Return type: NDArray

property ctx¶: context of this array

property dtype¶: Type of this array

property ndim¶: ndim of this array

prepare_bcast_shape(value, bcast_shape)¶: Return a broadcast NDArray of shape bcast_shape with same context and dtype as self. value: numeric types or array like. bcast_shape: a shape tuple.

same_as(other)¶

Check object identity equality

Parameters: other (object) – The other object to compare to
Returns: same – Whether other is same as self.
Return type: bool

property shape¶: Shape of this array

cvm.ndarray.array(arr, ctx=@cpu(0))¶

Create an array from source arr.

Parameters

arr (numpy.ndarray) – The array to be copied from
ctx (cvm.CVMContext) – The device context to create the array, CPU context by default.

Returns

ret – The created array

Return type

cvm.ndarray.NDArray

cvm.ndarray.empty(shape, dtype='int32', ctx=@cpu(0))¶

C wrapper method of NDArray generator.

Notice: the allocated memory is supposed as empty, which means the memory is not formated and the real data created is random and unuseful.

Returns: nd_arr – An empty NDArray.
Return type: cvm.ndarray.NDArray

cvm.ndarray.save_param_dict(dict_data)¶

Transform the python cvm.ndarray.NDArray handle into bytes.

Returns: seq – The bytes binary of parameters dict.
Return type: bytes

class cvm.CVMContext(device_type, device_id)¶

Python wrapper for C backend context.

Parameters

device_type (int) – Device type of model loaded into.
device_id (int) – The device id corresponding to device type.

cvm.cpu(dev_id=0)¶

Context wrapper function.

Returns: context – CPU context
Return type: cvm.CVMContext

cvm.gpu(dev_id=0)¶

Context wrapper function.

Returns: context – GPU context
Return type: cvm.CVMContext

cvm.formal(dev_id=0)¶

Context wrapper function.

Returns: context – Formalization context
Return type: cvm.CVMContext

cvm.opencl(dev_id=0)¶

Context wrapper function.

Returns: context – Opencl context, not well supported.
Return type: cvm.CVMContext

cvm.context(dev_type, dev_id=0)¶

Context creator.

Does some safe-type checking.

Returns: context – Context type.
Return type: cvm.CVMContext