MRT Utils API

mrt.sim_quant_helper

Collection of MRT helper functions.

Simplification of MRT implementation.

mrt.sim_quant_helper.load_sim_data(data, name, inputs_ext)

Load data for MRT simulation.

Parameters

  • data (nd.NDArray) – The input raw data.

  • name (str) – The name of the data symbol.

Returns

ret – The data for simulation.

Return type

nd.NDArray

mrt.sim_quant_helper.load_real_data(data, name, inputs_ext)

Load realized data.

Parameters

  • data (nd.NDArray) – The input raw data.

  • name (str) – The name of the data symbol.

Returns

ret – The realized data.

Return type

nd.NDArray

mrt.sim_quant_helper.save_ext(fname, *infos, logger=<module 'logging' from '/home/docs/.pyenv/versions/3.7.9/lib/python3.7/logging/__init__.py'>)

Save ext files into disk.

Parameters

  • fname (str) – File path used to save.

  • infos (list) – Dict items to be saved.

mrt.sim_quant_helper.load_ext(fname)

Load absolute ext file names.

Parameters

fname (str) – File name to be extended.

Returns

ret – The extended file names.

Return type

tuple

mrt.sim_quant_helper.get_sim_scale(alpha, target_bit)

Get the scale from MRT simulation process.

Parameters

  • alpha (float) – The input threshold.

  • target_bit (int) – The target precision.

Returns

ret – The calculated scale.

Return type

float

mrt.sim_quant_helper.int_realize(data, target_bit, logger=<module 'logging' from '/home/docs/.pyenv/versions/3.7.9/lib/python3.7/logging/__init__.py'>)

Clip the data within the target precision.

Parameters

  • data (nd.NDArray) – The input data.

  • target_bit (int) – The target precision to clip on.

Returns

ret – The clipped data of the input.

Return type

int

mrt.sim_quant_helper.extract_float(number)

Extract single precision float value.

Parameters

number (float) – The input float value.

Returns

ret – The float value with its corresponding bits to be shifted.

Return type

tuple

mrt.sim_quant_helper.float_bin(number, places=24)

Single precision float convert into binary

Parameters

  • number (float) – The input float value.

  • places (int) – The target bits to represent the value.

Returns

ret – The sign along with the float value. If sign == -1, the value is negative. If sign == 1, the value is positive.

Return type

tuple

mrt.sim_quant_helper.cvm_float(number, bits=24)

Recalculate the float value within the given range of bits.

Parameters

  • number (float) – The input float value.

  • bits (int) – The target bits to represent the value.

Returns

ret – The recalculated float value with its corresponding bits to be shifted.

Return type

tuple

mrt.tfm_utils

Collection of MRT quantization tool functions.

Simplification of MRT quantization process.

mrt.tfm_utils.requant_operator(X, oprec, oscale=None, **kwargs)

MRT operator requantization interface.

If the tight precision of the input is greater than the output precision by ‘sb’, the input will be right shifted ‘sb’ bits.

If the infer precision of the input is greater than the output precision, or if oscale is specified, the operator will be requantized.

rescale = oscale / iscale
bits = MAX_BIT - iprec
frac, exp = cvm_float(rescale, bits)
sim_scale = frac * (2 ** exp)
oscale = iscale * frac * (2 ** exp)
X = realize(X, -exp, oprec)

See mrt.sim_quant_helper.cvm_float for reference.

Parameters

  • X (mxnet.symbol) – The input data.

  • oprec (int) – The output precision

  • oscale (float) – The output scale.

Returns

ret – The requantized operator of the input.

Return type

mxnet.symbol

mrt.tfm_utils.requant_parameter(wname, oprec, oscale=None, **kwargs)

MRT paramter requantization interface.

The input parameter will be clipped with respect to the given output precision as well as the output scale.

w = int_realize(param*oscale, oprec)

See mrt.sim_quant_helper.int_realize for reference.

Parameters

  • wname (str) – The name of the input parameter.

  • oprec (int) – The output precision

  • oscale (float) – The output scale.

Returns

ret – The requantized parameter of the input.

Return type

mxnet.symbol

mrt.tfm_utils.requant(sym, oprec, oscale=None, **kwargs)

Generalized interface for MRT requantization.

Parameters

  • sym (mxnet.symbol) – The input symbol to be requantized.

  • oprec (int) – The output precision.

  • oscale (float) – The output scale.

Returns

ret – The requantized symbol of the input.

Return type

mxnet.symbol

mrt.tfm_utils.scale(threshold, precision)

Get the scale in terms of the threshold and precision.

Parameters

  • threshold (float) – The input scale.

  • precision (int) – The input precision.

Returns

ret – The scale of the input.

Return type

float

mrt.tfm_utils.get_bit(opt)

Get the precision of the data.

Parameters

opt (nd.NDArray or scalar) – The input data.

Returns

ret – The precision of the input.

Return type

int

mrt.tfm_utils.get_bit_cnt(cnt)

Get the precision of the data aligned with cvm-runtime.

Parameters

cnt (int) – The input data.

Returns

ret – The precision of the input.

Return type

int

mrt.tfm_utils.get_range(prec)

Get the range in terms of the precision.

Parameters

prec (int) – The input precision.

Returns

ret – The precision of the input.

Return type

int

mrt.tfm_utils.realize(X, sb, prec, name=None)

Realize the symbol with respect to the shift bits.

Parameters

  • X (mxnet.symbol) – The input symbol to be realized.

  • sb (int) – The shift bits. If zero, the symbol will be clipped only. If negative, the symbol will be left shifted. If positive, the symbol will be right shifted.

  • prec (int) – The input precision.

Returns

ret – The realized symbol of the input.

Return type

mxnet.symbol

mrt.gluon_zoo

Gluon model zoo for MRT quantizatoin.

Only crucial parts of the module are elaborated.

Current supported MxNet Model List:

resnet18_v1, resnet34_v1, resnet50_v1, resnet101_v1, resnet152_v1, resnet18_v2, resnet34_v2, resnet50_v2, resnet101_v2, resnet152_v2, se_resnet18_v1, se_resnet34_v1, se_resnet50_v1, se_resnet101_v1, se_resnet152_v1, se_resnet18_v2, se_resnet34_v2, se_resnet50_v2, se_resnet101_v2, se_resnet152_v2, vgg11, vgg13, vgg16, vgg19, vgg11_bn, vgg13_bn, vgg16_bn, vgg19_bn, alexnet, densenet121, densenet161, densenet169, densenet201, squeezenet1.0, squeezenet1.1, inceptionv3, mobilenet1.0, mobilenet0.75, mobilenet0.5, mobilenet0.25, mobilenetv2_1.0, mobilenetv2_0.75, mobilenetv2_0.5, mobilenetv2_0.25, ssd_300_vgg16_atrous_voc, ssd_300_vgg16_atrous_coco, ssd_300_vgg16_atrous_custom, ssd_512_vgg16_atrous_voc, ssd_512_vgg16_atrous_coco, ssd_512_vgg16_atrous_custom, ssd_512_resnet18_v1_voc, ssd_512_resnet18_v1_coco, ssd_512_resnet50_v1_voc, ssd_512_resnet50_v1_coco, ssd_512_resnet50_v1_custom, ssd_512_resnet101_v2_voc, ssd_512_resnet152_v2_voc, ssd_512_mobilenet1.0_voc, ssd_512_mobilenet1.0_coco, ssd_512_mobilenet1.0_custom, faster_rcnn_resnet50_v1b_voc, faster_rcnn_resnet50_v1b_coco, faster_rcnn_fpn_resnet50_v1b_coco, faster_rcnn_fpn_bn_resnet50_v1b_coco, faster_rcnn_resnet50_v1b_custom, faster_rcnn_resnet101_v1d_voc, faster_rcnn_resnet101_v1d_coco, faster_rcnn_fpn_resnet101_v1d_coco, faster_rcnn_resnet101_v1d_custom, mask_rcnn_resnet50_v1b_coco, mask_rcnn_fpn_resnet50_v1b_coco, mask_rcnn_resnet101_v1d_coco, mask_rcnn_fpn_resnet101_v1d_coco, cifar_resnet20_v1, cifar_resnet56_v1, cifar_resnet110_v1, cifar_resnet20_v2, cifar_resnet56_v2, cifar_resnet110_v2, cifar_wideresnet16_10, cifar_wideresnet28_10, cifar_wideresnet40_8, cifar_resnext29_32x4d, cifar_resnext29_16x64d, fcn_resnet50_voc, fcn_resnet101_coco, fcn_resnet101_voc, fcn_resnet50_ade, fcn_resnet101_ade, psp_resnet101_coco, psp_resnet101_voc, psp_resnet50_ade, psp_resnet101_ade, psp_resnet101_citys, deeplab_resnet101_coco, deeplab_resnet101_voc, deeplab_resnet152_coco, deeplab_resnet152_voc, deeplab_resnet50_ade, deeplab_resnet101_ade, resnet18_v1b, resnet34_v1b, resnet50_v1b, resnet50_v1b_gn, resnet101_v1b_gn, resnet101_v1b, resnet152_v1b, resnet50_v1c, resnet101_v1c, resnet152_v1c, resnet50_v1d, resnet101_v1d, resnet152_v1d, resnet50_v1e, resnet101_v1e, resnet152_v1e, resnet50_v1s, resnet101_v1s, resnet152_v1s, resnext50_32x4d, resnext101_32x4d, resnext101_64x4d, se_resnext50_32x4d, se_resnext101_32x4d, se_resnext101_64x4d, senet_154, darknet53, yolo3_darknet53_coco, yolo3_darknet53_voc, yolo3_darknet53_custom, yolo3_mobilenet1.0_coco, yolo3_mobilenet1.0_voc, yolo3_mobilenet1.0_custom, nasnet_4_1056, nasnet_5_1538, nasnet_7_1920, nasnet_6_4032, simple_pose_resnet18_v1b, simple_pose_resnet50_v1b, simple_pose_resnet101_v1b, simple_pose_resnet152_v1b, simple_pose_resnet50_v1d, simple_pose_resnet101_v1d, simple_pose_resnet152_v1d, residualattentionnet56, residualattentionnet92, residualattentionnet128, residualattentionnet164, residualattentionnet200, residualattentionnet236, residualattentionnet452, cifar_residualattentionnet56, cifar_residualattentionnet92, cifar_residualattentionnet452, resnet18_v1b_0.89, resnet50_v1d_0.86, resnet50_v1d_0.48, resnet50_v1d_0.37, resnet50_v1d_0.11, resnet101_v1d_0.76, resnet101_v1d_0.73, mobilenet1.0_int8, resnet50_v1_int8, ssd_300_vgg16_atrous_voc_int8, ssd_512_mobilenet1.0_voc_int8, SSD_512_RESNET50_V1_VOC_INT8, SSD_512_VGG16_ATROUS_VOC_INT8

mrt.gluon_zoo.get_model(name, ctx=gpu(0), **kwargs)

Returns a pre-defined model by name

Parameters

  • name (str) – Name of the model.

  • classes (int) – Number of classes for the output layer.

  • ctx (mxnet.context()) – Devices to get model.

Returns

ret – The model.

Return type

HybridBlock

mrt.gluon_zoo.save_model(name, data_dir=None, ctx=gpu(0), **kwargs)

Returns a pre-defined model by name

Parameters

  • name (str) – Name of the model.

  • data_dir (str) – Directory to store the model.

  • ctx (mxnet.context()) – Devices to get model.

Returns

ret – The symbol path and the model path.

Return type

tuple

mrt.utils

Collection of MRT Model evaluation and file management tool functions.

Simplification of MRT evaluation process.

Only crucial parts of the module are elaborated.

mrt.utils.extend_fname(prefix, with_ext=False)

Get the precision of the data.

Parameters

  • prefix (str) – The model path prefix.

  • with_ext (bool) – Whether to include ext_file path in return value.

Returns

ret – The symbol path, params path; and with_ext is True, also return ext file path.

Return type

tuple

mrt.utils.load_parameters(graph, params, prefix='', ctx=None, dtype=None)

Unify the parameters dict with the given graph def and parameters dict.

Parameters

  • graph (dict) – MxNet symbol name (str) maps to mxnet.symbol

  • params (dict) – MxNet symbol name (str) maps to mxnet.NDArray.

Returns

ret – The unified MxNet symbol name (str) maps to mxnet.NDArray

Return type

dict

mrt.utils.multi_validate(base_func, data_iter, *comp_funcs, iter_num=10, logger=<module 'logging' from '/home/docs/.pyenv/versions/3.7.9/lib/python3.7/logging/__init__.py'>, batch_size=16)

Quantization precision comparative function for the original model and quantized models.

Parameters

  • base_func (function) – The original model evaluation function before quantization.

  • data_iter (function) – Data iter function.

  • comp_funcs (list) – A List of quantized model evaluation functions.

  • iter_num (int) – The number of iteration steps.

  • batch_size (int) – The customized evaluation batch size of the input data.