Welcome to our Support Center
Updated

QuantizeBFP

Description

The BFP quantization operator. It consumes a full precision tensor and computes an BFP tensor. More documentation on the BFP format can be found in this paper: https://www.microsoft.com/en-us/research/publication/pushing-the-limits-of-narrow-precision-inferencing-at-cloud-scale-with-microsoft-floating-point/

 

Input parameters

 

specified_outputs_name : array, this parameter lets you manually assign custom names to the output tensors of a node.
x (heterogeneous) – T1 : object, N-D full precision input tensor to be quantized.

 Parameters : cluster,

bfp_type : enum, the type of BFP.
Default value “BFP_1_8_8_16”.
block_dim : integer, each bounding box spans this dimension. Typically, the block dimension corresponds to the reduction dimension of the matrix multipication that consumes the output of this operator.For example, for a 2D matrix multiplication A@W, QuantizeBFP(A) would use block_dim 1 and QuantizeBFP(W) would use block_dim 0. The default is the last dimension.
Default value “0”.
 training? : boolean, whether the layer is in training mode (can store data for backward).
Default value “True”.
 lda coeff : float, defines the coefficient by which the loss derivative will be multiplied before being sent to the previous layer (since during the backward run we go backwards).
Default value “1”.

 name (optional) : string, name of the node.

Output parameters

 Graphs out : cluster, ONNX model architecture.

y (heterogeneous) – T2 : object, 1-D, contiguous BFP data.
shape (heterogeneous) – T3 : object, shape of x.
strides (heterogeneous) – T3 : object, strides of x.

Type Constraints

T1 in (tensor(float), tensor(float16)), tensor(bfloat16) : Constrain the input to float and bfloat.

T2 in (tensor(uint8)) : Constrain y to uint8.

T3 in (tensor(int64)) : Constrain shape and strides to uint64.

Example

All these exemples are snippets PNG, you can drop these Snippet onto the block diagram and get the depicted code added to your VI (Do not forget to install Deep Learning library to run it).
Table of Contents