Welcome to our Support Center
Updated

QOrderedLayerNormalization

Description

QOrderedLayerNormalization applies a LayerNormalization operation on quantized tensors.

 

Input parameters

 

specified_outputs_name : array, this parameter lets you manually assign custom names to the output tensors of a node.

 Graphs in : cluster, ONNX model architecture.

X (heterogeneous) – Q : object, input data tensor from the previous layer.
scale_X (heterogeneous) – S : object, scale of the quantized X.
scale (heterogeneous) – F : object, scale tensor, i.e., gamma vector.
B (heterogeneous) – F : object, bias tensor.
scale_Y (heterogeneous) – S : object, scale of the quantized X.

 Parameters : cluster,

axis : integer, the first normalization dimension: normalization will be performed along dimensions axis : rank(inputs).
Default value “-1”.
epsilon : float, the epsilon value to use to avoid division by zero.
Default value “1E-5”.
order_X : enum, cublasLt order of input X. Optional. See the schema of QuantizeWithOrder for order definition.
Default value “ORDER_ROW”.
order_Y : enum, cublasLt order of matrix Y, must be same as order_X if specified together. Optional.
Default value “ORDER_ROW”.
 training? : boolean, whether the layer is in training mode (can store data for backward).
Default value “True”.
 lda coeff : float, defines the coefficient by which the loss derivative will be multiplied before being sent to the previous layer (since during the backward run we go backwards).
Default value “1”.

 name (optional) : string, name of the node.

Output parameters

 

 Y (heterogeneous) – Q : object, output data tensor.

Type Constraints

F in (tensor(float), tensor(float16)) : Constrain input gamma and bias could be float16/float tensors. float may get better precision, float16 runs faster.

S in (tensor(float)) : Quantization scale must be float tensors.

Q in (tensor(int8)) : Quantization tensor must be int8 tensors.

Example

All these exemples are snippets PNG, you can drop these Snippet onto the block diagram and get the depicted code added to your VI (Do not forget to install Deep Learning library to run it).
Table of Contents