AttnLSTM

 Graphs in : cluster, ONNX model architecture.

X – T : object, the input sequences packed (and potentially padded) into one 3-D tensor with the shape of [seq_length, batch_size, input_size].
W – T : object, the weight tensor for the gates. Concatenation of W[iofc] and WB[iofc] (if bidirectional) along dimension 0. The tensor has shape [num_directions, 4*hidden_size, input_size].
R – T : object, the recurrence weight tensor. Concatenation of R[iofc] and RB[iofc] (if bidirectional) along dimension 0. This tensor has shape [num_directions, 4*hidden_size, hidden_size].
B – T : object, the bias tensor for input gate. Concatenation of [Wb[iofc], Rb[iofc]], and [WBb[iofc], RBb[iofc]] (if bidirectional) along dimension 0. This tensor has shape [num_directions, 8*hidden_size]. Optional: If not specified – assumed to be 0.
sequence lens – T1 : object, optional tensor specifying lengths of the sequences in a batch. If not specified – assumed all sequences in the batch to have length seq_length. It has shape [batch_size].
initial_h – T : object, optional initial value of the hidden. If not specified – assumed to be 0. It has shape [num_directions, batch_size, hidden_size].
initial_c – T : object, optional initial value of the cell. If not specified – assumed to be 0. It has shape [num_directions, batch_size, hidden_size].
P – T : object, the weight tensor for peepholes. Concatenation of P[iof] and PB[iof] (if bidirectional) along dimension 0. It has shape [num_directions, 3*hidde_size]. Optional: If not specified – assumed to be 0.
QW – T : object, the weight tensor of the query layer in the attention mechanism. Should be of shape [num_directions, am_query_depth(hidden_size of lstm), am_attn_size].
MW – T : object, the weight tensor of the memory layer in the attention mechanism. Should be of shape [num_directions, memory_depth, am_attn_size].
V – T : object, the attention_v tensor in the attention mechanism. Should be of shape [num_directions, am_attn_size].
M – T : object, the sequence of the memory (input) for attention mechanism. Should be of [batch_size, max_memory_step, memory_depth].
memory seq lens – T1 : object, the sequence length of the input memory for the attention mechanism. Should be of [batch_size].
AW – T : object, the weights of attention layer in the attention wrapper. If exists, should be of shape [num_directions, memory_depth+hidden_size, aw_attn_size]. Please note that attention mechanism context depth is also memory_depth in the attention mechanism.

 Parameters : cluster,

activation alpha : array, optional scaling values used by some activation functions. The values are consumed in the order of activation functions, for example (f, g, h) in LSTM. Default values are the same as of corresponding ONNX operators. For example with LeakyRelu, the default alpha is 0.01.
Default value “empty”.
activation beta : array, optional scaling values used by some activation functions. The values are consumed in the order of activation functions, for example (f, g, h) in LSTM. Default values are the same as of corresponding ONNX operators.
Default value “empty”.
activations : array, a list of 3 (or 6 if bidirectional) activation functions for input, output, forget, cell, and hidden. The activation functions must be one of the activation functions specified above. Optional: See the equations for default if not specified.
Default value “empty”.
clip : float, cell clip threshold. Clipping bounds the elements of a tensor in the range of [-threshold, +threshold] and is applied to the input of activations. No clip if not specified.
Default value “0”.
direction : enum, specify if the RNN is forward, reverse, or bidirectional. Must be one of forward, reverse, or bidirectional.
Default value “forward”.
hidden size : integer, number of neurons in the hidden layer.
Default value “0”.
input forget : boolean, couple the input and forget gates if true.
Default value “False”.
 training? : boolean, whether the layer is in training mode (can store data for backward).
Default value “True”.
 lda coeff : float, defines the coefficient by which the loss derivative will be multiplied before being sent to the previous layer (since during the backward run we go backwards).
Default value “1”.

 name (optional) : string, name of the node.

 Graphs out : cluster, ONNX model architecture.

Y – T : object, a tensor that concats all the intermediate output values of the hidden. It has shape [seq_length, num_directions, batch_size, hidden_size].
Y_h – T : object, the last output value of the hidden. It has shape [num_directions, batch_size, hidden_size].
Y_c – T : object, the last output value of the cell. It has shape [num_directions, batch_size, hidden_size].