AdditiveAttention

Description

Gets the weights of the AdditiveAttention layer selected by the name. Type : polymorphic.

Input parameters

 Model in : model architecture.
 name : string, name of layer.

Output parameters

 Model out : model architecture.

Dimension

• scale = query[2] = value[2] = key[2]

The size of scale depends on the size of the query, value and key entries in the AdditiveAttention layer.
For example, if query has a size of [batch_size = 5, Tq = 3, dim = 1], value a size of [batch_size = 10, Tv = 4, dim = 1] and key a size of [batch_size = 8, Tv = 6, dim = 1] then the size of scale is [dim = 1].
Another example, if query has a size of [batch_size = 10, Tq = 9, dim = 5], value a size of [batch_size = 15, Tv = 10, dim = 5] and key a size of [batch_size = 9, Tv = 7, dim = 5] then the size of scale is [dim = 5].
query, value and key will always have the same value at index 2 of their size, which will be the size of scale.

Example