WhisperBeamSearch

 Graphs in : cluster, ONNX model architecture.

input_ids (heterogeneous) – F : object, the sequence used as a prompt for the generation in the encoder subgraph. Shape is (batch_size, sequence_length).
max_length (heterogeneous) – I : object, the maximum length of the sequence to be generated. Shape is (1).
min_length (optional, heterogeneous) – I : object, the minimum length below which the score of eos_token_id is set to -Inf. Shape is (1).
num_beams (heterogeneous) – I : object, number of beams for beam search. 1 means no beam search. Shape is (1).
num_return_sequences (heterogeneous) – I : object, the number of returned sequences in the batch. Shape is (1).
length_penalty (optional, heterogeneous) – T : object, exponential penalty to the length. Default value 1.0 means no penalty. Value > 1.0 encourages longer sequences, while values < 1.0 produces shorter sequences. Shape is (1,).
repetition_penalty (optional, heterogeneous) – T : object, the parameter for repetition penalty. Default value 1.0 means no penalty. Accepts value > 0.0. Shape is (1).
vocab_mask (optional, heterogeneous) – M : object, mask of vocabulary. Words that masked with 0 are not allowed to be generated, and 1 is allowed. Shape is (vocab_size).
prefix_vocab_mask (optional, heterogeneous) – M : object, mask of vocabulary for first step. Words that masked with 0 are not allowed to be generated, and 1 is allowed. Shape is (batch_size, vocab_size).
attention_mask (optional, heterogeneous) – I : object, custom attention mask. Shape is (batch_size, sequence_length).
decoder_input_ids (optional, heterogeneous) – I : object, the forced input id sequence for the decoder subgraph. Shape is (batch_size, initial_sequence_length).
logits_processor (optional, heterogeneous) – I : object, specific logits processor for different types of beamsearch models. Default value 0 means no specific logit processor. Accepts value >= 0. Shape is (1).
cross_qk_layer_head (optional, heterogeneous) – I : object, only keep this list of (layer, head) of QK in the final cross_qk output when use_cross_qk is set. Default collect all its shape is (number of (layer, head) to keep, 2), i.e., [[layer_id1, head_id1], [layer_id2, head_id2]……].
extra_decoding_ids (optional, heterogeneous) – I : object, part of the decoder_input_ids that we need cross qk for it. it is of shape (batch_size, extra_decoding_ids_len).In such case, we should remove this from the tail of the decoder_input_ids, and put it here. ids < 0 in it (for multiple batch) are treated as stop of the extra_decoding_ids for corresponding batch.
temperature (optional, heterogeneous) – T : object, temperature value to apply to logits processing during this execution’s decoding. Shape is (1).

 Parameters : cluster,

axis : integer, the id of the first timestamp.
Default value “0”.
decoder : object, decoder subgraph to execute in a loop.
decoder_output_cross_qk : integer, if nozero, decoder subgraph contains output Q*K from cross attentions.
Default value “0”.
decoder_start_token_id : integer, the id of the token that indicates decoding starts (i.e. the start of transcription token id).
Default value “0”.
early_stopping : integer, early stop or not.
Default value “0”.
encoder : object, the subgraph for initialization of encoder and decoder. It will be called once before decoder subgraph.
eos_token_id : integer, the id of the end-of-sequence token.
Default value “0”.
init_decoder : object, the subgraph for the first decoding run. It will be called once before `decoder` subgraph. This is relevant only for the GPT2 model. If this attribute is missing, the `decoder` subgraph will be used for all decoding runs.
model_type : integer, must be 2 for whisper.
Default value “0”.
no_repeat_ngram_size : integer, no repeat ngrams size.
Default value “0”.
no_speech_token_id : integer, the token in whisper model that marks all sequence empty. With this model, whisper could output no_speech_prob after.
Default value “0”.
no_timestamps_token_id : integer, the id of the token that indicates no timestamps.
Default value “0”.
pad_token_id : integer, the id of the padding token.
Default value “0”.
start_of_lm_token_id : integer, the id of the token that indicates LM starts.
Default value “0”.
transcribe_token_id : integer, the id of the transcribe task.
Default value “0”.
translate_token_id : integer, the id of the translate task.
Default value “0”.
vocab_size : integer, size of the vocabulary. If not provided, it will be inferred from the decoder subgraph’s output shape.
Default value “0”.
 training? : boolean, whether the layer is in training mode (can store data for backward).
Default value “True”.
 lda coeff : float, defines the coefficient by which the loss derivative will be multiplied before being sent to the previous layer (since during the backward run we go backwards).
Default value “1”.

 name (optional) : string, name of the node.

 Graphs out : cluster, ONNX model architecture.

sequences (heterogeneous) – I : object, word IDs of generated sequences. Shape is (batch_size, num_return_sequences, max_sequence_length).
sequences_scores (optional, heterogeneous) – T : object, final beam score of the generated sequences. Shape is (batch_size, num_return_sequences).
scores (optional, heterogeneous) – T : object, processed beam scores for each vocabulary token at each generation step. Beam scores consisting of log softmax scores for each vocabulary token and sum of log softmax of previously generated tokens in this beam. Shape is (max_length – sequence_length, batch_size, num_beams, vocab_size).
cross_qk (optional, heterogeneous) – V : object, output the accumulated stacked Q*K in cross attentions. Let H = number of Head of cross attention, F = the frames or kv-seq-len of the cross attention input, T = real decoded token length, L = number of layers, B = batch size, R = num_return_sequences. It then should return tensor of shape [B, R, L*H, T, F]. If cross_qk_layer_head is given, shape is [B, R, cross_qk_layer_head.shape[0], T, F].
non_speech_probs (optional, heterogeneous) – T : object, for whisper model, output the probabilities from logits after encoder and context decoding for the no_speech_token_id. The shape of non_speech_probs is [B].