Welcome to our Support Center
Updated

Create Inference Session From Model

Description

Initialize an Inference Session from a DeepLearning Toolkit Model. Type : polymorphic.

 

Input parameters

 

 Execution Device : enum, selects the hardware device on which the model will run.
 Model in : object, model architecture.

 Parameters : cluster,

 Sessions Parameters : cluster

 intra_op_num_threads : integer, number of threads used within each operator to parallelize computations. If the value is 0, ONNX Runtime automatically uses the number of physical CPU cores.
 inter_op_num_threads : integer, number of threads used between operators, to execute multiple graph nodes in parallel. If set to 0, this parameter is ignored when execution_mode is ORT_SEQUENTIAL. In ORT_PARALLEL mode, 0 means ORT automatically selects a suitable number of threads (usually equal to the number of cores).
 execution_mode : enum, controls whether the graph executes nodes one after another or allows parallel execution when possible.ORT_SEQUENTIAL runs nodes in order, ORT_PARALLEL runs them concurrently.
 deterministic_compute : boolean, forces deterministic execution, meaning results will always be identical for the same inputs.
 graph_optimization_level : enum, defines how much ONNX Runtime optimizes the computation graph before running the model.
 optimized_model_file_path : path, file path to save the optimized model after graph analysis.

 CUDA Parameters : cluster

 device id : integer, selects which GPU to use (0 = first GPU).
 algo : enum, controls the algorithm used for cuDNN convolutions.

Output parameters

 

Inference out : object, inference session.

Example

All these exemples are snippets PNG, you can drop these Snippet onto the block diagram and get the depicted code added to your VI (Do not forget to install Deep Learning library to run it).
Table of Contents