FP16 ONNX model outputs NaN after the first successful execution

[MiningIrving]

Describe the issue

I have an FP16 (half-precision floating point) ONNX model. When I load and execute this model using the onnxruntime library in Python, the first execution is successful and produces correct results. However, starting from the second execution, all subsequent outputs are NaN (Not a Number). My codes as fellows:
providers =['CPUExecutionProvider'] model = onnxruntime.InferenceSession( model_path,providers=providers ) for d in os.listdir(dataset_path): if os.path.isdir(os.path.join(dataset_path, d)): time1 =time.time() image_path = dataset_path+d+"/frame.png" image = cv2.imread(image_path) image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) shape_img = image.shape[0:2] txt_path = dataset_path+d+"/eyepointData.txt" with open(txt_path, "r") as file: eyepoint = file.readline().strip().split(',') eye_x, eye_y = map(float, eyepoint) list_txt = [] list_txt.append(eye_x) list_txt.append(eye_y) #print(list_txt) input_point = np.array([list_txt]) point_data = process_point(image,input_point).astype(np.float16) # print(point_data) image_data = process_img(image).astype(np.float16) # print(image_data) #image_data = np.random.random((684, 1024, 3)).astype(np.float16) ort_inputs = { "encoder_input_image": image_data, "point_coords": point_data } masks = model.run(['masks'], ort_inputs) print(masks)

I am experiencing a very strange issue where all outputs are normal only when I reload the model before each inference. Could someone please help me solve this problem? Thank you.

To reproduce

rename the file as .onnx

Urgency

No response

Platform

Linux

OS Version

Ubuntu 22.04

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.16.0

ONNX Runtime API

Python

Architecture

X86

Execution Provider

Default CPU

Execution Provider Library Version

No response

[tianleiwu]

Please prepare an easy way to reproduce including: python test script file, onnx model, test image or other data file, python version and requirements.txt for installing required python packages, and command lines.

[MiningIrving]

Please prepare an easy way to reproduce including: python test script file, onnx model, test image or other data file, python version and requirements.txt for installing required python packages, and command lines.

my code and onnx model in my cloud .the address is https://drive.google.com/file/d/1pTHam3bgzNnwjDujxfwqSNc_HjBgKVOO/view?usp=drive_link

`import numpy as np
providers =['CPUExecutionProvider']
import os
model_path = "512_mobilesam_sim_fp16.onnx"
import onnxruntime

model = onnxruntime.InferenceSession(
model_path,providers=providers
)
for i in range(10):
image_data = np.random.random((1, 3, 256,256)).astype(np.float16)
point_data = np.random.random((1, 2,2)).astype(np.float16)
ort_inputs = {
"encoder_input_image": image_data,
"point_coords": point_data
}
masks = model.run(['masks'], ort_inputs)
print(masks)

`