Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to create custom op with fp16 input #23373

Open
CapJunkrat opened this issue Jan 15, 2025 · 0 comments
Open

How to create custom op with fp16 input #23373

CapJunkrat opened this issue Jan 15, 2025 · 0 comments

Comments

@CapJunkrat
Copy link

Describe the issue

I tried to use custom op functionality to create a custom op (what it does is exactly the same with QuantizeLinear) with fp16 type input, but when I try to inference it, onnxruntime throws error:

onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running HzQuantize node. Name:'HzQuantize_0' Status Message: unknow input type

I checked the function that throws the error, it is "Variadic" initialization function in file "onnxruntime_lite_custom_op.h" , and Float16_t and BFloat16 types are not in the "switch (type)" command, and "ORT_CXX_API_THROW("unknow input type", ORT_RUNTIME_EXCEPTION);" is given from the default switch branch.

Is this done on purpose or just fp16 function not supported yet? Is there a way to give custom op fp16 input type?

To reproduce

N/A

Urgency

No response

Platform

Linux

OS Version

Centos 7

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

latest(cff0ec5)

ONNX Runtime API

C++

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant