support for libcudnn-9 and libcublas-12

[JohnRDOrazio]

I have been trying to install without success the Stable-Diffusion-WebUI-TensorRT extension in Stable Diffusion Forge.

I see in the Readme that cudNN-8.9 is recommended with cuda-12.

Is there any chance that this dependency will be updated to cudNN-9? I'm going down a rabbit hole trying to get everything to work together, and I haven't yet succeeded.

/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/tensorrt_bindings/../tensorrt_libs/libnvinfer_plugin.so.9 is requiring libcudnn.so.8 and libcublas.so.11, whereas I have libcudnn.so.9 and libcublas.so.12. I have tried downgrading so many different things to get this to work, I downgraded pytorch from 2.5.1 to 2.3.1, but then I start getting errors about other xformers requiring pytorch 2.5.1. I tried downgrading xformers, but each time I try to downgrade something, something else starts complaining about incompatible versions.

Perhaps the easiest solution would be to support more recent versions of libcudnn and libcublas?

[JohnRDOrazio]

In the meantime I seem to have finally succeeded in getting tensorrt to install in forge with pytorch==2.5.1 by manually running pip install --pre --extra-index-url https://pypi.nvidia.com tensorrt==9.0.1.post11.dev4 --no-cache-dir, this seems to have installed all the requirements. I seem to have wound up with some duplicated packages but at least now the extension is loaded and I can now see the tensorrt tab in the web ui.

This is the current state when I run pip freeze:

absl-py==2.1.0
accelerate==0.21.0
aenum==3.1.15
aiofiles==23.2.1
aiohappyeyeballs==2.4.4
aiohttp==3.11.11
aiosignal==1.3.2
albucore==0.0.23
albumentations==1.4.24
annotated-types==0.7.0
antlr4-python3-runtime==4.9.3
anyio==3.7.1
async-timeout==5.0.1
attrs==24.3.0
bitsandbytes==0.43.3
blendmodes==2022
certifi==2024.12.14
cffi==1.17.1
chardet==5.2.0
charset-normalizer==3.4.1
clean-fid==0.1.35
click==8.1.8
clip==1.0
coloredlogs==15.0.1
colorlog==6.9.0
contourpy==1.3.1
cssselect2==0.7.0
cycler==0.12.1
Cython==3.0.11
datasets==3.2.0
deprecation==2.1.0
depth_anything @ https://github.com/huchenlei/Depth-Anything/releases/download/v1.0.0/depth_anything-2024.1.22.0-py2.py3-none-any.whl
depth_anything_v2 @ https://github.com/MackinationsAi/UDAV2-ControlNet/releases/download/v1.0.0/depth_anything_v2-2024.7.1.0-py2.py3-none-any.whl
diffusers==0.31.0
dill==0.3.8
diskcache==5.6.3
easydict==1.13
einops==0.4.1
embreex==2.17.7.post5
exceptiongroup==1.2.2
facexlib==0.3.0
fastapi==0.104.1
ffmpy==0.5.0
filelock==3.16.1
filterpy==1.4.5
flatbuffers==24.12.23
fonttools==4.55.3
frozenlist==1.5.0
fsspec==2024.9.0
ftfy==6.3.1
fvcore==0.1.5.post20221221
gitdb==4.0.11
GitPython==3.1.32
gradio==4.40.0
gradio_client==1.2.0
gradio_imageslider==0.0.20
gradio_rangeslider==0.0.6
h11==0.12.0
handrefinerportable @ https://github.com/huchenlei/HandRefinerPortable/releases/download/v1.0.1/handrefinerportable-2024.2.12.0-py2.py3-none-any.whl
httpcore==0.15.0
httpx==0.24.1
huggingface-hub==0.26.2
humanfriendly==10.0
idna==3.10
imageio==2.36.1
importlib_metadata==8.5.0
importlib_resources==6.4.5
inflection==0.5.1
insightface==0.7.3
iopath==0.1.9
jax==0.4.38
jaxlib==0.4.38
Jinja2==3.1.5
joblib==1.4.2
jsonmerge==1.8.0
jsonschema==4.23.0
jsonschema-specifications==2024.10.1
kiwisolver==1.4.8
kornia==0.6.7
lark==1.1.2
lazy_loader==0.4
lightning-utilities==0.11.9
llvmlite==0.43.0
loadimg==0.1.2
lxml==5.3.0
manifold3d==3.0.1
mapbox_earcut==1.0.3
markdown-it-py==3.0.0
MarkupSafe==2.1.5
matplotlib==3.10.0
mdurl==0.1.2
mediapipe==0.10.15
ml_dtypes==0.5.0
mpmath==1.3.0
multidict==6.1.0
multiprocess==0.70.16
networkx==3.4.2
numba==0.60.0
numpy==1.26.2
nvidia-cublas-cu11==2022.4.8
nvidia-cublas-cu117==11.10.1.25
nvidia-cublas-cu12==12.4.5.8
nvidia-cuda-cupti-cu12==12.4.127
nvidia-cuda-nvrtc-cu12==12.4.127
nvidia-cuda-runtime-cu11==2022.4.25
nvidia-cuda-runtime-cu117==11.7.60
nvidia-cuda-runtime-cu12==12.4.127
nvidia-cudnn-cu11==2022.5.19
nvidia-cudnn-cu116==8.4.0.27
nvidia-cudnn-cu12==9.1.0.70
nvidia-cufft-cu12==11.2.1.3
nvidia-curand-cu12==10.3.5.147
nvidia-cusolver-cu12==11.6.1.9
nvidia-cusparse-cu12==12.3.1.170
nvidia-nccl-cu12==2.21.5
nvidia-nvjitlink-cu12==12.4.127
nvidia-nvtx-cu12==12.4.127
nvidia-pyindex==1.0.9
nvidia-tensorrt==99.0.0
omegaconf==2.2.3
onnx==1.17.0
onnx-graphsurgeon==0.5.2
onnxruntime==1.20.1
open-clip-torch==2.20.0
opencv-contrib-python==4.10.0.84
opencv-python==4.10.0.84
opencv-python-headless==4.10.0.84
opt_einsum==3.4.0
optimum==1.23.3
orjson==3.10.13
packaging==24.2
pandas==2.2.3
peft==0.13.2
piexif==1.1.3
Pillow==9.5.0
pillow-avif-plugin==1.4.3
platformdirs==4.3.6
polygraphy==0.49.14
pooch==1.8.2
portalocker==3.1.0
prettytable==3.12.0
propcache==0.2.1
protobuf==3.20.0
psutil==5.9.5
py-cpuinfo==9.0.0
pyarrow==19.0.0
pycollada==0.8
pycparser==2.22
pydantic==2.8.2
pydantic_core==2.20.1
pydub==0.25.1
Pygments==2.18.0
PyMatting==1.1.13
pyparsing==3.2.0
python-dateutil==2.9.0.post0
python-multipart==0.0.20
pytorch-lightning==1.9.4
pytz==2024.2
PyWavelets==1.8.0
PyYAML==6.0.2
referencing==0.35.1
regex==2024.11.6
rembg==2.0.50
reportlab==4.2.5
requests==2.32.3
resize-right==0.0.2
rich==13.9.4
rpds-py==0.22.3
Rtree==1.3.0
ruff==0.8.4
safetensors==0.4.2
scikit-image==0.21.0
scikit-learn==1.6.0
scipy==1.14.1
seaborn==0.13.2
semantic-version==2.10.0
sentencepiece==0.2.0
shapely==2.0.6
shellingham==1.5.4
simsimd==6.2.1
six==1.17.0
smmap==5.0.1
sniffio==1.3.1
sounddevice==0.5.1
spandrel==0.3.4
spandrel_extra_arches==0.1.1
starlette==0.27.0
stringzilla==3.11.3
svg.path==6.3
svglib==1.5.1
sympy==1.13.1
tabulate==0.9.0
tensorrt==9.0.1.post11.dev4
tensorrt-bindings==9.0.1.post11.dev4
tensorrt-cu12-bindings==10.7.0
tensorrt-cu12-libs==10.7.0
tensorrt-libs==9.0.1.post11.dev4
tensorrt_cu12==10.7.0
termcolor==2.5.0
threadpoolctl==3.5.0
tifffile==2024.12.12
timm==1.0.12
tinycss2==1.4.0
tokenizers==0.20.3
tomesd==0.1.3
tomli_w==1.1.0
tomlkit==0.12.0
torch==2.5.1
torchdiffeq==0.2.3
torchmetrics==1.6.1
torchsde==0.2.6
torchvision==0.20.1
tqdm==4.66.1
trampoline==0.1.2
transformers==4.46.1
trimesh==4.5.3
triton==3.1.0
typer==0.15.1
typing_extensions==4.12.2
tzdata==2024.2
ultralytics==8.3.56
ultralytics-thop==2.0.13
urllib3==2.3.0
uvicorn==0.34.0
vhacdx==0.0.8.post1
wcwidth==0.2.13
webencodings==0.5.1
websockets==12.0
xatlas==0.0.9
xformers==0.0.29.post1
xxhash==3.5.0
yacs==0.1.8
yarl==1.18.3
zipp==3.21.0

However there seem to be other issues with forge, I can't "export default engine" (see NVIDIA/Stable-Diffusion-WebUI-TensorRT/issues/345 and all the open issues on the forge repo )