401 Unauthorized Error When Connecting to Azure OpenAI WebSocket Endpoint

[oswandor]

Minimal steps to reproduce

1. I set up the following variables in my .env file:

AZURE_OPENAI_ENDPOINT=wss://aihubrealtimem6781383663.openai.azure.com
AZURE_OPENAI_REALTIME_DEPLOYMENT=deployment-gpt4orealtime
AZURE_OPENAI_API_KEY=<api_key>
AZURE_SEARCH_ENDPOINT=https://modelstreamingreal.search.windows.net
AZURE_SEARCH_INDEX=azureblob-index
AZURE_SEARCH_API_KEY=<search_api_key>

2. Ran the application in development mode and attempted to establish a WebSocket connection to the configured endpoint.

Any log messages given by the failure

Error logs:

INFO:voicerag:Running in development mode, loading from .env file
======== Running on http://localhost:8765 ========
(Press CTRL+C to quit)
ERROR:aiohttp.server:Error handling request
Traceback (most recent call last):
  File "C:\path\to\project\.venv\Lib\site-packages\aiohttp\web_protocol.py", line 452, in _handle_request
    resp = await request_handler(request)
  ...
aiohttp.client_exceptions.WSServerHandshakeError: 401, message='Invalid response status', url=URL('wss://aihubrealtimem6781383663.openai.azure.com/openai/realtime?api-version=2024-10-01-preview&deployment=deployment-gpt4orealtime')

Expected/desired behavior

The WebSocket connection to the Azure OpenAI endpoint should establish successfully without any authentication errors.

OS and Version

Operating System: Windows 10 / 11 (please specify)

Additional Context / Help Needed

I keep receiving a 401 Unauthorized error when trying to establish the WebSocket connection, which seems to be related to authentication. I am using the recommended values in the .env file, but I still can't connect successfully. Could someone help identify the possible causes or suggest a solution for this error?

attached is the image of the screenshot from where I'm getting the information from the .env file

this information I am adding these doing something wrong ?
thank you, I am waiting for your help.

[pamelafox]

FYI, GitHub reports that you revealed your search key in this issue. Make sure you rotate tha now.
Also, part of your key is in the screenshot for GPT, better to blank that out.

As for the error: how did you provision that GPT model? Did you provision it in the Portal or via our infrastructure (azd up)? If you used azd up, then you won't be able to use keys with it, since it's disabled:
https://github.com/Azure-Samples/aisearch-openai-rag-audio/security/secret-scanning/1
If you provisioned it yourself, then you'll need to check the Azure OpenAI resource in the Portal to see if key access is allowed. For example, this is what I see:

Assuming that key access is allowed, then your setup looks reasonable. I'd suggest adding more logging in the code that setups the credential to ensure it's using AzureKeyCredential with your key.

[bilalqayoom]

instead of wss: use https: