Copy/paste images and Speech capabilities in the front-end application to allow GPT-4o scenarios

[davidcervigon]

Hi

Any plans to update the front end application to include the same picture upload/embed in the chat and speech features, as in the Azure AI Studio playground?. Any other repo with a frontend ready for these multimodal features, yet able to chat with your data sources?

Thanks

[ddarmonski]

I am also very interested.

[dominikbrand]

I am also very interested.

[fredparadis]

Same here ! Anyone found anything about this ?

[stbere]

I deployed this app thru ai.azure.com and it led me to this code repo. In the chat playground, there is an option under Chat capabilities to add the voice feature to the model. There also looks like an option to download a .zip file to deploy as a Teams app, which looks awesome. I am going to try deploying using these features and methods tomorrow at work! Will report back.

[stbere]

I also see Promptflow variables in the .env.sample file, so it looks like we can plug in some promptflow into the app:

USE_PROMPTFLOW=False
PROMPTFLOW_ENDPOINT=
PROMPTFLOW_API_KEY=
PROMPTFLOW_RESPONSE_TIMEOUT=120
PROMPTFLOW_REQUEST_FIELD_NAME=query
PROMPTFLOW_RESPONSE_FIELD_NAME=reply
PROMPTFLOW_CITATIONS_FIELD_NAME=documents

[aashish181188]

I integrated Promptflow with my WebApp and I am able to run model evaluation on Promptflow at azure side..
How are you using this repo without promptflow ?? Are you using Azure Search endpoints ?? like below : OR Something else ?

Chat with data: Azure AI Search

AZURE_SEARCH_SERVICE=
AZURE_SEARCH_INDEX=
AZURE_SEARCH_KEY=
AZURE_SEARCH_SEMANTIC_SEARCH_CONFIG=
AZURE_SEARCH_INDEX_IS_PRECHUNKED=
AZURE_SEARCH_TOP_K=
AZURE_SEARCH_ENABLE_IN_DOMAIN=
AZURE_SEARCH_CONTENT_COLUMNS=
AZURE_SEARCH_FILENAME_COLUMN=
AZURE_SEARCH_TITLE_COLUMN=
AZURE_SEARCH_URL_COLUMN=
AZURE_SEARCH_VECTOR_COLUMNS=
AZURE_SEARCH_QUERY_TYPE=
AZURE_SEARCH_PERMITTED_GROUPS_COLUMN=