Feature request: Transcription of conversation

[jakeatmsft]

Is there anyway to add transcript to the demo?

[chuwik]

It's not currently displayed in the app, but you have the hooks ready to plug it in.

If you want a transcript of the user input audio:

• useRealTime takes a enableInputAudioTranscription param, pass true
• useRealTime takes a onReceivedInputAudioTranscriptionCompleted param, pass a function that decides what to do with the message.

If you want a transcript of the responses:

• useRealTime takes a onReceivedResponseDone param, pass a function that decides what to do with the message. Checkout ResponseDone in types.ts to see what it looks like.

I recommend you hack these together in App.tsx with simple console.log to start, then go from there.

We have plans to surface these plus the grounding data in a 'transcript' view, but no ETA.

[jakeatmsft]

This code has been implemented in my fork: https://github.com/jakeatmsft/promptflow_patterns/blob/main/voice_app/app/frontend/src/App.tsx

[pamelafox]

Thanks @jakeatmsft for sharing! I'm re-opening this issue as it's a common request. @chuwik says that we also want a way to link the transcript to the grounding files ideally, for a full solution.

[chuwik]

#44 PR in review adding this functionality