Document Upload (In Progress)

[cbutler-captech]

All, I have a team internally working on a document upload feature (chat with your document). The goal is secure and isolated chat with oneoff documents. We are building this internally but have plans to push it back as a pull request when complete.

Once complete you will be able to:

Upload a document
Chat with it - ensuring that noone else can access it. Document will only be available to the current conversation (or user, depending on configuration)
Chat with any documents marked as global (configurable)

Tech Stack:
Azure AI Search
Azure OpenAI/Vector Embeddings
Azure Storage Blobs

Please add your thoughts...

[ianlcassidy]

This would be a great addition to this repo!

[cbutler-captech]

Update on this. This is going into internal testing this week. We have successfully uploaded documents, chatted with them and restricted them by either conversation or user such that they are private and noone else can see them and the chat just returns a generic result on other conversations. Working on some front end items and edge cases.

Other features:
Delete blobs on conversation delete (configurable)
Indexer cloning to increase concurrency
Folder configuration
Global document configuration by tag

[ianlcassidy]

@cbutler-captech - thanks for the update. do you know if this functionality will allow you to upload a document and compare it to an existing AI Search index? for example, you upload a document and then ask the bot to make any suggestions or point out gaps based on similar documents that have already been indexed? i have not seen a tool for doing this yet, but am interested in this functionality.

[cbutler-captech]

If those documents are accessible (forgot we also have global document configuration), it is likely that it can do this.

[cbutler-captech]

Note we are testing with gpt-3.5 just to keep costs low for now. A great addition to this repo is model selection by conversation. I will add a new thread for that.

[ianlcassidy]

got it, that all makes sense.

[ianlcassidy]

hey @cbutler-captech - any updates on this?

[eshwarhs]

Can you put up a WIP open PR so that we can see the code and test it out as well?

[cbutler-captech]

I have a first pass at the documentation done, once we get the rework of the index polling mechanism completed I will push it out there

[jasredyy5]

Hey @cbutler-captech, do you have an estimate on when this can become a PR? Thank you!

[cbutler-captech]

Still jumping through internal hoops. Trying to push this forward.

[jasredyy5]

Thank You. Looking forward to this!

[cbutler-captech]

Thank you for your patience. We are new to the open source world and I had to make sure I had all my ducks in a row. Setup is kindof tricky. See the readme_docupload.md and each out if you have any issues.

#862

[cbutler-captech]

I may need to update the security section to be more helpful, Basically, all managed servies need rights to the other services.

• Blob storage
  • Grant Reader and Data Access to both your OpenAI and AISearch managed identities
  • Grant Storage and Blob Data Reader to you r OpenAI and AISearch managed identities
• OpenAI
  • Grant Azure AI Developer to your AI Search Managed identity (may be need to examine for a lesser privilige)

[cbutler-captech]

Excellent work! I have a proposal for a small adaptation. It would be beneficial to add an option to load the file content directly into the chat context without using blob storage. This would simplify the process by eliminating the need to save the file or make it accessible to others. It would enhance the functionality and make it even more user-friendly. What do you think?

EDIT: To clarify further, we want to maintain control over which documents are stored in the VectorDB, but we would like to give users the ability to upload documents directly into the context of a chat.

Technically speaking the blobs are unnecessary after getting into the vector search. If you can figure out the code to replicate what AI search is doing to create the chunks and just push them directly in there with the conversation and userids, that would definately be better. I looked into it for a while but was going in circles and all docs were telling me to use AI Search. I know I can make the vectors and chunks but AI Search does a little more and we decided to let it be.

[jasredyy5]

I may need to update the security section to be more helpful, Basically, all managed servies need rights to the other services.

• Blob storage

  • Grant Reader and Data Access to both your OpenAI and AISearch managed identities
  • Grant Storage and Blob Data Reader to you r OpenAI and AISearch managed identities

• OpenAI

  • Grant Azure AI Developer to your AI Search Managed identity (may be need to examine for a lesser privilige)

This is helpful. Thank You!

[eshwarhs]

@cbutler-captech I set up the code locally and I observed this behaviour -
After a file is uploaded in a conversation and a new conversation is started using the '+', any new file uploaded, is appending it to the previous conversation. This is probably due to the callback function "handleUploadSuccess" and "handleConversationIdUpdate" along with the SET_ACTIVE_CONVERSATION_ID event. Can you confirm if this behavior is consistent on your code as well?

[cbutler-captech]

I will look into this one. My front end expert is no longer available but I may be able to get it.

[patelha-ccm]

Hey @cbutler-captech , any update on this feature? When can we expect it to roll out. I am really excited of its use-case specifically within our team :)

[cbutler-captech]

We had some more front end improvements going but I think this will be the PR as is. I will update and take it out of WIP.

[stbere]

Thank you! This looks pretty good, can't wait to get this integrated into my app.

[ukiguy]

Looking forward to using this feature. Do we have any ETA on this feature release? Thank you for the hard work.

[misslapina123]

What is the status of this feature?

[2gad]

Not sure who this is helpful for but there is this other repo (managed by Azure Samples) https://github.com/Azure-Samples/azure-search-openai-demo that has a similar chat app but has a file upload function

I am planning to try to integrate this into what I have forked from sample-app-aoai-chatGPT

[stbere]

So I use that repo and this one as well. This repo we are in is tied to the ai.azure.com studio, which I like. but there are stark differences I'm seeing, and one is that the citation for this particular repo is lacking, whereas the link you shared has a beautiful PDF web viewer for the citation panel.

[cbutler-captech]

My apologies, my team has disbanded and I no longer have the front end resources to dedicate to this feature. I hope someone can take this over and get any issues resolved and get it into the main branch as we are evaluating other front ends and the potential to put effort into custom Copilots.

[davidgristwood]

That's a real shame. Are you saying this feature is effectively dead unless someone picks up the work and does a pull request?

[davidgristwood]

Copilot been getting a lot of attention as a UI and orchestrator - I'd appreciate any insights into whether it has a role in this scenario, or is this a prioritisation on copilot?

[jeannot-prvlimburg]

You could switch to azure-search-openai-demo, which already has it implented.

[stbere]

When did they add this? I've been using both. I do like how the openai-demo now has voice integration, but haven't implemented it into my product yet.