Docs/oct updates v2 (#193)

* docs / refactored

Move versions to docs/workshop, keep core README in sync with docs/README for Samples browser.

* fix / docs for clarity

.gitignore

@@ -33,4 +33,5 @@ src/api/result.jsonl
 src/api/eval_results.jsonl
 src/api/eval_results.md
 docs/workshop/site
-sandbox/*
+sandbox/*
+docs/workshop/docs/.DS_Store

README.md

@@ -26,7 +26,7 @@ description: Build, evaluate, and deploy, a RAG-based retail copilot that respon
 ## Table of Contents
 
 - [Overview](#overview)
-- [Features](#features) · [Architecture Diagram](#architecture-diagram) · [Demo Video](#demo-video)
+- [Features](#features) · [Architecture Diagram](#architecture-diagram) 
 - [Pre-Requisites](#pre-requisites)
 - [Getting Started](#getting-started) 
     - [GitHub Codespaces](#github-codespaces) 
@@ -45,6 +45,8 @@ description: Build, evaluate, and deploy, a RAG-based retail copilot that respon
 
 This template, the application code and configuration it contains, has been built to showcase Microsoft Azure specific services and tools. We strongly advise our customers not to make this code part of their production environments without implementing or enabling additional security features.  
 
+For a more comprehensive list of best practices and security recommendations for Intelligent Applications, visit our [official documentation](https://learn.microsoft.com/azure/developer/ai/get-started-securing-your-ai-app).
+
 > [!WARNING]  
 >
 > **Some of the features used in this repository are in preview.** Preview versions are provided without a service level agreement, and they are not recommended for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/en-us/support/legal/preview-supplemental-terms/).
@@ -92,10 +94,6 @@ It also comes with:
 ### Architecture Diagram 
 ![Architecture](./docs/img/arch-contoso-retail-aca.png)
 
-### Demo Video
-
-(In Planning) - Get an intuitive sense for how simple it can be to go from template discovery, to codespaces launch, to application deployment with `azd up`. Watch this space for a demo video.
-
 ## Pre-requisites
 
 To deploy and explore the sample, you will need:
@@ -348,23 +346,12 @@ The sample has a `docs/workshop` folder with step-by-step guidance for developer
 
 Have issues or questions about the workshop? Submit [a new issue](https://github.com/Azure-Samples/contoso-chat/issues/new) with a `documentation` tag.
 
-### Versions
-
-The Contoso Chat sample has undergone numerous architecture and tooling changes since its first version back in 2023. The table below links to legacy versions for awareness only. **We recommend all users start with the latest version to leverage the latest tools and practices**.
-
-> | Version | Description |
-> |:---|:---|
-> | v0 : [#cc2e808](https://github.com/Azure-Samples/contoso-chat/tree/cc2e808eee29768093866cf77a16e8867adbaa9c) | MSAITour 2023-24 (dag-flow, jnja template) - Skillable + Script |
-> | v1 : [msbuild-lab322](https://github.com/Azure-Samples/contoso-chat/tree/msbuild-lab322) | MSBuild 2024 (dag-flow, jnja template) - Skillable + Script |
-> | v2 : [raghack-24](https://github.com/Azure-Samples/contoso-chat/tree/raghack-24) | RAG Hack 2024 (flex-flow, prompty asset) - AZD Template |
-> | v3 : [aitour-WRK550](https://github.com/Azure-Samples/contoso-chat/tree/raghack-24)  🆕| MSAITour 2024-25 (prompty asset, ACA)- Skillable + AZD |
-> | v3.x : [main](https://github.com/Azure-Samples/contoso-chat/tree/raghack-24) | Latest version of codebase (in active development) |
-> | | |
 
 ## Resources
 
 1. [Prompty Documentation](https://prompty.ai)
 1. [Azure AI Studio Documentation](https://aka.ms/aistudio)
+1. [Develop AI Apps using Azure AI Services](https://aka.ms/ai-apps-docs)
 1. [Azure AI Templates with Azure Developer CLI](https://aka.ms/ai-studio/azd-templates)
 
 

docs/workshop/README.md

@@ -1,29 +1,59 @@
-# Contoso-Chat Workshop 
+# Contoso-Chat: Hands-on Workshop
+
+[![Open in GitHub Codespaces](https://img.shields.io/static/v1?style=for-the-badge&label=GitHub+Codespaces&message=Open&color=brightgreen&logo=github)](https://github.com/codespaces/new?hide_repo_select=true&machine=basicLinux32gb&repo=725257907&ref=main&devcontainer_path=.devcontainer%2Fdevcontainer.json&geo=UsEast)
+[![Open in Dev Containers](https://img.shields.io/static/v1?style=for-the-badge&label=Dev%20Containers&message=Open&color=blue&logo=visualstudiocode)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/azure-samples/contoso-chat)
+
+---
+
+## About Contoso Chat
+
+The Contoso Chat repository provides a reference sample for an Azure AI Architecture and workflow to build a custom RAG-based copilot **code-first** on Azure AI Studio. The sample has been actively used to skill up developers in core Azure AI tools, services, and practices, since its creation in 2023.
+
+**The current version (v3) of the sample follows this architecture**.
+
+![](./../img/arch-contoso-retail-aca.png)
+
+## Workshop Versions
+
+The Contoso Chat sample has been used to run hands-on workshops for different internal and external developer audiences. This table tracks the versions for historical reference, identifying the key capabilities that were in focus at the time.
+
+
+> | Version | Description | Technologies |
+> |:---|:---|:---|
+> | [v0](https://github.com/Azure-Samples/contoso-chat/tree/cc2e808eee29768093866cf77a16e8867adbaa9c) | #MSAITour Nov 2023 (Skillable) | Prompt flow (DAG), Jnja (template), Provisioning (scripts) |
+> | [v1](https://github.com/Azure-Samples/contoso-chat/tree/msbuild-lab322) | #MSBuild May 2024 Lab322 (Skillable) | Prompt flow (DAG), Jnja (template), Provisioning (scripts) |
+> | [v2](https://github.com/Azure-Samples/contoso-chat/tree/raghack-24) | #RAGHack 2024 (Self-Guided) | Prompt flow (Flex), Prompty (template), Provisioning (AZD), Hosting (AIP) | 
+> | [v3](https://github.com/Azure-Samples/contoso-chat/tree/raghack-24)  🆕| MSAITour 2024-25 (prompty asset, ACA)- Skillable + AZD | Prompty (template), Python (runtime), Provisioning (AZD), Hosting (ACA) | 
+> | [main](https://github.com/Azure-Samples/contoso-chat/tree/raghack-24) | Version that will be in active development (RAG, GenAIOps) | Provisioning Ideation - Evaluation - Deployment - Monitoring - CI/CD | 
+> | | |
+
 This folder contains the content for the Contoso-Chat workshop. It is written in Markdown using the [mkdocs Admonitions](https://squidfunk.github.io/mkdocs-material/reference/admonitions/?h=ad) extensions. 
 
 You can read this content with any Markdown viewer (for example, Visual Studio Code or GitHub). Start here: [Build a Retail Copilot Code-First on Azure AI](docs/index.md).
 
 For the best experience build the documentation and view it in a browser window using the instructions below.
 
-## How to Buid the Workshop Content from CodeSpaces
-
-Using CodeSpaces on this repository, you can do this as follows:
-
-```
-cd docs/workshop
-mkdocs serve -a localhost:5000
-```
+## Workshop Guide
 
-VS Code will prompt to open the content in a browser window for you.
+The current repository is instrumented with a `docs/workshop/` folder that contains the step-by-step lab guide for developers, covering the entire workflow from resource provisioning to ideation, evaluation, deployment, and usage.
 
+The workshop is designed to be used in two modes:
+ - Instructor led workshops (e.g., #MSAITour, #MSIgnite)
+ - Self-guided workshop (individually, at home)
 
-## How to Buid the Workshop Content in your own Environment
+You can view [a hosted version of the workshop guide](https://aka.ms/aitour/contoso-chat/workshop) on the Azure AI Tour website for quick reference. You can also **preview and extend** the workshop directly from this source:
 
-You will need to install `mkdocs-material` first:
+1. Install the `mkdocs-material` package
+    ```bash
+    pip install mkdocs-material
+    ```
 
-```
-pip install mkdocs-material
-```
+2. Run the `mkdocs serve` command from the `workshop` folder
+    ```bash
+    cd docs/workshop
+    mkdocs serve -a localhost:5000
+    ```
+This should open the dev server with a preview of the workshop guide on the specified local address. Simply open a browser and navigate to `http://localhost:5000` to view the content.
 
-Then, follow the Codespaces instructions above to launch the server. Open a browser on `http://localhost:5000/` to view the content.
 
+**Note:** If you are currently viewing the repo from GitHub Codespaces or a Docker Desktop hosted _dev container_ launched from this repo, then you should already have the `mkdocs-material` package installed - and can go directly to step 2.

docs/workshop/docs/01-Tour-Guide-Setup/02-validate.md

@@ -62,7 +62,7 @@ From the VS Code Online Terminal pane (in Tab 2️⃣):
 
 1. You also need to log into the Azure Developer CLI, `azd`. Enter the command below at the terminal, and follow the same process to copy the code, select the account, and close the tab.
 ```
-azd auth login
+azd auth login --use-device-code
 ```
     - You won't need to enter the password again. Simply select your Skillable Lab account.
 

docs/workshop/docs/03-Workshop-Build/03-infra.md

@@ -25,7 +25,7 @@ The Azure CosmosDB resource holds the customer data for our application. It is a
 1. **Click** the `Azure Cosmos DB account` resource name to visit its details page
 1. **Click** `Data Explorer` in the top-nav menu 
     - dismiss the popup dialog to skip the movie
-    - see: `contoso-outdoors` container with `customers` database
+    - see: `contoso-outdoor` container with `customers` database
     - click `customers`, then select `Items`
     - you should see: **12 data items in database**
 
@@ -70,7 +70,7 @@ You will get a response body with `question`, `answer` and `context` components.
     - The products selected may depend on `customer_id` and the associated order history. 
     - The web app provides `chat_history` from the chat window - which can serve as additional context that the model can use to ground the response.
 
-!!! note "Exercise → Repeat exercise above with a different customer ID number. How did the response change?"
+!!! note "Exercise → Repeat exercise with a different customer ID (between 1 and 12). How did the response change?"
 
 
 ✅ | Your Contoso Chat AI is deployed - and works with valid inputs!

docs/workshop/docs/03-Workshop-Build/04-ideation.md

@@ -39,7 +39,8 @@ _Now it's time to understand how that application was developed - and specifical
 
 ![The extension 'Prompty' wants to sign in using Microsoft.](../img/prompty-auth.png)
 
-- Result: **You will get an Error** in the Output pane. This is because we haven't yet configured a model for Prompty to use.
+- Result: The Visual Studio Code console will switch to the "Output" tab.
+    - **You will get an Error** in the Output pane as shown below. This is expected. It is because we haven't yet configured a model for Prompty to use.
     - ❌ | ` Error: 404 The API deployment for this resource does not exist.`
 
 ## Step 2: Update model configuration and basic info
@@ -53,7 +54,9 @@ For a Prompty file to run, we need to specify a generative AI model to use.
 
 ### 1. Update model configuration
 
-1. Copy the previous prompty to a new one. From the Terminal pane:
+1. Return to the Visual Studio Code terminal pane. 
+1. If you are still seeing the error message from the previous step, then you are in the _Output_ tab. Switch to the _Terminal_ tab to get a command prompt.
+1. Now, use this command to copy the previous prompty to a new one. 
     ```
     cp basic.prompty chat-0.prompty
     ```
@@ -141,7 +144,7 @@ From here, we'll supply data in a JSON file to provide context for the generativ
     cp ../docs/workshop/src/1-build/chat-1.json .
     ```
 
-    !!! note "Open the file to take a look at its contents. It provides a customer's name, age, membership level, and purchase history. It also provides the customer's question to the chatbot: What can you tell me about your tents?."
+    !!! note "Open the file to take a look at its contents. It has the customer's name, age, membership level, and purchase history. It also has the default customer question for our chatbot: _What cold-weather sleeping bag would go well with what I have already purchased?_"
 
 2. Replace the `sample:` section of `chat-1.prompty` (lines 16-18) with the following:
 
@@ -329,7 +332,9 @@ cp chat-1.json chat-2.json
     
 1. Execute `chat-3.py` by clicking the "play" at the top-right of its VS Code window. You should now see a valid response being generated.
 
-<!-->
+    !!! tip "Press Alt-Z (or Cmd-Z on Mac) to toggle word wrap. This will make the prompts in the `.prompty` file easier to read within the limited screen view."
+
+<!--
 ### 3. Troubleshooting
 
 _The [Prompty](https://prompty.ai) tooling is in preview. This section captures any issues and workarounds that can be used to resolve them (till fixed in a new release)._

docs/workshop/docs/03-Workshop-Build/05-evaluation.md

@@ -120,7 +120,8 @@ Navigate to the `src/api` folder in Visual Studio Code.
 
 !!! warning "Troubleshooting: Evaluation gives an error message in the Notebook"
 
-    On occasion, the evaluation notebook may throw an error after a couple of iterations. This is typically a transient error. To fix it, `Clear inputs` in the Jupyter Notebook, then `Restart` it. It should complete the run this time.
+    On occasion, the evaluation notebook may throw an error after a few iterations. This is typically a transient error. To fix it, `Clear outputs` in the Jupyter Notebook, then `Restart` the kernel. `Run All` should complete the run this time.
+
 
 ### 3.2 Watch Evaluation Runs
 
@@ -177,33 +178,35 @@ Once the trace file is displayed, explore the panel to get an intuition for usag
 ### 4.3 Explore: Create Summary
 
 1. When notebook execution completes, look in the `src/api/evaluators` folder:
-    - See: **Chat Responses** in `result.jsonl`
-    - See: **Evaluated Results** in `result_evaluated.jsonl` (scores at end of each line)
-    - See: **Evaluation Summary** computed from `eval_results.jsonl` (complete data.)
+    - You see: **Chat Responses** in `result.jsonl`
+    - You see: **Evaluated Results** in `result_evaluated.jsonl` (scores at end of each line)
+    - You see: **Evaluation Summary** computed from `eval_results.jsonl` (complete data.)
 
-1. Scroll to the bottom of the notebook and view the results cell:
+1. Scroll to the bottom of the notebook to view the results cell:
     - Click the `View as scrollable element` link to redisplay output
     - Scroll to the bottom of redisplayed cell to view scores table
-    - You should see something like this
+    - You should see something like the table below - we reformatted it manually for clarity.
 
 ![Eval](./../img/tabular-eval.png)
 
 ### 4.4 Understand: Eval Results
 
 The figure shows you what that tabulated data looks like in the notebook results. Ignore the formatting for now, and let's look at what this tells us:
 
-1. You see 12 rows of data - correspoding to 12 test inputs
-1. You see 3 columns of metrics - corresponding to evaluators
-1. Each metric records a score between `1` and `5`
+1. You see 12 rows of data - corresponding to 12 test inputs (in `data.jsonl`)
+1. You see 4 metrics from custom evaluators - `groundedness`,`fluency`,`coherence`,`relevance`
+1. Each metric records a score - between `1` and `5`
 
 Let's try to put the scores in context of the responses we see. Try these exercises:
 
-1. Find a row that has a `groundedness` of 5.
-    - View the related row in the evaluation results file
-    - Observe the answer and context provided - _was the answer grounded in the context?_
-1. Find a row that has a `groundedness` of 1.
-    - View the related row in the evaluation results file
-    - Observe the answer and context provided - _was THIS answer grounded in the context?_
+1. Pick a row above that has a `groundedness` of 5.
+    - View the related row in the `result_evaluation.jsonl` file
+    - Observe related answer and context in file - _was the answer grounded in the context?_
+1. Pick a row that has a `groundedness` of 1.
+    - View the related row in the `result_evaluation.jsonl` file
+    - Observe related answer and context in file - _was THIS answer grounded in the context?_
+
+As one example, we can see that the first response in the visualized results (`row 0`) had a groundedness of 5, while the third row from the bottom (`row 9`) had a groundedness of 1. You might find that in the first case the answers provided matched the data context. While in the second case, the answers may quote specific context but did not actually reflect correct usage.
 
 !!! note "Explore the data in more detail on your own. Try to build your intuition for how scores are computed, and how that assessment reflects in the quality of your application."