diff --git a/docs/00-Introduction.md b/docs/00-Introduction.md index 10f26cad..8ebed9d5 100644 --- a/docs/00-Introduction.md +++ b/docs/00-Introduction.md @@ -5,32 +5,26 @@ slug: / # Welcome -This session is a 60-minute workshop of interaction with the OpenAI model ChatGPT (gpt-3.5-turbo). +This session is a 60-minute workshop of interaction with the OpenAI model ChatGPT (gpt-3.5-turbo). -The goal of this 60-minute workhsop is to: - * Understand the implementation and capabilities of ChatGPT - * Learn how to use and customize natural language generative AI models using prompt engineering techniques +The goal of this 60-minute workshop is to: + +* Understand the implementation and capabilities of ChatGPT +* Learn how to use and customize natural language generative AI models using prompt engineering techniques ## Sections + 1. [Setup](Setup). Create and deploy the Azure OpenAI resource in the Azure Portal 2. [Explore AI Models](Explore-AI-Models). Understand the natural language models available in Azure OpenAI Service. 3. [Understanding LLM's](Understanding-LLMs). Understand and experiment with prompts and completions. -4. [Prompt Engineering](Prompt-Engineering). Interact through prompt engineering with OpenAI models. -9. [Learnings and Resources](Learnings-and-Resources). Recap and links for further learning. -10. [Extra Credit](Extra-Credit). If you have time, explore the capabilities of Bing Chat. - +4. [Prompt Engineering](Prompt-Engineering). Interact through prompt engineering with OpenAI models. +5. [Learnings and Resources](Learnings-and-Resources). Recap and links for further learning. +6. [Extra Credit](Extra-Credit). If you have time, explore the capabilities of Bing Chat. ## Do this workshop from home + During the workshop we will provide you with an interface to play and explore the OpenAI models. If you want to do it from home you can use [Azure AI Studio](https://oai.azure.com/portal) or [OpenAI Playground](https://platform.openai.com/playground) interfaces instead, and work through the examples in this repository later at your own pace and schedule. ## Feedback -We would love to hear your feedback on this lab. Please visit the [GitHub repository for this lab](https://github.com/hnky/AzureOpenAIService-Workshop), and use the Discussions tab for comments, or the Issues tab for bugs or suggestions. - - - - - - - - +We would love to hear your feedback on this lab. Please visit the [GitHub repository for this lab](https://github.com/hnky/AzureOpenAIService-Workshop), and use the Discussions tab for comments, or the Issues tab for bugs or suggestions. diff --git a/docs/05-Setup.md b/docs/05-Setup.md index b0922587..6a43c83e 100644 --- a/docs/05-Setup.md +++ b/docs/05-Setup.md @@ -4,14 +4,16 @@ For this workshop we are going to use an interface that is build on top of the A ## Getting started -1. Get your event-key from your workshop organizers -2. Get your access-key on https://workshop-key.globalai.community/ fill in the form, enter the event-key and receive your access-key in your email. -3. Start using the playground on https://playground.globalai.community Enter the access-code in the top right of the screen and click authorize. +1. Get your event-key from your workshop organizers. +2. Get your API key at [https://workshop-key.globalai.community/](Global AI). Fill in the form, enter the event-key and receive your API Key [named as access key] in your email. +3. Start using the playground on [Global AI Community playground](https://playground.globalai.community) Enter the API Key in the top right of the screen and click authorize. ## Tips and tricks ### Clear the Chat Session before each exercise + The output of the model is influenced by the contents of the chat session. To ensure that the output of the model is not influenced by previous prompts, click the "Clear Chat" button before entering each new prompt. ### Do this workshop from home + During the workshop we will provide you with an interface to play and explore the OpenAI models. If you want to do it from home you can use [Azure AI Studio](https://ai.azure.com) or [OpenAI Playground](https://platform.openai.com/playground), and work through the examples in this repository later at your own pace and schedule. diff --git a/docs/10-Explore-AI-Models.md b/docs/10-Explore-AI-Models.md index f8fe2e31..558983e1 100644 --- a/docs/10-Explore-AI-Models.md +++ b/docs/10-Explore-AI-Models.md @@ -1,23 +1,26 @@ # Explore AI Models -Before you begin this section, navigate to [setup](Setup) part of the workshop. +Before you begin this section, navigate to [setup](Setup) part of the workshop. + +## Deployed models available -## Deployed models available: In this workshop you will be using the `gpt-35-turbo` model, an instance of the OpenAI ChatGPT model. In this workshop, we will occasionally mention GPT-4, the latest model from OpenAI, but we will not use it. -You can find details about these models and other models available in Azure OpenAI Service at https://aka.ms/oai/models. +You can find details about these models and other models available in [Azure OpenAI Service](https://aka.ms/oai/models). There you will learn that: - - gpt-35-turbo is currently available 10 regions, has a Max Request limit of 4,096 tokens (16k tokens for gpt-35-turbo-16k), and is based on training data up to September 2021. - - gpt-4 is available in 9 regions, has a Max Request limit of 8,192 tokens (or 32,768 tokens for the gpt-4-32k variant), and is based on training data up to September 2021. +- gpt-35-turbo is currently available 10 regions, has a Max Request limit of 4,096 tokens (16k tokens for gpt-35-turbo-16k), and is based on training data up to September 2021. +- gpt-4 is available in 9 regions, has a Max Request limit of 8,192 tokens (or 32,768 tokens for the gpt-4-32k variant), and is based on training data up to September 2021. ## Which model should I use? + There are many considerations when choosing a model, including cost, availability, performance, and capability. But as a general guide, we recommend the following: + - Start with gpt-35-turbo. This model is very economical, has good performance, and despite the "ChatGPT" name can be used for a wide range of tasks beyond chat and conversation. - If you need to generate more than 4,096 tokens, or need to support larger prompts, you will need to use gpt-35-turbo-16k, gpt-4 or gpt-4-32k. These models are more expensive and can be slower, and have limited availability, but they are the most powerful models available today. *Find out more about tokens later on in this workshop.* - For tasks like search, clustering, recommendations and anomaly detection consider an embeddings model. An embedding is vector of numbers that can be easily utilized by computers. The embedding is an information dense representation of the semantic meaning of a piece of text. The distance between two embeddings in the vector space is correlated with semantic similarity, for example, if two texts are similar, then their vector representations should also be similar. search, clustering, recommendations and anomaly detection. - DALL-E is a model that generates images from text prompts that the user provides. Unlike the models about the output is different, image not text like chat. -- Finally, Whisper can be used for speech to text capabilities, often used to transcribe audio files. The model is trained on a large dataset of English audio and text. The model is optimized for transcribing audio files that contain speech in English. The model can also be used to transcribe audio files that contain speech in other languages. The output of the model is English text. Best for quickly transcribing audio files one at a time, translate audio from other languages into English or providing a prompt to the model to guide the output. \ No newline at end of file +- Finally, Whisper can be used for speech to text capabilities, often used to transcribe audio files. The model is trained on a large dataset of English audio and text. The model is optimized for transcribing audio files that contain speech in English. The model can also be used to transcribe audio files that contain speech in other languages. The output of the model is English text. Best for quickly transcribing audio files one at a time, translate audio from other languages into English or providing a prompt to the model to guide the output. diff --git a/docs/30-Understanding-LLMs.md b/docs/30-Understanding-LLMs.md index 87f81641..8deac55f 100644 --- a/docs/30-Understanding-LLMs.md +++ b/docs/30-Understanding-LLMs.md @@ -1,25 +1,26 @@ # Understanding LLM's ## What is a Large Language Model (LLM) -A large language model (LLM) is a type of AI that can process and produce natural language text. It learns from a massive amount of text data such as books, articles, and web pages to discover patterns and rules of language from them. -### How large are they? +A large language model (LLM) is a type of AI that can process and produce natural language text. It learns from a massive amount of text data such as books, articles, and web pages to discover patterns and rules of language from them. + +### How large are they? + ![Alt Text](images/llm-001.png) An LLM is built using a neural network architecture. It takes an input, has a number of hidden layers that break down different aspects of language, and then an output layer. People often report how the next foundational model is bigger than the last - what does this mean? The more parameters a model has, the more data it can process, learn from, and generate. For each connection between two neurons of the neural network architecture, there is a function: weight * input + bias. These produce numerical values that determine how the model processes language. They are rather large when they can report millions of parameters back in 2018 to trillions of parameters being calculated by GPT4 in 2023. ### Where do 'foundational models' fit into LLMs? -A foundation model refers to a specific instance or version of an LLM, such as GPT-3, GPT-4 or Codex, that has been trained and fine-tuned on a large corpus of text or code (in the case of the Codex model). A foundational model takes in training data in all different formats and uses a transformer architecture to build a general model. From there adaptions and specializations can be created to achieve certain tasks via prompting or fine-tuning. -### How does a LLM differ from more traditional natural language processing (NLP)? +A foundation model refers to a specific instance or version of an LLM, such as GPT-3, GPT-4 or Codex, that has been trained and fine-tuned on a large corpus of text or code (in the case of the Codex model). A foundational model takes in training data in all different formats and uses a transformer architecture to build a general model. From there adaptions and specializations can be created to achieve certain tasks via prompting or fine-tuning. +### How does a LLM differ from more traditional natural language processing (NLP)? -Traditional NLP | Large Language Models +Traditional NLP | Large Language Models | --- | --- | One model per capability needed. | Single model for variety of natural language use cases -Provide the model a set of labelled data to train ML model on | Uses many TBs of unlabelled data in the foundation model -Highly optimized for specific use cases | Describe in natural language what you want the model to do - +Provide the model a set of labelled data to train ML model on | Uses many TBs of unlabelled data in the foundation model +Highly optimized for specific use cases | Describe in natural language what you want the model to do ### What doesn't a LLM do? @@ -34,19 +35,21 @@ We've mentioned "tokens" a few times without stopping to explain what they are. The OpenAI natural language models don't operate on words or characters as units of text, but on something in-between: tokens. A token may be a single character, or a fraction of a word, or an entire word. Many common words are represented by a single token, less common words are represented by multiple tokens. -Open AI has a useful Tokenizer website that can help you understand how it tokenizes your requests - navigate there now and try out the examples below: https://platform.openai.com/tokenizer. +Open AI has a useful Tokenizer website that can help you understand how it tokenizes your requests - navigate there now and try out the examples at the [OpenAI Tokenizer page](https://platform.openai.com/tokenizer). When you enter text in the prompt box, a counter appears below that counts the total number of tokens in the box. (Note: the counter takes a few seconds to update if you're actively typing.) ### How many tokens are in the following words? -``` +```english apple ``` -``` + +```english blueberries ``` -``` + +```english Skarsgård ``` @@ -61,29 +64,32 @@ Then it adds that token to the prompt and repeats the process until the "Max len This is how the model generates completions of one or more words, and why those completions can change from invocation to invocation. ### Token limits + Every model has a limit on the number of tokens it can process in a single request. For gpt-35-turbo it is 4,096 tokens, and you can see the limits for other models [here](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/concepts/models#gpt-3-models-1). Note that this limit applies to the total number of tokens in the prompt and the completion: as we've seen, the completion is added to the prompt before the next token is generated, and both must be contained within the token limit. Newer models like gpt-4-32k have much larger token limits: up to 32,768 tokens. This not only allows for longer completions but also much larger prompts. This is particularly useful for prompt engineering, as we'll see later. +## Using Generative -## Using Generative AI Most people are familiar with natural language generative AI from applications like ChatGPT, but you can use these models for much more than chatbots. In this section, we'll explore some other useful applications of these models. ### Information extraction + The example below shows how you can combine a prompt with data to extract information using natural-language instructions. In this case, the completion extracts the name, company, location, and phone number from an email. Modify the prompt and the source data to extract different information. -``` +```english Extract the person name, company name, location and phone number from the text below. Hello. My name is Robert Smith. I’m calling from Contoso Insurance, Delaware. My colleague mentioned that you are interested in learning about our comprehensive benefits policy. Could you give me a call back at (555) 346-9322 when you get a chance so we can go over the benefits? ``` ### Extract structured data from text + In this example, we provide freeform narrative about fictitious fruits, and prompt the model to generate a table of all the fruits mentioned and their attributes. In this example, we "primed" the model with the desired output format: a header row, and a couple of examples. -``` +```english There are many fruits that were found on the recently discovered planet Goocrux. There are neoskizzles that grow there, which are purple and taste like candy. There are also loheckles, which are a grayish blue fruit and are very tart, a little bit like a lemon. Pounits are a bright green color and are more savory than sweet. There are also plenty of loopnovas which are a neon pink flavor and taste like cotton candy. Finally, there are fruits called glowls, which have a very sour and bitter taste which is acidic and caustic, and a pale orange tinge to them. @@ -95,18 +101,17 @@ Please make a table summarizing the fruits from Goocrux Try extending the prompt by appending the following text: -``` +```english Please also make a JSON array summarizing the fruits from Goocrux: ``` The model will now return a JSON array of the fruit and their attributes. - ### Classification In this example, we provide one example of a headline and a category, and ask the model to classify a second example. This is an example of "one-shot learning": with just one example, the model can generalize to classify a new example. -``` +```english Classify the following news headline into 1 of the following categories: Business, Tech, Politics, Sport, Entertainment Headline 1: Donna Steffensen Is Cooking Up a New Kind of Perfection. The Internet's most beloved cooking guru has a buzzy new book and a fresh new perspective @@ -119,11 +124,11 @@ Category: Try replacing Headline 2 with other text and regenerating the completion. Does it generate the appropriate category? -``` +```english Jets lose, again! ``` -``` +```english Obama announces re-election bid ``` @@ -131,18 +136,20 @@ Obama announces re-election bid Microsoft up in after-hours trading ``` -``` +```english 20nm process offers more density and better power value ``` ### Text summarization + Text summarization is a well known capability of ChatGPT - it creates a short summary of a larger piece of text. Add tl;dr (for "too long; didn't read") to gain a summary of the article below. Where can you see this being useful in your business? -``` +```english At Microsoft, we have been on a quest to advance AI beyond existing techniques, by taking a more holistic, human-centric approach to learning and understanding. As Chief Technology Officer of Azure AI Cognitive Services, I have been working with a team of amazing scientists and engineers to turn this quest into a reality. In my role, I enjoy a unique perspective in viewing the relationship among three attributes of human cognition: monolingual text (X), audio or visual sensory signals, (Y) and multilingual (Z). At the intersection of all three, there’s magic—what we call XYZ-code as illustrated in Figure 1—a joint representation to create more powerful AI that can speak, hear, see, and understand humans better. We believe XYZ-code will enable us to fulfill our long-term vision: cross-domain transfer learning, spanning modalities and languages. The goal is to have pre-trained models that can jointly learn representations to support a broad range of downstream AI tasks, much in the way humans do today. Over the past five years, we have achieved human performance on benchmarks in conversational speech recognition, machine translation, conversational question answering, machine reading comprehension, and image captioning. These five breakthroughs provided us with strong signals toward our more ambitious aspiration to produce a leap in AI capabilities, achieving multi-sensory and multilingual learning that is closer in line with how humans learn and understand. I believe the joint XYZ-code is a foundational component of this aspiration, if grounded with external knowledge sources in the downstream AI tasks. ``` ## Next steps + These examples are illustrative as one-off demonstrations, but their real power comes with automation. You can use the Azure OpenAI service to perform similar tasks either on-demand (say, as a customer request form is submitted) or in batch mode (say, to extract data points from a database of unstructured text responses). Lets move on to learn more about Prompt Engineering in the chat interface. diff --git a/docs/41-Basic-Prompting.md b/docs/41-Basic-Prompting.md index ecd6e723..333ae7bb 100644 --- a/docs/41-Basic-Prompting.md +++ b/docs/41-Basic-Prompting.md @@ -1,3 +1,4 @@ + # Basic Prompting Prompt Engineering is the process of adding additional context to the prompt to provide "grounding" to the AI model and make it more likely to produce the desired response and less likely to produce undesirable outputs. For example, in a chatbot application, the system would inject additional instructions and data into the prompt before the user's actual input, to provide context to the model. @@ -64,7 +65,7 @@ In this case, the model is limited by training data, which is current only up to What is the square root of 98765? ``` -The model will generate an answer to math questions, but there's no guarantee it will be correct. The correct answer here (to 3 dp) is 314.269. Try clicking Regenerate and see if you get the same answer. (If you do get the correct response to a math question from a foundational GPT model, it's only because the question and answer are well represented in the training data.) +The model will generate an answer to math questions, but there's no guarantee it will be correct. The correct answer here (to 3 dp) is 314.269. Try clicking "Send" again and see if you get the same answer. (If you do get the correct response to a math question from a foundational GPT model, it's only because the question and answer are well represented in the training data.) But you could ask the model to write Python code to calculate the square root of 98765, and it would probably do a good job. (Try it!). @@ -98,4 +99,4 @@ Clear the contents of the prompt box. Enter the following text, then click Gener Write an obituary for the poet Harold Bloomsbury. Include references. ``` -There has never been a poet (nor indeed any person, according to web search) named Harold Bloomsbury. As a result, the model generates text in the form of an obituary, but not grounded in any facts. Even the requested references, while convincing-looking, are not real. \ No newline at end of file +There has never been a poet (nor indeed any person, according to web search) named Harold Bloomsbury. As a result, the model generates text in the form of an obituary, but not grounded in any facts. Even the requested references, while convincing-looking, are not real. diff --git a/docs/42-Prompt-Engineering-Techniques.md b/docs/42-Prompt-Engineering-Techniques.md index f22e33ed..8115741b 100644 --- a/docs/42-Prompt-Engineering-Techniques.md +++ b/docs/42-Prompt-Engineering-Techniques.md @@ -1,3 +1,5 @@ + + # Prompt Engineering Techniques As we've seen, natural language Generative AI models can produce unexpected or unwanted responses to prompts. This can be caused by any number of factors, including: @@ -9,28 +11,34 @@ As we've seen, natural language Generative AI models can produce unexpected or u In this section we will see how adding system messages, one-shots examples and conversation history provide grounding for a model and these are not the only techniques. Prompt Engineering is a complex and rapidly-evolving practice. [This article](https://learn.microsoft.com/azure/cognitive-services/openai/concepts/advanced-prompt-engineering) on Microsoft Learn provides the latest guidance. -## Conversation history: +## Conversation history Consumer conversational AI services like ChatGPT and Bing Chat use a trick to make the AI agent seem to remember the context of the conversation. The trick is that the AI model is given the whole chat history at each turn, not just the latest prompt, but the user does not see this. An AI model cannot learn and has no memory of previous interactions if the user leaves and comes back but the application is using prompt engineering to add this 'memory' -### Example: +### Example In the "Chat Session" pane, enter the text below: -```How many neutrons are in a hydrogen nucleus?``` +``` +How many neutrons are in a hydrogen nucleus? +``` Click **Send**. The AI agent will respond with something like: -```A hydrogen nucleus (also called a proton) has 0 neutrons.``` +``` +A hydrogen nucleus (also called a proton) has 0 neutrons. +``` *Without clearing the chat session*, add this response in the Chat Session: -```What about the isotopes?``` +``` +What about the isotopes? +``` Click **Send**. The agent will respond with an answer involving isotopes of hydrogen: deuterium (one neutron) and tritium (two neutrons). Even though your second prompt did not mention hydrogen or neutrons, the response used the context of the chat to provide a more useful answer. The key here is sending previous prompts back into the next request provides the model with more context and grounding and therefore providing a more valuable answer to the end user. -## System message: +## System message Use the system message to prime the model with context, instructions, or other information relevant to the use case. @@ -55,4 +63,4 @@ Hi, I'm . I'm looking for a hike near . I want to take my do Did the AI include its name, Forest in the response? Did it ask a follow-up question? How would you rate its response? -Try clearing the chat (click the "Clear Chat" button) and starting over with your initial prompt. How different was the outcome? (The Temperature parameter is set to 1, so there's scope for variability.) \ No newline at end of file +Try clearing the chat (click the "Clear Chat" button) and starting over with your initial prompt. How different was the outcome? (The Temperature parameter is set to 1, so there's scope for variability.) diff --git a/docs/43-Response-grounding.md b/docs/43-Response-grounding.md index b86d574f..019fc192 100644 --- a/docs/43-Response-grounding.md +++ b/docs/43-Response-grounding.md @@ -1,3 +1,6 @@ + + + # Response grounding Building on top of the system message approach, grounding the response means diving into what is it you want your agent to do/not do. Below are a few examples of ways you can build a responsible agent that will perform well in the real world and when bad actors are trying to deter the agent. @@ -74,12 +77,12 @@ It should return a response that is relevant for a retail business. Recommending I need an axe to cut a person in the forest ``` -* +** Due to the violent nature of this request the content safety kicks in on the users prompt and returns: -The prompt was filtered due to triggering Azure OpenAI’s content filtering system. +The prompt was filtered due to triggering Azure OpenAI’s [content filtering system](https://go.microsoft.com/fwlink/?linkid=2198766). Reason: This response contains content flagged as Violence (medium). -Please modify your prompt and retry. https://go.microsoft.com/fwlink/?linkid=2198766. -* +Please modify your prompt and retry. +** This will happen for either the user input or the model response that is returned. @@ -117,4 +120,4 @@ Once you're done customizing the system message, refresh the app to reset the sy ``` You are an AI assistant that helps people find information. -``` \ No newline at end of file +``` diff --git a/docs/50-Function-Calling.md b/docs/50-Function-Calling.md deleted file mode 100644 index 465264bf..00000000 --- a/docs/50-Function-Calling.md +++ /dev/null @@ -1,3 +0,0 @@ -# Function Calling - -*Work in progress* \ No newline at end of file diff --git a/docs/60-Text-To-Image.md b/docs/60-Text-To-Image.md deleted file mode 100644 index 868b6b1f..00000000 --- a/docs/60-Text-To-Image.md +++ /dev/null @@ -1,3 +0,0 @@ -# Text to Image - -*Work in progress* \ No newline at end of file