Welcome! Content Understanding is a solution that analyzes and comprehends various media content, such as documents, images, audio, and video, transforming it into structured, organized, and searchable data.
- The samples in this repository default to the latest preview API version: (2024-12-01-preview).
Azure AI Content Understanding is a new Generative AI-based Azure AI service, designed to process/ingest content of any type (documents, images, audio, and video) into a user-defined output format. Content Understanding offers a streamlined process to reason over large amounts of unstructured data, accelerating time-to-value by generating an output that can be integrated into automation and analytical workflows.
| File | Description |
|---|---|
| content_extraction.ipynb | In this sample we will show content understanding API can help you get semantic information from your file. For example OCR with table in document, audio transcription, and face analysis in video. |
| field_extraction.ipynb | In this sample we will show how to create an analyzer to extract fields in your file. For example invoice amount in the document, how many people in an image, names mentioned in an audio, or summary of a video. You can customize the fields by creating your own analyzer template. |
| analyzer_training.ipynb | If you want to futher boost the performance for field extraction, we can do training when you provide few labeled samples to the API. Note: This feature is available to document scenario now. |
| management.ipynb | This sample will demo how to create a minimal analyzer, list all the analyzers in your resource, and delete the analyzer you don't need. |
You can run this repo virtually by using GitHub Codespaces, which will open a web-based VS Code in your browser.
-
Make sure the following tools are installed:
-
Make a new directory called
azure-ai-content-understanding-pythonand clone this template into it using theazdCLI:azd init -t azure-ai-content-understanding-python
You can also use git to clone the repository if you prefer.
- Make sure you have permission to grant roles under subscription
- Login Azure
azd auth login
- Setting up environment, following prompts to choose location
azd up
- Create Azure AI Services resource
- Go to
Access Control (IAM)in resource, grant yourself roleCognitive Services User - Copy
notebooks/.env.sampletonotebooks/.env - Fill AZURE_AI_ENDPOINT with the endpoint from your Azure portal Azure AI Services instance.
Navigate to the notebooks directory and select the sample notebook you are interested in. Since Codespaces is pre-configured with the necessary environment, you can directly execute each step in the notebook.
Azure Search with Content Understanding
Azure Content Understanding with OpenAI
-
Trademarks - This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos is subject to those third-party’s policies.
-
Data Collection - The software may collect information about you and your use of the software and send it to Microsoft. Microsoft may use this information to provide services and improve our products and services. You may turn off the telemetry as described in the repository. There are also some features in the software that may enable you and Microsoft to collect data from users of your applications. If you use these features, you must comply with applicable law, including providing appropriate notices to users of your applications together with a copy of Microsoft’s privacy statement. Our privacy statement is located at https://go.microsoft.com/fwlink/?LinkID=824704. You can learn more about data collection and use in the help documentation and our privacy statement. Your use of the software operates as your consent to these practices.