Skip to content

Commit

Permalink
Rename unstructured pipeline in llm-app (#6547)
Browse files Browse the repository at this point in the history
GitOrigin-RevId: 5537fcbe81ee0883af45d631f5e70082dabfc824
  • Loading branch information
szymondudycz authored and Manul from Pathway committed May 28, 2024
1 parent 234dffc commit 0543cbe
Show file tree
Hide file tree
Showing 5 changed files with 4 additions and 4 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ Pick one that is closest to your needs.
| [`contextless`](examples/pipelines/contextless/app.py) | This simple example calls OpenAI ChatGPT API but does not use an index when processing queries. It relies solely on the given user query. We recommend it to start your Pathway LLM journey. |
| [`contextful`](examples/pipelines/contextful/app.py) | This default example of the app will index the jsonlines documents located in the [`data/pathway-docs`](examples/data/pathway-docs) directory. These indexed documents are then taken into account when processing queries. |
| [`contextful-s3`](examples/pipelines/contextful_s3/app.py) | This example operates similarly to the contextful mode. The main difference is that the documents are stored and indexed from an S3 bucket, allowing the handling of a larger volume of documents. This can be more suitable for production environments. |
| [`unstructured`](examples/pipelines/unstructured/app.py) | Process unstructured documents such as PDF, HTML, DOCX, PPTX, and more. Visit [unstructured-io](https://unstructured-io.github.io/unstructured/) for the full list of supported formats. |
| [`contextful-parsing`](examples/pipelines/contextful_parsing/app.py) | Process unstructured documents such as PDF, HTML, DOCX, PPTX, and more. Visit [unstructured-io](https://unstructured-io.github.io/unstructured/) for the full list of supported formats. |
| [`local`](examples/pipelines/local/app.py) | This example runs the application using Huggingface Transformers, which eliminates the need for the data to leave the machine. It provides a convenient way to use state-of-the-art NLP models locally. |
| [`unstructured-to-sql`](examples/pipelines/unstructured_to_sql_on_the_fly/app.py) | This example extracts the data from unstructured files and stores it into a PostgreSQL table. It also transforms the user query into an SQL query which is then executed on the PostgreSQL table. |
| [`alert`](examples/pipelines/alert/app.py) | Ask questions, get alerted whenever response changes. Pathway is always listening for changes, whenever new relevant information is added to the stream (local files in this example), LLM decides if there is a substantial difference in response and notifies the user with a Slack message. |
Expand Down Expand Up @@ -168,7 +168,7 @@ If you are unfamiliar with how to contribute to GitHub projects, here is a [Get

## ☁️ Hosted Version ☁️

Please see <a href="https://cloud.pathway.com/">cloud.pathway.com</a> for hosted services. You can quickly set up variants of the `unstructured` app, which connect live data sources on Google Drive and Sharepoint to your Gen AI app.
Please see <a href="https://cloud.pathway.com/">cloud.pathway.com</a> for hosted services. You can quickly set up variants of the `demo-document-indexing` app, which connect live data sources on Google Drive and Sharepoint to your Gen AI app.

## Need help?

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Unstructured Pipeline
# Contextful Parsing Pipeline

This example implements a RAG pipeline, similarly to [contextful pipeline](). It uses, however, [Unstructured](https://unstructured.io/) library for parsing documents, which are then split into smaller chunks.
This example implements a RAG pipeline, similarly to [contextful pipeline](). It uses, however, [Unstructured](https://unstructured.io/) library for parsing documents, e.g. PDFs, which are then split into smaller chunks.

## How to run the project

Expand Down
File renamed without changes.
File renamed without changes.

0 comments on commit 0543cbe

Please sign in to comment.