Initial content commit v1.0

Author: danielfleischer

.gitignore

@@ -0,0 +1,136 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# custom
+/.vscode/
+/outputs/
+/__main__
+
+.DS_Store

.pre-commit-config.yaml

@@ -0,0 +1,18 @@
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.3.0
+    hooks:
+      - id: end-of-file-fixer
+      - id: trailing-whitespace
+      - id: check-yaml
+  - repo: https://github.com/psf/black
+    rev: 22.6.0
+    hooks:
+      - id: black
+        args: ["--line-length", "100"]
+  - repo: https://github.com/pycqa/isort
+    rev: 5.10.1
+    hooks:
+      - id: isort
+        name: isort (python)
+        args: ["--profile", "black", "--filter-files", "--line-length", "100"]

CITATION.cff

@@ -0,0 +1,19 @@
+cff-version: 1.2.0
+message: "If you use this software, please cite it as below."
+authors:
+- family-names: "Izsak"
+  given-names: "Peter"
+  orcid: "https://orcid.org/0000-0001-8354-6823"
+- family-names: "Berchansky"
+  given-names: "Moshe"
+  orcid: "https://orcid.org/0000-0001-9227-8939"
+- family-names: "Fleischer"
+  given-names: "Daniel"
+  orcid: "https://orcid.org/0000-0003-4031-4410"
+- family-names: "Laperdon"
+  given-names: "Ronen"
+title: "fastRAG: Efficient Retrieval Augmentation and Generation Framework"
+version: 1.0
+license: Apache-2.0
+date-released: 2023-02-16
+url: "https://github.com/IntelLabs/fastrag"

CONTRIBUTING.md

@@ -0,0 +1,43 @@
+# Contributing to fastRAG
+
+The following document describes the process of contributing and developing extensions to fastRAG.
+
+## Setting up your development environment
+
+Preliminary requirements:
+
+- fastRAG installed in a developement enviroment (via `pip install -e`)
+- Python 3.8+
+- Pytorch
+- Any 3rd party store engine package
+
+## Installing development packages
+
+To install extra packages required for library development:
+
+```sh
+pip install -e .[dev]
+```
+
+We use [pre-commit](https://pre-commit.com/) to automatically format the code in a consistent way. Installing the dev packages will install `pre-commit`. To manually install it:
+
+```sh
+pip install pre-commit
+pre-commit install        # in project's dir
+```
+
+It uses the `black` and `isort` utilities. No need to download them, they are handled by `pre-commit`. Have a look at [pre-commit-config](./.pre-commit-config.yaml) to see what it does.
+To explicitly run the hooks:
+
+```sh
+pre-commit run --all-files
+```
+
+That's it!
+
+## New component contribution process
+
+1. Fork the repository to your own github space
+2. Add your component as an extension or new addition following Haystack components hierarchy and according to the structure of the library (e.g., a new reader to `fastrag/readers/`).
+3. Provide a pipeline or script to validate the flow of the new component (a `yaml` file or modifications in `scripts/generate_pipeline.py`).
+4. Open a pull request and describe your contribution.

LICENSE

@@ -186,7 +186,7 @@
       same "printed page" as the copyright notice for easier
       identification within third-party archives.
 
-   Copyright [yyyy] [name of copyright owner]
+   Copyright 2023 Intel Corporation
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

MANIFEST.in

@@ -0,0 +1 @@
+global-include *.cpp *.cu