Skip to content

Added retries to requests wrapper #56

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Aug 31, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 65 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Contributing

## Installation

After cloning this repo, create a [virtualenv](https://virtualenv.pypa.io/en/stable/) and ensure dependencies are installed by running:

```sh
Expand Down Expand Up @@ -30,4 +32,66 @@ If you wish to run against a specific version defined in the `tox.ini` file:
tox -e py36
```

Tox can only use whatever versions of Python are installed on your system. When you create a pull request, Travis will also be running the same tests and report the results, so there is no need for potential contributors to try to install every single version of Python on their own system ahead of time. We appreciate opening issues and pull requests to make PyMS even more stable & useful!
Tox can only use whatever versions of Python are installed on your system. When you create a pull request, Travis will also be running the same tests and report the results, so there is no need for potential contributors to try to install every single version of Python on their own system ahead of time.

## Pipenv

### Advantages over plain pip and requirements.txt
[Pipenv](https://pipenv.readthedocs.io/en/latest/) generates two files: a `Pipfile`and a `Pipfile.lock`.
* `Pipfile`: Is a high level declaration of the dependencies of your project. It can contain "dev" dependencies (usually test related stuff) and "standard" dependencies which are the ones you'll need for your project to function
* `Pipfile.lock`: Is the "list" of all the dependencies your Pipfile has installed, along with their version and their hashes. This prevents two things: Conflicts between dependencies and installing a malicious module.

### How to...

Here the most 'common' `pipenv` commands, for a more in-depth explanation please refer to the [official documentation](https://pipenv.readthedocs.io/en/latest/).

#### Install pipenv
```bash
pip install pipenv
```

#### Install dependencies defined in a Pipfile
```bash
pipenv install
```

#### Install both dev and "standard" dependencies defined in a Pipfile
```bash
pipenv install --dev
```

#### Install a new module
```bash
pipenv install django
```

#### Install a new dev module (usually test related stuff)
```bash
pipenv install nose --dev
```

#### Install dependencies in production
```bash
pipenv install --deploy
```

#### Start a shell
```bash
pipenv shell
```

## Documentation

This project use MkDocs

* `mkdocs new [dir-name]` - Create a new project.
* `mkdocs serve` - Start the live-reloading docs server.
* `mkdocs build` - Build the documentation site.
* `mkdocs help` - Print this help message.

### Project layout

mkdocs.yml # The configuration file.
docs/
index.md # The documentation homepage.
... # Other markdown pages, images and other files.
101 changes: 32 additions & 69 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,30 @@
PyMS, Python MicroService, is a collections of libraries, best practices and recommended ways to build
microservices with Python.

## Documentation

To know how use, install or build a project see the docs: https://py-ms.readthedocs.io/en/latest/

## Motivation

When we started to create microservice with no idea, we were looking for tutorials, guides, best practices, but we found
nothing to create professional projects. Most articles say:
- "Install flask"
- "Create routes"
- (Sometimes) "Create a swagger specs"
- "TA-DA! you have a microservice"

But... what happens with our configuration out of code like Kubernetes configmap? what happens with transactionality?
If we have many microservices, what happens with traces?.

There are many problems around Python and microservices and we can`t find anyone to give a solution.

We start creating these projects to try to solve all the problems we have found in our professional lives about
microservices architecture.

Nowadays, is not perfect and we have a looong roadmap, but we hope this library could help other felas and friends ;)


## Installation
```bash
pip install py-ms
Expand All @@ -23,80 +45,21 @@ pip install py-ms
Module to read yaml or json configuration from a dictionary or a path.

### pyms/flask/app
With the funcion `create_app` initialize the Flask app, register [blueprints](http://flask.pocoo.org/docs/0.12/blueprints/)
and intialize all libraries like Swagger, database, trace system, custom logger format, etc.
With the function `create_app` initialize the Flask app, register [blueprints](http://flask.pocoo.org/docs/0.12/blueprints/)
and initialize all libraries such as Swagger, database, trace system, custom logger format, etc.

### pyms/flask/services
Integrations and wrappers over common libs like request, swagger, connexion

### pyms/flask/healthcheck
This views is usually used by Kubernetes, Eureka and other systems to check if our application is up and running.
This view is usually used by Kubernetes, Eureka and other systems to check if our application is running.

### pyms/logger
Print logger in JSON format to send to server like Elasticsearch. Inject span traces in logger.

### pyms/rest_template
Encapsulate common rest operations between business services propagating trace headers if configured.

### pyms/tracer
Create an injector `flask_opentracing.FlaskTracer` to use in our projects

## Pipenv

### Advantages over plain pip and requirements.txt
[Pipenv](https://pipenv.readthedocs.io/en/latest/) generates two files: a `Pipfile`and a `Pipfile.lock`.
* `Pipfile`: Is a high level declaration of the dependencies of your project. It can contain "dev" dependencies (usually test related stuff) and "standard" dependencies which are the ones you'll need for your project to function
* `Pipfile.lock`: Is the "list" of all the dependencies your Pipfile has installed, along with their version and their hashes. This prevents two things: Conflicts between dependencies and installing a malicious module.

### How to...

Here the most 'common' `pipenv` commands, for a more in-depth explanation please refer to the [official documentation](https://pipenv.readthedocs.io/en/latest/).

#### Install pipenv
```bash
pip install pipenv
```

#### Install dependencies defined in a Pipfile
```bash
pipenv install
```

#### Install both dev and "standard" dependencies defined in a Pipfile
```bash
pipenv install --dev
```

#### Install a new module
```bash
pipenv install django
```

#### Install a new dev module (usually test related stuff)
```bash
pipenv install nose --dev
```

#### Install dependencies in production
```bash
pipenv install --deploy
```

#### Start a shell
```bash
pipenv shell
```

## Documentation

This project use MkDocs

* `mkdocs new [dir-name]` - Create a new project.
* `mkdocs serve` - Start the live-reloading docs server.
* `mkdocs build` - Build the documentation site.
* `mkdocs help` - Print this help message.

### Project layout

mkdocs.yml # The configuration file.
docs/
index.md # The documentation homepage.
... # Other markdown pages, images and other files.
Create an injector `flask_opentracing.FlaskTracer` to use in our projects.

## How To Contrib
We appreciate opening issues and pull requests to make PyMS even more stable & useful! See [This doc](COONTRIBUTING.md)
for more details
20 changes: 20 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,26 @@
PyMS, Python MicroService, is a collection of libraries, best practices and recommended ways to build
microservices with Python.

## Motivation

When we started to create microservice with no idea, we were looking for tutorials, guides, best practices, but we found
nothing to create professional projects. Most articles say:
- "Install flask"
- "Create routes"
- (Sometimes) "Create a swagger specs"
- "TA-DA! you have a microservice"

But... what happens with our configuration out of code like Kubernetes configmap? what happens with transactionality?
If we have many microservices, what happens with traces?.

There are many problems around Python and microservices and we can`t find anyone to give a solution.

We start creating these projects to try to solve all the problems we have found in our professional lives about
microservices architecture.

Nowadays, is not perfect and we have a looong roadmap, but we hope this library could help other felas and friends ;)


## Index:
* [PyMS structure](structure.md)
* [Configuration](configuration.md)
Expand Down
3 changes: 3 additions & 0 deletions docs/structure.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ Module to read yaml or json configuration from a dictionary or a path.
With the function `create_app` initialize the Flask app, register [blueprints](http://flask.pocoo.org/docs/0.12/blueprints/)
and initialize all libraries such as Swagger, database, trace system, custom logger format, etc.

### pyms/flask/services
Integrations and wrappers over common libs like request, swagger, connexion

### pyms/flask/healthcheck
This view is usually used by Kubernetes, Eureka and other systems to check if our application is running.

Expand Down
2 changes: 1 addition & 1 deletion pylintrc
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ confidence=
# --enable=similarities". If you want to run only the classes checker, but have
# no Warning level messages displayed, use"--disable=all --enable=classes
# --disable=W"
disable=C0301
disable=C0301,C0111,C0103,logging-format-interpolation

# Enable the message, report, category or checker with the given id(s). You can
# either give multiple identifier separated by comma (,) or put this option
Expand Down
2 changes: 1 addition & 1 deletion pyms/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@

__email__ = "[email protected]"

__version__ = "1.1.0"
__version__ = "1.2.0"
17 changes: 10 additions & 7 deletions pyms/flask/app/create_app.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
from pyms.config.conf import get_conf
from pyms.constants import LOGGER_NAME, SERVICE_ENVIRONMENT
from pyms.flask.healthcheck import healthcheck_blueprint
from pyms.flask.services.driver import ServicesManager
from pyms.flask.services.driver import ServicesManager, DriverService
from pyms.logger import CustomJsonFormatter
from pyms.tracer.main import init_lightstep_tracer

Expand All @@ -35,6 +35,8 @@ def __call__(cls, *args, **kwargs):
class Microservice(metaclass=SingletonMeta):
service = None
application = None
swagger = DriverService
requests = DriverService

def __init__(self, *args, **kwargs):
self.service = kwargs.get("service", os.environ.get(SERVICE_ENVIRONMENT, "ms"))
Expand Down Expand Up @@ -70,14 +72,15 @@ def init_logger(self):
def init_app(self) -> Flask:
if getattr(self, "swagger", False):
app = connexion.App(__name__, specification_dir=os.path.join(self.path, self.swagger.path))
app.add_api(self.swagger.file,
arguments={'title': self.config.APP_NAME},
base_path=self.config.APPLICATION_ROOT
)
app.add_api(
self.swagger.file,
arguments={'title': self.config.APP_NAME},
base_path=self.config.APPLICATION_ROOT
)

# Invert the objects, instead connexion with a Flask object, a Flask object with
application = app.app
application._connexion_app = app
application.connexion_app = app
else:
application = Flask(__name__, static_folder=os.path.join(self.path, 'static'),
template_folder=os.path.join(self.path, 'templates'))
Expand Down Expand Up @@ -121,4 +124,4 @@ def add_error_handler(self, code_or_exception, handler):
:param code_or_exception: HTTP error code or exception
:param handler: callback for error handler
"""
self.application._connexion_app.add_error_handler(code_or_exception, handler)
self.application.connexion_app.add_error_handler(code_or_exception, handler)
2 changes: 1 addition & 1 deletion pyms/flask/app/create_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@

def config():
ms = Microservice()
return ms.config
return ms.config
10 changes: 2 additions & 8 deletions pyms/flask/healthcheck/__init__.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,3 @@
"""Init file
"""
from __future__ import unicode_literals, print_function, absolute_import, division
from pyms.flask.healthcheck.healthcheck import healthcheck_blueprint

from flask import Blueprint

healthcheck_blueprint = Blueprint('healthcheck', __name__, static_url_path='/static')

from pyms.flask.healthcheck import healthcheck
__all__ = ['healthcheck_blueprint']
8 changes: 5 additions & 3 deletions pyms/flask/healthcheck/healthcheck.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
"""Healthcheck
"""
from pyms.flask.healthcheck import healthcheck_blueprint
from __future__ import unicode_literals, print_function, absolute_import, division

from flask import Blueprint

healthcheck_blueprint = Blueprint('healthcheck', __name__, static_url_path='/static')


@healthcheck_blueprint.route('/healthcheck', methods=['GET'])
Expand Down
Loading