Skip to content

Feature/visualisation #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Feature/visualisation #9

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1c7b9da
Put product type at the end (CSV/PNG) of url
Oct 9, 2015
2e9db45
Add PNG chart functionality to view
Oct 9, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 21 additions & 2 deletions weather_server/show_weather/views.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@
from django.template import Context
from django.template.loader import get_template

import matplotlib.pyplot as plt

from .models import Station, Observation


Expand Down Expand Up @@ -56,7 +58,6 @@ def csv_stations(request):
response.seek(0)
return HttpResponse(response)


def csv_observation_request(request, station, start, end):
"""Return CSV of observations in requested time range."""
response = StringIO()
Expand All @@ -75,4 +76,22 @@ def csv_observation_request(request, station, start, end):
)
)
response.seek(0)
return HttpResponse(response)
return HttpResponse(response, content_type="text/html")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't actually have to specify the content_type here. From the Django docs:

content_type is the MIME type optionally completed by a character set encoding and is used to fill the HTTP Content-Type header. If not specified, it is formed by the DEFAULT_CONTENT_TYPE and DEFAULT_CHARSET settings, by default: “text/html; charset=utf-8”.

So we don't necessarily have to specify this. If we do, though - not a bad idea since some browsers don't format the response as plaintext - it should actually be text/plain. Even without specification, though, downloading works just fine.


def png_observation_request(request, station, start, end):
"""Return PNG of observations in requested time range."""
obs = Observation.objects.filter(station__station_id=station,
obs_date__gte=start,
obs_date__lte=end)

#TODO create n sub plots for all relevant parameter or merge them in 1 plot
# TODO how to loop over all elements and extract name along the way?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be easiest to just consume the CSV output by csv_observation_request. If you read the response into a StringIO buffer you could pass it to pandas using read_csv and you'd have the data as a DataFrame without having to manipulate the observations directly. There'd be no network bottleneck because there'd be no HTTP communication - we'd just use the responses.

The content would be accessible like this:

csv = csv_observation_request(request, station, start, end).content
# Continue processing

obsdata = [o.temperature for o in obs]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here a generator expression would be more appropriate - it just saves the instructions for pulling out the data and yields it dynamically, so you don't end up passing around large data structures in memory. If I remember correctly, plt.plot will evaluate a generator expression as an iterable, consuming it to produce the data it needs.

Pulling the data out of the DB as a list isn't a bad thing, but it does requrie a lot of memory manipulation that we'll notice as soon as we're working with a lot of data.


fig = plt.figure()
ax1 = fig.add_subplot(1,1,1)
plt.plot(obsdata)

response = HttpResponse(content_type="image/png")
plt.savefig(response, format="png")
return response
15 changes: 8 additions & 7 deletions weather_server/weather_server/urls.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,16 +19,17 @@
"""
from django.conf.urls import include, url
from django.contrib import admin
from show_weather.views import index, csv_observation_request, csv_stations
from show_weather.views import index, csv_observation_request, csv_stations, png_observation_request

data_patterns = [
url(r'^csv/'
r'([0-9]+)/' # Station ID
r'([0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2})' # Start date
url(r'^(?P<station>[0-9]+)/' # Station ID
r'(?P<start>[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2})' # Start date
r' - '
r'([0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2})$', # End date
csv_observation_request),
url(r'^csv/stations/$', csv_stations)
r'(?P<end>[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2})/', include([ # End date
url(r'csv$', csv_observation_request),
url(r'png$', png_observation_request),])),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to build this so that png and csv are at the beginning, not at the end. Not sure how to best implement that - off the top of my head I'm thinking something like putting those up front and then passing them to a date_parser or something, somehow passing it the harvested request type so that it knows what function to call. I prefer that because that would be more consistent with line 31, where the pattern begins with the request type. Before the merge we'll need to find a DRY way of expressing that, or modify the API for requestion a station list.

I'm thinking something like this:

data_patterns = [
    url(r'^(?P<content_type>[a-z]+)/'  # Content type
        r'^(?P<station>[0-9]+)/'  # Station ID
        r'(?P<start>[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2})'    # Start date
        r' - '
        r'(?P<end>[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2})/', data_request),
        url(r'csv$', csv_observation_request)

And then in views.py something like:

DATA_TYPES_MAP = {"csv": csv_observation_request,
                  "png": png_observation_request}
def data_request(request, content_type, station, start, end):
    """Route a data request."""
    return DATA_TYPES_MAP[content_type](request, station, start, end)

Just a very rough prototype.

url(r'^csv/stations/$', csv_stations),

]

urlpatterns = [
Expand Down