From f978dc8083b19e755d1cbf5e1a4662bd70fb6b86 Mon Sep 17 00:00:00 2001 From: shimizukawa Date: Sat, 23 Jun 2018 08:28:57 +0900 Subject: [PATCH] [WIP] re-construct documentation --- .gitignore | 1 + AUTHORS.rst | 7 +++ CHANGES.rst | 111 ++++++++++++++++++++++++++++++++++++++++++++++++ LICENSE | 13 ++++++ doc/autohrs.rst | 2 + doc/changes.rst | 2 + doc/dev.rst | 12 ++++++ doc/index.rst | 13 +++++- doc/usage.rst | 93 ++++++++++++++++++++++++++++++++++++++++ 9 files changed, 253 insertions(+), 1 deletion(-) create mode 100644 AUTHORS.rst create mode 100644 CHANGES.rst create mode 100644 LICENSE create mode 100644 doc/autohrs.rst create mode 100644 doc/changes.rst create mode 100644 doc/dev.rst create mode 100644 doc/usage.rst diff --git a/.gitignore b/.gitignore index 01a8d5c..ea2bcab 100644 --- a/.gitignore +++ b/.gitignore @@ -6,3 +6,4 @@ dist/ .cache .tox .pytest_cache/ +doc/_build/ diff --git a/AUTHORS.rst b/AUTHORS.rst new file mode 100644 index 0000000..29bba13 --- /dev/null +++ b/AUTHORS.rst @@ -0,0 +1,7 @@ +======= +AUTHORS +======= + +* Takayuki Shimizukawa + + diff --git a/CHANGES.rst b/CHANGES.rst new file mode 100644 index 0000000..57ffbb3 --- /dev/null +++ b/CHANGES.rst @@ -0,0 +1,111 @@ +CHANGES +======= + +0.9 (Unreleased) +---------------- + +* Drop support for Django 1.8, 1.9 and 1.10 + + +0.8.1 (2018-06-19) +------------------ + +* #38: Fix 0.8 doesn't compatible with Python 2. Thanks to Benjy Weinberger. + +0.8 (2018-06-01) +---------------- + +Incompatible Changes: + +* #23,#10 Redshift support time zones in time stamps for migration + + **IMPORTANT**: + With this change, the newly created DateTimeField column will be timestamp + with timezone (TIMESTAMPTZ) by migration. Therefore, the existing + DateTimeField and the new DateTimeField will have different data types as a + redshift schema column type. + There are no migration feature by django-redshift-backend. + see also: https://github.com/shimizukawa/django-redshift-backend/pull/23 + +New Features: + +* #20,#26: Support for sortkey. Thanks to Maxime Vdb and Kosei Kitahara. +* #24: Add UUIDField support. Thanks to Sindri Guðmundsson. +* #14: More compat with redshift: not use SELECT DISTINCT ON. + +Bug Fixes: + +* #15,#21: More compat with redshift: not use CHECK. Thanks to Vasil Vangelovski. +* #18: Fix error on migration with django-1.9 or later that raises AttributeError + of 'sql_create_table_unique'. +* #27: annotate() does not work on Django-1.9 and later. Thanks to Takayuki Hirai. + + +Documentation: + +* Add documentation: http://django-redshift-backend.rtfd.io/ + + +0.7 (2017-06-08) +---------------- + +* Drop Python-3.4 +* Drop Django-1.7 +* Support Python-3.6 +* Support Django-1.11 + +0.6 (2016-12-15) +---------------- + +* Fix crush problem when using bulk insert. + +0.5 (2016-10-05) +---------------- + +* Support Django-1.10 +* #9: Add support for BigAutoField. Thanks to Maxime Vdb. +* Fix crush problem on sqlmigrate when field modified. + +0.4 (2016-05-17) +---------------- + +* Support Python-3.4 and 3.5 +* #7: Restore support django-1.7. Version 0.3 doesn't support django-1.7. +* #4: More compat with redshift: not use SET CONSTRAINTS. Thanks to Maxime Vdb. +* #6: More compat with redshift: not use sequence reset query. Thanks to Maxime Vdb. +* #5: Add REDSHIFT_VARCHAR_LENGTH_MULTIPLIER settings. Thanks to Maxime Vdb. +* Support column type changing on migration. + +0.3 (2016-05-14) +---------------- + +* #3: more compat with Redshift (AutoField, DateTimeField, Index). Thanks to Maxime Vdb. +* More compat with redshift: add TextField +* More compat with redshift: not use DEFERRABLE, CONSTRAINT, DROP DEFAULT +* More compat with redshift: support modify column + + +0.2.1 (2016-02-01) +------------------ + +* "SET TIME_ZONE" warning is changed as debug log for 'django.db.backend' logger. + +0.2 (2016-01-08) +---------------- + +* Disable "SET TIME_ZONE" SQL execution even if settings.TIME_ZONE is specified. + +0.1.2 (2015-06-5) +----------------- + +* Support Django-1.8 + +0.1.1 (2015-03-27) +------------------ +* Disable "SELECT FOR UPDATE" SQL execution. + +0.1 (2015-03-24) +---------------- +* Support Django-1.7 +* Support "INSERT INTO" SQL execution without "RETURNING" clause. + diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..ef2c302 --- /dev/null +++ b/LICENSE @@ -0,0 +1,13 @@ +Copyright 2018 Django Redshift Backend team + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. diff --git a/doc/autohrs.rst b/doc/autohrs.rst new file mode 100644 index 0000000..1befc19 --- /dev/null +++ b/doc/autohrs.rst @@ -0,0 +1,2 @@ +.. include:: ../AUTHORS.rst + diff --git a/doc/changes.rst b/doc/changes.rst new file mode 100644 index 0000000..fe7473a --- /dev/null +++ b/doc/changes.rst @@ -0,0 +1,2 @@ +.. include:: ../CHANGES.rst + diff --git a/doc/dev.rst b/doc/dev.rst new file mode 100644 index 0000000..e54a2b5 --- /dev/null +++ b/doc/dev.rst @@ -0,0 +1,12 @@ +=========== +Development +=========== + +TESTING +======= + +Testing this package requires: + +* tox-1.8 or later +* virtualenv-15.0.1 or later +* pip-8.1.1 or later diff --git a/doc/index.rst b/doc/index.rst index cf8c871..e59eaf0 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -1,2 +1,13 @@ -.. include:: ../README.rst +======================= +Django Redshift Backend +======================= + +This is a Redshift database backend for Django. + +.. toctree:: + + usage + dev + changes + contributors diff --git a/doc/usage.rst b/doc/usage.rst new file mode 100644 index 0000000..f85c923 --- /dev/null +++ b/doc/usage.rst @@ -0,0 +1,93 @@ +====== +Usage +====== + +Support versions +================ + +This product is tested with: + +* python-2.7, 3.5, 3.6 +* django-1.11 + + +Differences from postgres_psycopg2 backend +========================================== + +Type mapping: + +* 'integer identity(1, 1)' for AutoField +* 'bigint identity(1, 1)' for BigAutoField +* 'timestamp with time zone' for DateTimeField +* 'varchar(max)' for TextField +* 'varchar(32)' for UUIDField +* Possibility to multiply VARCHAR length to support utf-8 string, using + `REDSHIFT_VARCHAR_LENGTH_MULTIPLIER` setting. + +Stop using: + +* RETURNING (single insert and bulk insert) +* SELECT FOR UPDATE +* SELECT DISTINCT ON +* SET CONSTRAINTS +* INDEX +* DEFERRABLE INITIALLY DEFERRED +* CONSTRAINT +* CHECK +* DROP DEFAULT + +To support migration: + +* To add column to existent table on Redshift, column must be nullable +* To support modify column, add new column -> data migration -> drop old column -> rename + +Please note that the migration support for redshift is not perfect yet. + + +Note and Limitation +-------------------- + +Amazon Redshift doesn't support RETURNING, so ``last_insert_id`` method retrieve MAX(pk) after insertion as a workaround. + +refs: + +* http://stackoverflow.com/q/19428860 +* http://stackoverflow.com/q/25638539 + +In some case, MAX(pk) workaround does not work correctly. +Bulk insertion makes non-contiguous IDs like: 1, 4, 7, 10, ... +and single insertion after such bulk insertion generates strange id value like 2 (smallest non-used id). + + +SETTINGS +======== + +ENGINE for DATABASES is 'django_redshift_backend'. You can set the name in your settings.py as:: + + DATABASES = { + 'default': { + 'ENGINE': 'django_redshift_backend', + 'NAME': '', + 'USER': '', + 'PASSWORD': '', + 'HOST': '', + 'PORT': '5439', + } + } + +REDSHIFT_VARCHAR_LENGTH_MULTIPLIER: + Possibility to multiply VARCHAR length to support utf-8 string. Default is 1. + +Using sortkey +--------------------------------- + +There is built-in support for this option for Django >= 1.9. To use `sortkey`, simply define an `ordering` on the model meta as follow:: + + class MyModel(models.Model): + ... + + class Meta: + ordering = ['col2'] + +N.B.: there is no validation of this option, instead we let Redshift validate it for you. Be sure to refer to the `documentation `_. +