Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when using UUIDField as EAV2_PRIMARY_KEY_FIELD value #478

Open
wants to merge 46 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
413cdf7
add uuid configuration for id generation of eav models
mathiasag7 Sep 6, 2023
8977495
add charfield possibility to eav id models field
mathiasag7 Sep 6, 2023
de7fb79
feat: natural key handler added to models except value model
mathiasag7 Sep 6, 2023
a73e327
feat: natural key handler added to value model
mathiasag7 Sep 6, 2023
6a01f16
Update CHANGELOG.md
mathiasag7 Sep 6, 2023
4be9a24
Update pyproject.toml
mathiasag7 Sep 6, 2023
832d4f3
fix: get attribute for value natural key function
mathiasag7 Sep 7, 2023
ac3c60b
Merge branch 'master' of github.com:mathiasag7/django-eav2-uuid
mathiasag7 Sep 7, 2023
7ebfb85
update package log
mathiasag7 Sep 7, 2023
ce2b606
fix: error on attribute creation
mathiasag7 Sep 7, 2023
dffbe60
Update README.md
mathiasag7 Oct 5, 2023
2c69da6
readme updated
mathiasag7 Oct 5, 2023
e275a7f
readme updated
mathiasag7 Oct 5, 2023
0aa172c
changelog updated
mathiasag7 Oct 6, 2023
33f6279
feat: natural key for serialization and UUIDField possibility for pk …
mathiasag7 Oct 6, 2023
f342f3c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 6, 2023
6e90e9c
test: test_attribute errors corrected on AutoField
mathiasag7 Oct 13, 2023
06e8cd5
Merge branch 'master' of github.com:mathiasag7/django-eav2-uuid
mathiasag7 Oct 13, 2023
13e5e99
Merge branch 'jazzband:master' into master
mathiasag7 Oct 13, 2023
668dcab
test: pytest.ini file added
mathiasag7 Oct 13, 2023
011965f
Merge branch 'master' of github.com:mathiasag7/django-eav2-uuid
mathiasag7 Oct 13, 2023
e64425d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 13, 2023
0d705d9
test: pyproject.toml updated"
mathiasag7 Oct 13, 2023
0000e26
migration file created
mathiasag7 Oct 13, 2023
7a00e04
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 13, 2023
d26f61a
test updated
mathiasag7 Oct 13, 2023
8bf3fee
Merge branch 'master' of github.com:mathiasag7/django-eav2-uuid
mathiasag7 Oct 13, 2023
4ef7f7a
test updated
mathiasag7 Oct 13, 2023
9407982
revert changes
mathiasag7 Oct 16, 2023
1d70d73
convert tests to pytest
Dresdn Oct 24, 2023
f7cd292
remove test cehcking for ValueError
Dresdn Oct 24, 2023
db893d3
feat: test_natural_keys implemented
mathiasag7 Oct 25, 2023
43365cd
Merge branch 'master' of github.com:mathiasag7/django-eav2-uuid
mathiasag7 Oct 25, 2023
b0c2b50
set charfield as default pk to avoid errors
mathiasag7 Dec 4, 2023
5c0b0d6
changelog updated
mathiasag7 Dec 4, 2023
2f1957f
fix: error in migration files
mathiasag7 Dec 20, 2023
ae01a4c
Merge branch 'jazzband:master' into master
mathiasag7 Mar 8, 2024
1d6f15d
Merge branch 'jazzband:master' into master
mathiasag7 Mar 15, 2024
2ff9292
tests + migration file updated
mathiasag7 Mar 22, 2024
3b8d28c
tests + migration file updated
mathiasag7 Mar 22, 2024
4c6ceed
tests + migration file updated
mathiasag7 Mar 22, 2024
08a2699
tests + migration file updated
mathiasag7 Mar 22, 2024
d35289f
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 22, 2024
b2f8124
tests + migration file updated
mathiasag7 Mar 25, 2024
b011b1b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 25, 2024
7fb9e7c
tests + migration file updated
mathiasag7 Mar 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
changelog updated
mathiasag7 committed Oct 6, 2023
commit 0aa172c030b1048e405e1b95d314a7c85f0ccfa0
19 changes: 5 additions & 14 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Version History
# Version History

We follow [Semantic Versions](https://semver.org/) starting at the `0.14.0` release.

@@ -8,24 +9,14 @@ We follow [Semantic Versions](https://semver.org/) starting at the `0.14.0` rele

### Bug Fixes

## 1.5.2 (2023-09-07)

### Bug Fixes

- Fixing error at new attribute creation at datatype field validation

## 1.5.1 (2023-09-07)

### Bug Fixes

- Fixing error on generate foreign key based on attribute natural key at deserialization of value object
- Fixes querying with multiple eav kwargs [#395](https://github.com/jazzband/django-eav2/issues/395)

## 1.5.0 (2023-09-06)
## 1.5.0 (2023-09-07)

### Features

- Support for many type of primary key (UUIDField, BigAutoField, CharField)
- Support for natural key use for some models (EnumValue, EnumGroup, Attribute, Value)
- Support for many type of primary key (UUIDField, BigAutoField)
- Support for natural key use for some models for serialization (EnumValue, EnumGroup, Attribute, Value)

## 1.4.0 (2023-07-07)

87 changes: 85 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -6,7 +6,82 @@

## Django EAV 2 - Entity-Attribute-Value storage for Django

Django EAV 2 is a fork of django-eav. You can find documentation <a href="https://django-eav2.rtfd.io">here</a> for more information.
Django EAV 2 is a fork of django-eav (which itself was derived from eav-django).
You can find documentation <a href="https://django-eav2.rtfd.io">here</a>.

## What is EAV anyway?

> Entity–attribute–value model (EAV) is a data model to encode, in a space-efficient manner, entities where the number of attributes (properties, parameters) that can be used to describe them is potentially vast, but the number that will actually apply to a given entity is relatively modest. Such entities correspond to the mathematical notion of a sparse matrix. (Wikipedia)

Data in EAV is stored as a 3-tuple (typically corresponding to three distinct tables):

- The entity: the item being described, e.g. `Person(name='Mike')`.
- The attribute: often a foreign key into a table of attributes, e.g. `Attribute(slug='height', datatype=FLOAT)`.
- The value of the attribute, with links both an attribute and an entity, e.g. `Value(value_float=15.5, person=mike, attr=height)`.

Entities in **django-eav2** are your typical Django model instances. Attributes (name and type) are stored in their own table, which makes it easy to manipulate the list of available attributes in the system. Values are an intermediate table between attributes and entities, each instance holding a single value.
This implementation also makes it easy to edit attributes in Django Admin and form instances.

You will find detailed description of the EAV here:

- [Wikipedia - Entity–attribute–value model](https://en.wikipedia.org/wiki/Entity%E2%80%93attribute%E2%80%93value_model)

## EAV - The Good, the Bad or the Ugly?

EAV is a trade-off between flexibility and complexity. As such, it should not be thought of as either an amelioration pattern, nor an anti-pattern. It is more of a [gray pattern](https://wiki.c2.com/?GreyPattern) - it exists in some context, to solve certain set of problems. When used appropriately, it can introduce great flexibility, cut prototyping time or deacrease complexity. When used carelessly, however, it can complicate database schema, degrade the performance and make maintainance hard. **As with every tool, it should not be overused.** In the following paragraphs we briefly discuss the pros, the cons and pointers to keep in mind when using EAV.

### When to use EAV?

Originally, EAV was introduced to workaround a problem which cannot be easily solved within relational model. In order to achieve this, EAV bypasses normal schema restrictions. Some refer to this as an example of the [inner-platform effect](https://en.wikipedia.org/wiki/Inner-platform_effect#Examples). Naturally, in such scenarios RDMS resources cannot be used efficiently.

Typical application of the EAV model sets to solve the problem of sparse data with a large number of applicable attributes, but only a small fraction that applies to a given entity that may not be known beforehand. Consider the classic example:

> A problem that data modelers commonly encounter in the biomedical domain is organizing and storing highly diverse and heterogeneous data. For example, a single patient may have thousands of applicable descriptive parameters, all of which need to be easily accessible in an electronic patient record system. These requirements pose significant modeling and implementation challenges. [1]

And:

> [...] what do you do when you have customers that demand real-time, on-demand addition of attributes that they want to store? In one of the systems I manage, our customers wanted to do exactly this. Since we run a SaaS (software as a service) application, we have many customers across several different industries, who in turn want to use our system to store different types of information about _their_ customers. A salon chain might want to record facts such as 'hair color,' 'hair type,' and 'haircut frequency'; while an investment company might want to record facts such as 'portfolio name,' 'last portfolio adjustment date,' and 'current portfolio balance.' [2]

In both of these problems we have to deal with sparse and heterogeneous properties that apply only to potentially different subsets of particular entities. Applying EAV to a sub-schema of the database allows to model the desired behaviour. Traditional solution would involves wide tables with many columns storing NULL values for attributes that don't apply to an entity.

Very common use case for EAV are custom product attributes in E-commerce implementations, such as Magento. [3]

As a rule of thumb, EAV can be used when:

- Model attributes are to be added and removed by end users (or are unknowable in some different way). EAV supports these without ALTER TABLE statements and allows the attributes to be strongly typed and easily searchable.
- There will be many attributes and values are sparse, in contrast to having tables with mostly-null columns.
- The data is highly dynamic/volatile/vulnerable to change. This problem is present in the second example given above. Other example would be rapidly evolving system, such as a prototype with constantly changing requirements.
- We want to store meta-data or supporting information, e.g. to customize system's behavior.
- Numerous classes of data need to be represented, each class has a limited number of attributes, but the number of instances of each class is very small.
- We want to minimise programmer's input when changing the data model.

For more throughout discussion on the appriopriate use-cases see:

1. [Wikipedia - Scenarios that are appropriate for EAV modeling](https://en.wikipedia.org/wiki/Entity%E2%80%93attribute%E2%80%93value_model#Scenarios_that_are_appropriate_for_EAV_modeling)
2. [StackOverflow - Entity Attribute Value Database vs. strict Relational Model E-commerce](https://stackoverflow.com/questions/870808/entity-attribute-value-database-vs-strict-relational-model-ecommerce)
3. [WikiWikiWeb - Generic Data Model](https://wiki.c2.com/?GenericDataModel)

## When to avoid it?

As we outlined in the opening section, EAV is a trade-off. It should not be used when:

##### 1. System is performance critical

> Attribute-centric query is inherently more difficult when data are stored in EAV form than when they are stored conventionally. [4]

In general, the more structured your data model, the more efficiently you can deal with it. Therefore, loose data storage such as EAV has obvious trade-off in performance. Specifically, application of the EAV model makes performing JOINs on tables more complicated.

##### 2. Low complexity/low maintenance cost is of priority

EAV complicates data model by splitting information across tables. This increases conceptual complexity as well as SQL statements required to query the data. In consequence, optimization in one area that also makes the system harder to understand and maintain.

However, it is important to note that:

> An EAV design should be employed only for that sub-schema of a database where sparse attributes need to be modeled: even here, they need to be supported by third normal form metadata tables. There are relatively few database-design problems where sparse attributes are encountered: this is why the circumstances where EAV design is applicable are relatively rare. [1]

## Alternatives

In some use-cases, JSONB (binary JSON data) datatype (Postgres 9.4+ and analogous in other RDMSs) can be used as an alternative to EAV. JSONB supports indexing, which amortizes performance trade-off. It's important to keep in mind that JSONB is not RDMS-agnostic solution and has it's own problems, such as typing.

## What is new here ?

@@ -38,6 +113,7 @@ Add `django.db.models.UUIDField` or `django.db.models.BigAutoField` as value of
``` python
PRIMARY_KEY_FIELD = "django.db.models.UUIDField" # as exemple
```

### Note: Primary key mandatory modification field

If the primary key of eav models are to be modified (UUIDField -> BigAutoField, BigAutoField -> UUIDField) in the middle of the project when the migrations are already done, you have to change the value of `PRIMARY_KEY_FIELD` in your settings.
@@ -119,6 +195,13 @@ Supplier.objects.filter(eav__city='London')
# = <EavQuerySet [<Supplier: Supplier object (1)>]>
```

**For futher information? Check out the django eav2 <a href="https://django-eav2.readthedocs.io/en/latest/#documentation">documentation</a>.**
**What next? Check out the <a href="https://django-eav2.readthedocs.io/en/latest/#documentation">documentation</a>.**

---

### References

[1] Exploring Performance Issues for a Clinical Database Organized Using an Entity-Attribute-Value Representation, https://doi.org/10.1136/jamia.2000.0070475 <br>
[2] What is so bad about EAV, anyway?, https://sqlblog.org/2009/11/19/what-is-so-bad-about-eav-anyway <br>
[3] Magento for Developers: Part 7—Advanced ORM: Entity Attribute Value, https://devdocs.magento.com/guides/m1x/magefordev/mage-for-dev-7.html <br>
[4] Data Extraction and Ad Hoc Query of an Entity— Attribute— Value Database, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC61332/