Generating pydantic models tree and exclude_parent_fields

0.10.10

✨ Features

• Add get_pydantic function that allows you to auto generate equivalent pydantic models tree from ormar.Model. This newly generated model tree can be used in requests and responses to exclude fields you do not want to include in the data.
• Add exclude_parent_fields parameter to model Meta that allows you to exclude fields from parent models during inheritance. Note that best practice is to combine models and mixins but if you have many similar models and just one that differs it might be useful tool to achieve that.

🐛 Fixes

• Fix is null filter with pagination and relations (by @erichaydel) #214
• Fix not saving child object on reverse side of the relation if not saved before #216

💬 Other

• Expand fastapi part of the documentation to show samples of using ormar in requests and responses in fastapi.
• Improve the docs in regard of default, ForeignKey.add etc.

Bump pydantic pin to fix security vulnerability

0.10.9

Important security fix

• Update pin for pydantic to fix security vulnerability CVE-2021-29510

You are advised to update to version of pydantic that was patched.
In 0.10.9 ormar excludes versions with vulnerability in pinned dependencies.

🐛 Fixes

• Fix OpenAPi schema for LargeBinary #204

Bug fixes

0.10.8

🐛 Fixes

• Fix populating default values in pk_only child models #202
• Fix mypy for LargeBinary fields with base64 str representation #199
• Fix OpenAPI schema format for LargeBinary fields with base64 str representation #199
• Fix OpenAPI choices encoding for LargeBinary fields with base64 str representation

New params in dict and base64 encoded strings in LargeBinary

0.10.7

✨ Features

• Add exclude_primary_keys: bool = False flag to dict() method that allows to exclude all primary key columns in the resulting dictionaru. #164
• Add exclude_through_models: bool = False flag to dict() that allows excluding all through models from ManyToMany relations #164
• Add represent_as_base64_str: bool = False parameter that allows conversion of bytes LargeBinary field to base64 encoded string. String is returned in dict(),
  on access to attribute and string is converted to bytes on setting. Data in database is stored as bytes. #187
• Add pk alias to allow field access by Model.pk in filters and order by clauses (python style)

🐛 Fixes

• Remove default None option for max_length for LargeBinary field #186
• Remove default None option for max_length for String field

💬 Other

• Provide a guide and samples of dict() parameters in the docs
• Major refactor of getting/setting attributes from magic methods into descriptors -> noticeable performance improvement

Add large binary, support for native pydantic fields, examples in openapi

0.10.6

✨ Features

• Add LargeBinary(max_length) field type #166

• Add support for normal pydantic fields (including Models) instead of pydantic_only
  attribute which is now deprecated #160.
  Pydantic fields should be declared normally as in pydantic model next to ormar fields,
  note that (obviously) ormar does not save and load the value for this field in
  database that mean that ONE of the following has to be true:

  • pydantic field declared on ormar model has to be Optional (defaults to None)
  • pydantic field has to have a default value set
  • pydantic field has default_factory function set
  • ormar.Model with pydantic field has to overwrite __init__() and provide the value there

  If none of the above ormar (or rather pydantic) will fail during loading data from the database,
  with missing required value for declared pydantic field.

• Ormar provides now a meaningful examples in openapi schema, including nested models.
  The same algorithm is used to iterate related models without looks
  as with dict() and select/load_all. Examples appear also in fastapi. #157

🐛 Fixes

• By default pydantic is not validating fields during assignment,
  which is not a desirable setting for an ORM, now all ormar.Models
  have validation turned-on during assignment (like model.column = 'value')

💬 Other

• Add connecting to the database in QuickStart in readme #180
• OpenAPI schema does no longer include ormar.Model docstring as description,
  instead just model name is provided if you do not provide your own docstring.
• Some performance improvements.

Bug fixes

0.10.5

🐛 Fixes

• Fix bug in fastapi-pagination #73
• Remove unnecessary Optional in List[Optional[T]] in return value for QuerySet.all() and Querysetproxy.all() return values #174
• Run tests coverage publish only on internal prs instead of all in github action.

Python style filter and order_by with field chain access

0.10.4

✨ Features

• Add Python style to filter and order_by with field access instead of dunder separated strings. #51
  • Accessing a field with attribute access (chain of dot notation) can be used to construct FilterGroups (ormar.and_ and ormar.or_)
  • Field access overloads set of python operators and provide a set of functions to allow same functionality as with dunder separated param names in **kwargs, that means that querying from sample model Track related to model Album now you have more options:
    • exact - exact match to value, sql column = <VALUE>
      • OLD: album__name__exact='Malibu'
      • NEW: can be also written as Track.album.name == 'Malibu
    • iexact - exact match sql column = <VALUE> (case insensitive)
      • OLD: album__name__iexact='malibu'
      • NEW: can be also written as Track.album.name.iexact('malibu')
    • contains - sql column LIKE '%<VALUE>%'
      • OLD: album__name__contains='Mal'
      • NEW: can be also written as Track.album.name % 'Mal')
      • NEW: can be also written as Track.album.name.contains('Mal')
    • icontains - sql column LIKE '%<VALUE>%' (case insensitive)
      • OLD: album__name__icontains='mal'
      • NEW: can be also written as Track.album.name.icontains('mal')
    • in - sql column IN (<VALUE1>, <VALUE2>, ...)
      • OLD: album__name__in=['Malibu', 'Barclay']
      • NEW: can be also written as Track.album.name << ['Malibu', 'Barclay']
      • NEW: can be also written as Track.album.name.in_(['Malibu', 'Barclay'])
    • isnull - sql column IS NULL (and sql column IS NOT NULL)
      • OLD: album__name__isnull=True (isnotnull album__name__isnull=False)
      • NEW: can be also written as Track.album.name >> None
      • NEW: can be also written as Track.album.name.is_null(True)
      • NEW: not null can be also written as Track.album.name.is_null(False)
      • NEW: not null can be also written as ~(Track.album.name >> None)
      • NEW: not null can be also written as ~(Track.album.name.is_null(True))
    • gt - sql column > <VALUE> (greater than)
      • OLD: position__gt=3
      • NEW: can be also written as Track.album.name > 3
    • gte - sql column >= <VALUE> (greater or equal than)
      • OLD: position__gte=3
      • NEW: can be also written as Track.album.name >= 3
    • lt - sql column < <VALUE> (lower than)
      • OLD: position__lt=3
      • NEW: can be also written as Track.album.name < 3
    • lte - sql column <= <VALUE> (lower equal than)
      • OLD: position__lte=3
      • NEW: can be also written as Track.album.name <= 3
    • startswith - sql column LIKE '<VALUE>%' (exact start match)
      • OLD: album__name__startswith='Mal'
      • NEW: can be also written as Track.album.name.startswith('Mal')
    • istartswith - sql column LIKE '<VALUE>%' (case insensitive)
      • OLD: album__name__istartswith='mal'
      • NEW: can be also written as Track.album.name.istartswith('mal')
    • endswith - sql column LIKE '%<VALUE>' (exact end match)
      • OLD: album__name__endswith='ibu'
      • NEW: can be also written as Track.album.name.endswith('ibu')
    • iendswith - sql column LIKE '%<VALUE>' (case insensitive)
      • OLD: album__name__iendswith='IBU'
      • NEW: can be also written as Track.album.name.iendswith('IBU')
• You can provide FilterGroups not only in filter() and exclude() but also in:
  • get()
  • get_or_none()
  • get_or_create()
  • first()
  • all()
  • delete()
• With FilterGroups (ormar.and_ and ormar.or_) you can now use:
  • & - as and_ instead of next level of nesting
  • | - as `or_' instead of next level of nesting
  • ~ - as negation of the filter group
• To combine groups of filters into one set of conditions use & (sql AND) and | (sql OR)

  # Following queries are equivalent:
  # sql: ( product.name = 'Test'  AND  product.rating >= 3.0 ) 
  
  # ormar OPTION 1 - OLD one
  Product.objects.filter(name='Test', rating__gte=3.0).get()
  
  # ormar OPTION 2 - OLD one
  Product.objects.filter(ormar.and_(name='Test', rating__gte=3.0)).get()
  
  # ormar OPTION 3 - NEW one (field access)
  Product.objects.filter((Product.name == 'Test') & (Product.rating >=3.0)).get()

• Same applies to nested complicated filters

  # Following queries are equivalent:
  # sql: ( product.name = 'Test' AND product.rating >= 3.0 ) 
  #       OR (categories.name IN ('Toys', 'Books'))
  
  # ormar OPTION 1 - OLD one
  Product.objects.filter(ormar.or_(
                            ormar.and_(name='Test', rating__gte=3.0), 
                            categories__name__in=['Toys', 'Books'])
                        ).get()
  
  # ormar OPTION 2 - NEW one (instead of nested or use `|`)
  Product.objects.filter(
                        ormar.and_(name='Test', rating__gte=3.0) | 
                        ormar.and_(categories__name__in=['Toys', 'Books'])
                        ).get()
  
  # ormar OPTION 3 - NEW one (field access)
  Product.objects.filter(
                        ((Product.name='Test') & (Product.rating >= 3.0)) | 
                        (Product.categories.name << ['Toys', 'Books'])
                        ).get()

• Now you can also use field access to provide OrderActions to order_by()
  • Order ascending:
    • OLD: Product.objects.order_by("name").all()
    • NEW: Product.objects.order_by(Product.name.asc()).all()
  • Order descending:
    • OLD: Product.objects.order_by("-name").all()
    • NEW: Product.objects.order_by(Product.name.desc()).all()
  • You can of course also combine different models and many order_bys:
    Product.objects.order_by([Product.category.name.asc(), Product.name.desc()]).all()

🐛 Fixes

• Not really a bug but rather inconsistency. Providing a filter with nested model i.e. album__category__name = 'AA'
  is checking if album and category models are included in select_related() and if not it's auto-adding them there.
  The same functionality was not working for FilterGroups (and_ and or_), now it works (also for python style filters which return FilterGroups).

One sided relations and more powerful save_related

0.10.3

✨ Features

• ForeignKey and ManyToMany now support skip_reverse: bool = False flag #118.
  If you set skip_reverse flag internally the field is still registered on the other
  side of the relationship so you can:

  • filter by related models fields from reverse model
  • order_by by related models fields from reverse model

  But you cannot:

  • access the related field from reverse model with related_name
  • even if you select_related from reverse side of the model the returned models won't be populated in reversed instance (the join is not prevented so you still can filter and order_by)
  • the relation won't be populated in dict() and json()
  • you cannot pass the nested related objects when populating from dict() or json() (also through fastapi). It will be either ignored or raise error depending on extra setting in pydantic Config.

• Model.save_related() now can save whole data tree in once #148
  meaning:

  • it knows if it should save main Model or related Model first to preserve the relation

  • it saves main Model if

    • it's not saved,
    • has no pk value
    • or save_all=True flag is set

    in those cases you don't have to split save into two calls (save() and save_related())

  • it supports also ManyToMany relations

  • it supports also optional Through model values for m2m relations

• Add possibility to customize Through model relation field names.

• By default Through model relation names default to related model name in lowercase.
  So in example like this:

  ... # course declaration ommited
  class Student(ormar.Model):
      class Meta:
          database = database
          metadata = metadata
  
      id: int = ormar.Integer(primary_key=True)
      name: str = ormar.String(max_length=100)
      courses = ormar.ManyToMany(Course)
  
  # will produce default Through model like follows (example simplified)
  class StudentCourse(ormar.Model):
      class Meta:
          database = database
          metadata = metadata
          tablename = "students_courses"
  
      id: int = ormar.Integer(primary_key=True)
      student = ormar.ForeignKey(Student) # default name
      course = ormar.ForeignKey(Course)  # default name

• To customize the names of fields/relation in Through model now you can use new parameters to ManyToMany:

  • through_relation_name - name of the field leading to the model in which ManyToMany is declared
  • through_reverse_relation_name - name of the field leading to the model to which ManyToMany leads to

  Example:

  ... # course declaration ommited
  class Student(ormar.Model):
      class Meta:
          database = database
          metadata = metadata
  
      id: int = ormar.Integer(primary_key=True)
      name: str = ormar.String(max_length=100)
      courses = ormar.ManyToMany(Course,
                                 through_relation_name="student_id",
                                 through_reverse_relation_name="course_id")
  
  # will produce default Through model like follows (example simplified)
  class StudentCourse(ormar.Model):
      class Meta:
          database = database
          metadata = metadata
          tablename = "students_courses"
  
      id: int = ormar.Integer(primary_key=True)
      student_id = ormar.ForeignKey(Student) # set by through_relation_name
      course_id = ormar.ForeignKey(Course)  # set by through_reverse_relation_name

Optimization & bug fixes, additional parameters in update methods

0.10.2

✨ Features

• Model.save_related(follow=False) now accept also two additional arguments: Model.save_related(follow=False, save_all=False, exclude=None).
  • save_all:bool -> By default (so with save_all=False) ormar only upserts models that are not saved (so new or updated ones),
    with save_all=True all related models are saved, regardless of saved status, which might be useful if updated
    models comes from api call, so are not changed in the backend.
  • exclude: Union[Set, Dict, None] -> set/dict of relations to exclude from save, those relation won't be saved even with follow=True and save_all=True.
    To exclude nested relations pass a nested dictionary like: exclude={"child":{"sub_child": {"exclude_sub_child_realtion"}}}. The allowed values follow
    the fields/exclude_fields (from QuerySet) methods schema so when in doubt you can refer to docs in queries -> selecting subset of fields -> fields.
• Model.update() method now accepts _columns: List[str] = None parameter, that accepts list of column names to update. If passed only those columns will be updated in database.
  Note that update() does not refresh the instance of the Model, so if you change more columns than you pass in _columns list your Model instance will have different values than the database!
• Model.dict() method previously included only directly related models or nested models if they were not nullable and not virtual,
  now all related models not previously visited without loops are included in dict(). This should be not breaking
  as just more data will be dumped to dict, but it should not be missing.
• QuerySet.delete(each=False, **kwargs) previously required that you either pass a filter (by **kwargs or as a separate filter() call) or set each=True now also accepts
  exclude() calls that generates NOT filter. So either each=True needs to be set to delete whole table or at least one of filter/exclude clauses.
• Same thing applies to QuerySet.update(each=False, **kwargs) which also previously required that you either pass a filter (by **kwargs or as a separate filter() call) or set each=True now also accepts
  exclude() calls that generates NOT filter. So either each=True needs to be set to update whole table or at least one of filter/exclude clauses.
• Same thing applies to QuerysetProxy.update(each=False, **kwargs) which also previously required that you either pass a filter (by **kwargs or as a separate filter() call) or set each=True now also accepts
  exclude() calls that generates NOT filter. So either each=True needs to be set to update whole table or at least one of filter/exclude clauses.

🐛 Fixes

• Fix improper relation field resolution in QuerysetProxy if fk column has different database alias.
• Fix hitting recursion error with very complicated models structure with loops when calling dict().
• Fix bug when two non-relation fields were merged (appended) in query result when they were not relation fields (i.e. JSON)
• Fix bug when during translation to dict from list the same relation name is used in chain but leads to different models
• Fix bug when bulk_create would try to save also property_field decorated methods and pydantic fields
• Fix wrong merging of deeply nested chain of reversed relations

💬 Other

• Performance optimizations
• Split tests into packages based on tested area

Add get_or_none, fix quoting sql keyword names in order_by queries

0.10.1

Features

• add get_or_none(**kwargs) method to QuerySet and QuerysetProxy. It is exact equivalent of get(**kwargs) but instead of raising ormar.NoMatch exception if there is no db record matching the criteria, get_or_none simply returns None.

Fixes

• Fix dialect dependent quoting of column and table names in order_by clauses not working
  properly in postgres.