Morphology model #282

dkeller9 · 2025-07-14T05:49:27Z

This branch includes a minimal set of commits to illustrate the data model, and is branched off of a more recent main. The primary files affected are: model.py, types.py, and morphology.py (although this is intended just to give context to the data model). "ReconstructionCellMorphology" is changed to "CellMorphology" because now the reconstruction morphologies are a subtype of cell morphology. Because the different sub-clases of morphology were not made programmatic subclasses of cell morphology for the sake of maintaining compatibility with how this structure is processed, the extra data needed for the subclasses is in the "extended_data" field. A text description of all the different morphology components is in: https://openbraininstitute.sharepoint.com/:w:/r/sites/OBI-Scientificstaff/_layouts/15/Doc.aspx?sourcedoc=%7BEEE86C08-A09D-469E-9849-CD85599E4026%7D&file=General_schema.docx&action=default&mobileredirect=true&DefaultItemOpen=1

app/db/model.py

eleftherioszisis · 2025-07-14T06:44:54Z

app/db/model.py

+    pipeline_state: Mapped[PipelineType | None] = mapped_column(
+        Enum(PipelineType, name="pipeline_type"), nullable=True
+    )
+    is_related_to: Mapped[list[uuid.UUID] | None] = mapped_column(


This should be a relationship, not an array of ids because there will be no referential integrity if these ids are deleted.

Yes, perhaps this could be handled through the is_derived_from source and target table we discussed previously, in which case the is_related_field could be removed. But even if that is done, deletion of an id means that it would still need to be removed from the relationship table.

I am not sure I understand. I read the document but I don't know what the relationship is . Is it a derivation ?

@jdcourcol this was preserved from the current implementation (we discussed in one of the first meetings). In my understanding it is the relationship of raw morphology - linked to a repaired - linked to a clone etc.

app/db/model.py

app/db/types.py

app/schemas/morphology.py

eleftherioszisis · 2025-07-14T07:06:13Z

app/schemas/morphology.py

+        Whether data has been corrected for shrinkage
+    """
+
+    protocol_design: str


This should be constrained with a StrEnum

@lidakanari what items should be in the protocol_design StrEnum?

I think the protocol design is added when we have a respective document for this. We only have a full protocol document for LNMC, and the rest are from papers. There was a list shared with @jdcourcol last year, but I'm not sure what is best to do here, should we register protocols, link to existing papers?

app/db/model.py

jdcourcol · 2025-07-23T06:25:33Z

app/db/model.py

+class ComputationallySynthesizedMorphologyMethod(MorphologyProtocol):
+    __tablename__ = "computationally_synthesized_morphology_protocol"
+    id: Mapped[uuid.UUID] = mapped_column(ForeignKey("morphology_protocol.id"), primary_key=True)
+    method: Mapped[str] # This 'method' might need further typing based on your vocabulary


why not "name" ? and why not a property on the base "MorphologyProtocol" class ?

My understanding is that this is particular to the SynthesizedMorphology class. But perhaps it could be changed to name, if @lidakanari thinks it is okay.

app/db/model.py

jdcourcol · 2025-07-23T06:28:05Z

app/db/model.py

+class ModifiedMorphologyMethod(MorphologyProtocol):
+    __tablename__ = "modified_morphology_protocol"
+    id: Mapped[uuid.UUID] = mapped_column(ForeignKey("morphology_protocol.id"), primary_key=True)
+    method: Mapped[MethodsType] # Assuming MethodsType is an Enum or similar


above it is a string. Here is it a enum ?

Yes, it is polymorphic because the methods can vary considerably.

app/db/model.py

jdcourcol · 2025-07-23T06:32:11Z

app/db/model.py

+    __tablename__ = "morphology_protocol"
+    protocol_document: Mapped[str | None]
+    protocol_design: Mapped[str]
+    type: Mapped[str]  # Discriminator column for polymorphism


do we need that ? @GianlucaFicarelli ?

jdcourcol · 2025-07-23T06:32:41Z

app/db/model.py

+
+class MorphologyProtocol(Identifiable): # Inherit from Identifiable for primary key and timestamps
+    __tablename__ = "morphology_protocol"
+    protocol_document: Mapped[str | None]


I don't understand what that is ? a URL ? a DOI ?

I think it is the url pointing to the protocol document. But perhaps we should upload the document into the system and have this point to it? What do you think?

jdcourcol · 2025-07-23T06:32:49Z

app/db/model.py

+class MorphologyProtocol(Identifiable): # Inherit from Identifiable for primary key and timestamps
+    __tablename__ = "morphology_protocol"
+    protocol_document: Mapped[str | None]
+    protocol_design: Mapped[str]


same here ?

app/db/model.py

eleftherioszisis · 2025-07-28T07:14:39Z

app/db/model.py

+        ForeignKey("morphology_protocol.id"), nullable=True, index=True
+    )
+    # Relationship to the polymorphic MorphologyProtocol
+    morphology_protocol: Mapped["MorphologyProtocol"] = relationship(


Given that the morphology protocol carries information about the generation_type, do we really need it as a separate column?

If we need to filter by generation_type for instance we could introduce a nested filter of the form morphology_protocol__type="experimental"

@eleftherioszisis are you proposing to remove the generation type and just use morphology protocol ?

The only issue I can see with that is that placeholder might not have a protocol.

But perhaps it would just be empty in the case of placeholder. Okay, I will remove generation type.

@eleftherioszisis please have a look and see if you agree if the way this is now done. I've also changed the schema to remove generation type.

Thanks. Let me clean and get the PR in shape first.

app/db/model.py

Co-authored-by: Eleftherios Zisis <[email protected]>

dkeller9 added 2 commits July 12, 2025 15:29

only db and schema

846b608

cleanup

e7a06e4

dkeller9 requested a review from jdcourcol July 14, 2025 05:49

dkeller9 added 2 commits July 14, 2025 08:31

linting

71b41cc

made types auto

c3ed72a

eleftherioszisis reviewed Jul 14, 2025

View reviewed changes

dkeller9 added 6 commits July 14, 2025 10:26

changed how pipelinetypes is referred to

bf32ac6

more linting

e6ab103

fixed import

8092632

fixed MorphologyStructureType

b1a03c7

more linting

1e9ae38

removed redundant imports

2340de3

jdcourcol reviewed Jul 15, 2025

View reviewed changes

app/db/model.py Outdated Show resolved Hide resolved

jdcourcol reviewed Jul 15, 2025

View reviewed changes

app/db/model.py Outdated Show resolved Hide resolved

dkeller9 added 4 commits July 15, 2025 14:53

reorganized types

8ab30b7

removed scoredict and separate metadata table

274ca8e

change morphology method t protocol

76ea9a7

augmented types

78a0f66

jdcourcol reviewed Jul 22, 2025

View reviewed changes

app/db/model.py Outdated Show resolved Hide resolved

jdcourcol reviewed Jul 22, 2025

View reviewed changes

app/db/model.py Outdated Show resolved Hide resolved

jdcourcol reviewed Jul 22, 2025

View reviewed changes

app/db/model.py Outdated Show resolved Hide resolved

jdcourcol reviewed Jul 22, 2025

View reviewed changes

app/db/model.py Outdated Show resolved Hide resolved

small changes in names

25ae533

jdcourcol reviewed Jul 23, 2025

View reviewed changes

app/db/model.py Outdated Show resolved Hide resolved

jdcourcol reviewed Jul 23, 2025

View reviewed changes

app/db/model.py Outdated Show resolved Hide resolved

jdcourcol reviewed Jul 23, 2025

View reviewed changes

app/db/model.py Outdated Show resolved Hide resolved

jdcourcol reviewed Jul 23, 2025

View reviewed changes

app/db/model.py Outdated Show resolved Hide resolved

jdcourcol reviewed Jul 23, 2025

View reviewed changes

app/db/model.py Outdated Show resolved Hide resolved

jdcourcol reviewed Jul 23, 2025

View reviewed changes

app/db/model.py Outdated Show resolved Hide resolved

dkeller9 added 3 commits July 23, 2025 09:41

small changes in field names

1102fb0

fixed table names

835cc3c

removed provenance

c47e87d

eleftherioszisis reviewed Jul 28, 2025

View reviewed changes

dkeller9 and others added 3 commits July 28, 2025 09:19

Update app/db/model.py

cc6c5aa

Co-authored-by: Eleftherios Zisis <[email protected]>

resturcturing

6d8ee21

removed generation from schema

15b5b03

Morphology model #282

Are you sure you want to change the base?

Morphology model #282

Uh oh!

Conversation

dkeller9 commented Jul 14, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

eleftherioszisis Jul 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

eleftherioszisis Jul 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dkeller9 Jul 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

eleftherioszisis Jul 14, 2025 •

edited

Loading

eleftherioszisis Jul 28, 2025 •

edited

Loading

dkeller9 Jul 28, 2025 •

edited

Loading