Skip to content

Feature/nmr mlp#8

Open
tomoqt wants to merge 3 commits intomainfrom
feature/NMR_MLP
Open

Feature/nmr mlp#8
tomoqt wants to merge 3 commits intomainfrom
feature/NMR_MLP

Conversation

@tomoqt
Copy link
Owner

@tomoqt tomoqt commented Jan 10, 2025

added a simple 2x MLP to encode raw NMR instead of interpolating them to 1k. both NMRs are projected to memory_dim and concatenated over seq_len.
Transformer decoder should be ok, as uses whatever seq_len comes in.
note this doesn't cntain info about the multiplets (for now), plus the projection is pretty sharp, especially for low embed_dim (maybe should be more bottleneck than expanding dimension MLP...). note that this way we use the full embed_dim for IR, hence modalities are less separated between different attention heads.
setting use_mlp_for_nmr=False gives back regular functionality.
maybe a next step could be validating the effect of regular vs this. eventually we can:
-keep convnext, change the backbone HPs to downsample more and get around regular seq_len as the IR, then pad and concatenate as usual over embed_dim.

  • change preprocessing over NMR (FT? CWT?discretize space of peaks like the paper did?)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant