feature: optional rotation for subvolume

[McHaillet]

@alisterburt what do you think about having an optional rotation matrix parameter for the subtomo reconstruction here?

torch-tomogram/src/torch_tomogram/tomogram.py

Line 70 in 1991b8a

self, point_zyx: torch.Tensor, sidelength: int

For me it makes sense because I can give an additional rotation to the subtomogram as an augmentation. In a wider sense it could be useful because you might have a known rotation of the subvolume (particle) and can immediately apply it here for reconstruction without requiring an additional interpolation step.

[alisterburt]

Sure thing! Thanks for discussing before ploughing ahead. I'd like to make sure we have a really good sense of what's going on physically when applying this extra matrix, not being careful can lead to all sorts of confusion

My suggestion

• column vectors of supplied rotation matrix should be rotated z/y/x vectors in tomogram coordinate system
• rotated z/y/x in matrix should become basis z/y/x in the reconstructed subvolume
• compose rotation matrices before the inversion step so that the composition is easy to interpret physically

If any of this doesn't make sense please ping and we can hash it out :-)

[alisterburt]

Thought came into my head whilst walking - access to this final matrix should also be provided without going via this method, so we should have something like:

Tomogram.get_composed_rotations(rotation_matrix)

this can be used within the reconstruction by passing the optional parameter, as you suggested

[alisterburt]

API is probably inconsistently named compared to what exists, didn't look before writing

[McHaillet]

My suggestion

* column vectors of supplied rotation matrix should be rotated z/y/x vectors in tomogram coordinate system

What do you mean here?

* rotated z/y/x in matrix should become basis z/y/x in the reconstructed subvolume

And here?

* compose rotation matrices before the inversion step so that the composition is easy to interpret physically

Defnitely!

The things above I don't exactly know what you mean. Let me just say how I was imagining it and you can fill on to that how you view it.

I though the argument is a rotation matrix (that works on zyx coordinates). The alignment parameters that are present in the Tomogram object first are used to sample the patches from the tilt series. Than the provided rotation matrix is multiplied with the tilt_rotation matrices for the Fourier inversion.

something like this (without the part of get_composed_matrix yet) :

    def reconstruct_subvolume(
        self, point_zyx: torch.Tensor, sidelength: int, subvolume_rotation_mtarix: torch.Tensor | None = None,
    ) -> torch.Tensor:
        """Reconstruct a subvolume at a 3D location in the sample."""
        point_zyx = torch.as_tensor(point_zyx).float()
        point_zyx = point_zyx.reshape((-1, 3))
        tilt_rotation_matrices = self.projection_matrices[:, :3, :3]
        tilt_rotation_matrices = torch.linalg.pinv(tilt_rotation_matrices)
        sidelength_padded = int(self._pad_factor * sidelength)
        particle_tilt_series = self.extract_particle_tilt_series(
            point_zyx, sidelength=sidelength_padded
        )
        if subvolume_rotation_matrix is not None:
            tilt_rotation_matrices = subvolume_rotatation_matrix @ tilt_rotation_matrices
        volume = backproject_2d_to_3d(
            images=particle_tilt_series[0],
            rotation_matrices=tilt_rotation_matrices,
            zyx_matrices=True,
            pad_factor=1.0,  # we already incorporate padding in subtilts
            fftfreq_max=0.5,
        )
        p = (sidelength_padded - sidelength) // 2
        volume = F.pad(volume, [-p] * 6)  # remove padding
        return volume

[alisterburt]

exercise in clarity incoming

I agree with your general approach of

• providing an additional rotation matrix
• this matrix being composed with the rotation component of the projection matrix

I'm trying to make sure we have a consistent, coherent physical interpretation of what each transform means - this then informs how to construct/manipulate each matrix when e.g. constructing priors on particle orientations

Coordinate systems

microscope coordinate system

• we have a sample in an electron beam
• origin is at center of sample
• beam direction is along z (moving positive to negative, electrons are raining down on the detector)
• detector is in the xy plane, perpendicular to the beam

sample coordinate system

this is what we reconstruct when reconstructing a tomogram (with an offset origin)

• origin at center of sample
• z would cover thickness of untransformed sample (i.e. 0 degree image)
• xy covers extent of sample in detector plane in 0 degree image

particle coordinate system

this is what we reconstruct when we average particles (with an offset origin)
For the concrete example of a membrane protein on a membrane

• origin at the center of the particle
• membrane spans xy plane
• z is normal to the membrane

Matrices

projection matrix

we have the 'projection matrices' in our implementation, let's call them $M_{proj}$ what's the physical interpretation of those?

• $M_{proj}$ is a 4x4 matrix which transforms (x, y, z, 1) points in the sample to their final positions (x', y', z', 1) in the microscope coordinate system, for a particular projection image
• (x', y') gives us the projected position of (x, y) on the detector

Let's call the rotation component of the 4x4 matrix for a particular projection image $R_{proj}$
fun facts about $R_{proj}$ !

• column vectors of $R_{proj}$ are the directions x/y/z from our sample coordinate system point in the microscope coordinate system after transformation

This means $R_{proj}$ defines the orientation of our sample within the microscope.

Particle matrix (proposed addition)

You're proposing to add a rotation which can be applied during the reconstruction. This rotation should define the orientation of our particle within the sample.

Let's call this $R_{particle}$. Column vectors of $R_{particle}$ should be the directions x/y/z from our particle coordinate system point within the sample coordinate system

Reconstruction

In math

By composing rotations, we can now calculate the orientation of the particle in the microscope for each projection image

$R_{final}$ = $R_{proj}*R_{particle}$

In code

I've rewritten the matrix handling snippet from the reconstruct_subvolume code to include the implicit particle matrix not currently present for clarity

• sample_orientation_in_scope is $R_{proj}$ for each tilt
• particle_orientation_in_sample is $R_{particle}$
• particle_orientation_in_scope is $R_{final}$

sample_orientation_in_scope = self.projection_matrices[:, :3, :3]  # (n_tilts, 3, 3)
particle_orientation_in_sample = torch.eye(3)  # (3, 3)
particle_orientation_in_scope = sample_orientation_in_scope @ particle_orientation_in_sample  # (n_tilts, 3, 3)

The reconstruction code takes a matrix which defines the orientation of our microscope within the particle being reconstructed, this is the inverse of $R_{final}$ which described the orientation of the particle within the microscope, hence a final inversion

sample_orientation_in_scope = self.projection_matrices[:, :3, :3]  # (n_tilts, 3, 3)
particle_orientation_in_sample = torch.eye(3)  # (3, 3)
particle_orientation_in_scope = sample_orientation_in_scope @ particle_orientation_in_sample  # (n_tilts, 3, 3)
microscope_orientation_within_particle = torch.linalg.pinv(particle_orientation_in_scope)

Does this make everything clear? Do you feel like you understand why I care about this being crystal clear? 😂

[McHaillet]

solid write up, you should add this to the website!

[McHaillet]

Let's call this $R_{particle}$ . Column vectors of $R_{particle}$ should be the directions x/y/z from our particle coordinate system point within the sample coordinate system.

So this defines how to forward transform our reference particle to what we find in the sample, correct?

[alisterburt]

Thanks! 🙏

Yes, you got it 🙂