residuals module?

[djkapner]

for discussion:
what do you think about a renderapi.residuals module addition. I've recently gone back to my resduals code for the solver, and made it more render-pythonic (not using mongo, using the transform module) and thus much cleaner than it was initially, when I had duplicated some of the functionality already here.

Right now, the capability lies in a qctools sub-module of EM_aligner_python. My rationale for moving it out of there is that it is solver-agnostic. All it needs are stacks and pointmatches.

For example, this would be one key function of the proposed module (can refine style later):

def transform_pq(tspecs,matches):
    p = []
    q = []
    p_transf = []
    q_transf = []
    tids = []
    for ts in tspecs:
        tids.append(np.array([x.tileId for x in ts]))
    tsp_ind0=0
    tsp_ind1=1
    if len(tspecs)==1:
        #montage
        tsp_ind1=0

    for match in matches:
        #cycle through each item, find and apply the appropriate transforms
        sind0 = np.argwhere(tids[tsp_ind0]==match['pId'])
        sind1 = np.argwhere(tids[tsp_ind1]==match['qId'])
        if (len(sind0)>0) & (len(sind1)>0):
            sind0 = sind0[0][0]
            sind1 = sind1[0][0]
            ip = np.flipud(np.rot90(np.array(match['matches']['p'])))
            iq = np.flipud(np.rot90(np.array(match['matches']['q'])))

            ip_transf = tspecs[tsp_ind0][sind0].tforms[-1].tform(ip)
            iq_transf = tspecs[tsp_ind1][sind1].tforms[-1].tform(iq)

            p.append(ip)
            q.append(iq)
            p_transf.append(ip_transf)
            q_transf.append(iq_transf)

    return [p,q,p_transf,q_transf]

another simple one (imagine as function), to turn these into residuals:

self.p,self.q,self.p_transf,self.q_transf = transform_pq(tspecs,matches)
self.xy_ave = [0.5*(p+q) for p,q in zip(self.p_transf,self.q_transf)]
self.xy_diff = [(p-q) for p,q in zip(self.p_transf,self.q_transf)]
self.rss = [np.sqrt(np.power(x[:,0],2.0)+np.power(x[:,1],2.0)) for x in self.xy_diff]

I am imagining calculations like this, which require only renderapi and numpy, and then, perhaps adding some of the plots I've been making to the externals module, where they would need matplotlib and mpl_scatter-density, for example.

[RussTorres]

I think the discussion on these kinds of things is ongoing -- that is, what do we put in the renderapi vs one of the dependent repos (render-modules/render-python-apps). In fact, render-modules has a rendermodules.residuals module which has some similar utility (just a function without schema validation.)
If we had an optimized helper function like this that was sufficiently general (e.g. does not assume our current criteria for pointmatch derivation and transform structure) we could definitely include it. The other (better?) option is to allow the render to make these associations and transform to a stack's global coordinates as a client or api call.