Skip to content

Interpolation of *very* large arrays is too slow #3

@alex-robinson

Description

@alex-robinson

The mapping algorithm used here requires searching the input grid/points to determine the nearest neighbors. If the input data contains many points, this can be extremely slow. This is improved by using the lat_lim parameter when calling map_init(): lat_lim specifies the maximum difference in latitude between the current point and the input data that can be considered in the neighborhood. In this way, an easy difference in latitude calculation is performed and then only neighbors within a certain latitude band are considered.

However the second reason the mapping can be slow is that during the actual mapping step, the map contains a set of indices of the input points. These indices can be spread all over the input points (in terms of memory), thus causing a lot of inefficient data access.

Any potential method to reduce computation time is complicated by the fact that all mapping algorithms are designed for 1D vectors. This means that if the input data is on a 2D grid, this is first transformed to a 1D vector. This makes it difficult to use inherent spatial relationships of the indices of the points (ie, the neighborhood should be index i,j +/- 10 points). Instead the only relative location information available for use is the lon/lat of each input point and each output point.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions