Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New PDL capability can reduce data-copying #81

Open
mohawk2 opened this issue Sep 6, 2024 · 3 comments
Open

New PDL capability can reduce data-copying #81

mohawk2 opened this issue Sep 6, 2024 · 3 comments

Comments

@mohawk2
Copy link

mohawk2 commented Sep 6, 2024

Hi, in PDL 2.091 (just released), this code:

my $pdl = PDL->new;
$pdl->set_datatype($pdl_t);
$pdl->setdims([$xdim, $ydim]);
my $data = $pdl->get_dataref();
# FIXME: see http://pdl.perl.org/PDLdocs/API.html how to wrap $buf into a piddle
$$data = $buf;
$pdl->upd_data;

could be slightly shorter and also more efficient, using the same capability incorporated for PDL::Parallel::threads (see PDLPorters/PDL-Parallel-threads@fc7c3d8), to something like:

my $pdl = PDL->new_around_datasv(0+\$buf);
$pdl->set_datatype($pdl_t);
$pdl->setdims([$xdim, $ydim]);

The new new_around_datasv constructor increases the refcount of the provided SV, so $buf going away at the end of the scope wouldn't be a problem. The advantage is that the data wouldn't need copying into the new ndarray's data area. Obviously, you'd need to bump your minimum-supported version of PDL to 2.091.

Please say if you'd like a PR executing this.

@shawnlaffan
Copy link
Collaborator

A PR would be good, but please version guard it.

my $pdl;
if ($PDL::VERSION >= 2.091) {
    ... #  new code
}
else {
    ... # old code
}

This will lead to duplicate code but it does mean that users do not need to update PDL to keep using the latest Geo::GDAL::FFI. The Strawberry Perl 5.40 PDL version, for example, comes with PDL 2.089. SP 5.38 has PDL 2.085. FreeBSD is mostly on PDL 2.089 at the moment.

The versioned code can be cleaned up once PDL 2.091 or greater is in common use.

@mohawk2
Copy link
Author

mohawk2 commented Sep 7, 2024

If you're reluctant to require the latest PDL, then I'll leave this with you.

@shawnlaffan
Copy link
Collaborator

If you're reluctant to require the latest PDL, then I'll leave this with you.

OK. I'll leave this open so it can be implemented.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants