Skip to content

Pivot rendering in the Koltin notebook plugin is slow #674

Open
@ermolenkodev

Description

@ermolenkodev

While working with a relatively big dataset (~10m rows), I observed that rendering the table for a pivot result is extremely slow—about 10 seconds for a 10m rows dataframe.
After a brief investigation, I found out that the problem is with converting the pivot object to a dataframe. That is implemented like that pivot.frames().toDataFrame()
TODO: investigate what is the bottleneck and optimize the Pivot->Dataframe conversion process

Metadata

Metadata

Assignees

No one assigned

    Labels

    performanceSomething related to how fast the library can handle data

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions