Significantly improve filtering and sorting performance for large datasets (500k+ rows)

[SISTF]

This is an attempt to improve filtering and sorting on larg datasets.

This is a first attempt to solve #195

This commit introduces a major refactor and optimization of the TableView filtering and sorting logic to address severe performance issues with large datasets (500,000+ rows).

Unified fast code path for sorting and filtering, used by both header clicks and options flyout, ensuring consistent and optimal performance.
Parallelized filtering and sorting using chunked processing and ConcurrentDictionary for thread safety and speed.
Smart caching of unique filter values, with cache keys that include filter state, to avoid unnecessary recomputation.
Scroll position preservation: The horizontal scroll position is now reliably preserved and restored during data operations, eliminating UI jumps.
Improved blank/null value handling in filters, ensuring consistent user experience.
API additions: Added ApplyUnifiedSort and ClearUnifiedSort to IColumnFilterHandler for unified, fast sorting from any UI entry point.
Error handling and thread safety: Added guards against infinite loops, race conditions, and UI glitches during data operations.
These changes dramatically reduce the time required for filtering and sorting on very large datasets, making TableView responsive and usable at scale.

My tests resulted in significant improvements, so i made it ready to review and test or discuss further.

[michael-hawker]

@w-ahmad is this something you need help reviewing/testing? It's good to know others are trying to use the component with really large data sets. (I was curious if being off of ListViewBase vs. ItemsRepeater would have any other impact, but maybe that's just a micro-optimization that's not worth-while if improvements are gained from other changes like this.)

It does seem like a lot of new code, so I imagine having stuff added to the unit tests could help, eh?

Would it make sense to setup any benchmark type tests as well?

[w-ahmad]

Hi @michael-hawker, thanks for your suggestions and for offering to help with review/testing! I’d be very happy if you could provide any insightful review or testing.

I saw this PR and yes, there is a lot of new code that I wanted to review after testing the TableView with a larger dataset to decide whether it really makes any improvements.

Regarding ListViewBase vs. ItemsRepeater, I think the TableView performs very well in terms of virtualization, cell creation, and displaying even with large datasets. However, filtering tasks do seem to be a bottleneck right now. When I initially created the control, I tested it with over a million rows and performance was fine, so it's interesting to why it became slow over the time.

I agree, setting up benchmarks sounds like a great idea, but to be honest, I’m not sure where to start with that. If you have any recommendations for tools or examples of how to set up meaningful benchmarks for WinUI projects, I’d really appreciate it! And if you’re able to help by adding any benchmark or unit tests for this purpose, that would be super helpful too.

Thanks again for your support!

[SISTF]

I agree and so does the title say, its allready really fast in displaying big datasets. Its just the filtering/sorting thats a real bottlneck.
While benchmarks could prove it, the difference is allready really good visible by just trying it out or look at the samples i provided in #195 .
Feel freee to reach out to me if i can help further on this topic.

[w-ahmad]

Thanks @SISTF! I checked the code and found two reasons why the filter is slow. First, when we filter a column, the filter method runs twice, doubling the time it takes. Second, the filtering process doesn’t use defer refresh, unlike the sorting process. If you allow, I can push the changes to this PR after reverting the changes that you made in commit 0f1329e or do I can create a new PR. There is still room for improvements that I'm trying to do.

here is an example. there are exact 500k items in this table and see how fast it performs filter operation on a single column.

Screen.Recording.2025-12-08.194136.mp4