Detail List - Paging not working with collection over 2000 record limit

[hugolini]

Hi All,

Paging is working fine on large data sets, when grid item source is a datatable, but when i change it to a collection it's seems limited to the max 2000 record limit.

Any step missing on my end?

[ssimbox]

This is not a DetailList issue, but a Powerapps choice by design. As you can read on Microsoft Learn, ClearCollect is a non-delegable function and for this reason the gallery is returing only 2000 records (that is your data row limit).

[hugolini]

Thanks for the reply.

I have a need to add a custom column to the dataset, that's why I'm using a collection instead of a dataverse table.

Any way to use pagination, when binded to a collection, with more than 2000 records?

[ssimbox]

Remember that the Pagination within the detailList is only visual: your app "downloads" che 2000 records and then they are splitted into multiple pages, there is not dynamic stream.
In order to have more than 2000 records ask a question on the Poweruser forum, but most probably you have to rearrange your queries or Data model because collection is not your need.

[yaseralhosani]

Thanks for the reply.

I have a need to add a custom column to the dataset, that's why I'm using a collection instead of a dataverse table.

Any way to use pagination, when binded to a collection, with more than 2000 records?

In Power Apps, there's a limitation known as the "delegation warning," which restricts the amount of data that can be retrieved from data sources at a time to 2,000 records. This is designed to ensure performance isn't hindered in large databases. However, when you need to work with large datasets, this can be restrictive.

To your question, using pagination with a collection of more than 2,000 records requires a bit of strategic data handling due to this limit. You'll often need to work around the delegation limits.

Why you would see more than 2k records could be due to a need for comprehensive data analysis, reporting, or even handling bulk operations, which require you to work with the entire data set. However, it's essential to consider that displaying vast amounts of data can affect the app's performance. Therefore, it's advisable to work with only necessary records for the best performance and user experience.

Regarding the solution, you can use the 'Collect' function along with other functions like 'ClearCollect' or Clear to manage and gather more than 2k records. Here's a simplified example of how you might do this:

Clear(colLargeData); // Clear the existing collection
Collect(colLargeData, Filter(DataSource, ID <= 2000)); // Collect the first 2000 records
Collect(colLargeData, Filter(DataSource, ID > 2000 && ID <= 4000)); // Collect the next 2000 records
// Continue this pattern as needed

This script clears the 'colLargeData' collection and then uses the 'Collect' function to pull in records from 'DataSource.' It's done in batches, using the 'Filter' function to handle different segments of records. You'd need to adjust the filtering conditions based on your data's unique identifiers and structure.

Remember, this method can be resource-intensive, and there are limits to how many records can be stored in a collection. Also, using multiple 'Collect' functions sequentially can lead to slower app performance, so it's recommended to use this approach judiciously and only when necessary. However, it's a bit complex to implement, as it requires careful management of the data loaded to avoid duplication and maintain performance.

[hugolini]

Many thanks to everybody that answered my questions.

Cheers!!! :)