Rendering Large amount of data

edited January 2014 in General

Hello All,

I try to display 45000 data in webix pivot table, The API is not able to render this amount of data and browser becomes unresponsive and eventually i have to close the browser.

The data is coming in following format from server :

[{"DBD":5,"SLF":44999,"Remark":"Test Remark49","date":"07.03.2014 12:09","comp":"C","CAP":45098,"number":"2"}]

Please suggest how can i resolve this problem. Might be i am doing something wrong.

Comments

  • Loading 450k dataset in pivot takes about 4 seconds in my case.
    How much column and rows has the result data table after data aggregation ?
    Run code like next to get the metrics

    var table = $$("pivot").$$("data");
    console.log(table.config.columns.length, table.count());
    
  • Hello Maksim,

    Thanks for the reply.

    The dimension of matrix is 50 x 900. Here the number of rows are 50 and columns are 900. So the total number of cells generated are 45000.

    Apart from loading data from server. I even try to load this data from flat json file placed at directory location but webix API is not able to render the data.

    Kindly suggest.

  • The size of result data is not too big, it must not cause any visible dealy ( as pivot will render only visible part initially )

    The problem may be related to the big ammount of the initially loaded data. You can try to use the profiling code from the above linked post, it will provide some info on which stage performance problem does occur.

  • Hello Makism,

    I have an requirement to display 365 * 3000 rows and columns along. For this I generate a JSON file which if of 45 MB. When i try to display this data in PIVOT table, the API is not able to render and browser went to non-responding state.

    I am creating a json statement as of below structure and this structure is repeated for 365 * 3000.

    {"A":1,"B":"XYZ","C":4,"DT":0,"CO":"RR","RMK":"TEST"},

    As per you previous post, I am even not able to profile the code because somehow browser becomes non-responsive.

    Please suggest how can i achieve my requirement

  • edited January 2014

    If I got the above correctly, you want to load about a 1 000 000 of json records (365x3000), which will contain about 5 000 000 values for processing(365x3000x5). That is quite a lot.

    The Pivot can't handle it currently, but I have checked the source code and it seems there is still a place for optimization. It is possible to tune the component, so it will have the rendering time below 5 seconds for such kind of dataset. ( in Google Chrome and Firefox, it will be slower in latest IE and will be not operable in older IE anyway )

    Unfortunately, I can't provide any time estimate, when the solutin will be available, as this is not a high priority task for now. If you interested in the solution, you can contact sales@webix.com about such enhancing http://webix.com/services/

  • Hello all,

    Have been two years since the last response and I still with a problem. I did all the tests that maksim suggested on the reply above and I'm still with a problem.

    Follows my Results:

    data parsing: 1.951ms
    list.js:86 data processing: 3286.053ms
    list.js:91 data rendering: 12781.174ms

    var table = $$("pivot").$$("data");
    console.log(table.config.columns.length, table.count());
    table.config.columns.length = 17 ; table.count() = 179

    This is not a big table, 17*179 gives us 3043 cells and I got 12 seconds to render the pivot. What can I do to improve this one ?

  • Can you share the configuration of pivot? 17*179 is relative small dataset, and it must not take more that 1 second for rendering.

    If you can share a full demo or demo link, please send me private message with details.

  • By the way, there were two major performance updates since the original topic, the current version of Pivot must work up to 10 times faster for datasets with a huge number of columns.

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!