Dealing with datatable parsing a large amount of data

Webix JS
#1

I’m sending an ajax request, and receiving a large JSON string which I’m parsing to a Datatable.
Before sending the AJAX request, I’m overlaying a loader sign on the page; which I remove after the “datatable.parse(mydata)” statement.
Now the problem seems to be this:
Because of the large size of the ‘mydata’ JSON that I’ve received from the server, Webix seems to be taking a lot of time on the datatable.parse(mydata) statement. And that is causing not just the loader, but also the tab itself to become unresponsive.

Is there any way to deal with this sort of thing? I don’t mind Webix taking an extra 30-40 seconds to parse the JSON, but if I could just prevent the loader from going unresponsive, it would be far better user experience.

#2

How many data objects in this big JSON string ?

The parsing operation is sync by design, and will block all other interaction. Still the parse time must be really small ( for json it mostly one call of JSON.parse and some array reconstructing )

For really huge datasets it may be better to use dynamic loading feature. If you need to process the huge chunk at once the next can be done to optimize data loading

  • a) use JSON.parse to convert string to array of objects
  • b) split array in chunks
  • c) call parse method with first chunk
  • d) repeat step (c) with next chunk

To break the freezing problem you need to call step (c) not directly by with 1ms timeout ( setTimeout )

#3

The dataset I’m loading has 160000 rows. I’ll try and implement the chunked loading method you described, thanks.
Meanwhile, what’s the way to clear all data in a datatable? It’s a very trivial question but I haven’t been able to figure out how to do it.

#4

For data deleting

$$("dt").clearAll();
#5

a) use JSON.parse to convert string to array of objects
b) split array in chunks
c) call parse method with first chunk
d) repeat step (c) with next chunk

Hey, I’m unable to implement this., I’ve been trying for a couple of hours now. I don’t mean to make you spoonfeed here but I think I’m out of my depth; so I’d really appreciate if you could help out. Thanks!

#6

Plain data loading, 200k rows - http://webix.com/snippet/1c99f34f
It takes about 2 seconds in my case

The same data but in “distributed” scenario - http://webix.com/snippet/9476e2b4
It does look as instant loading for me.