[solved] mongodb + dynamic datatable

Webix JS
#1

Hello !

I’m troubleshooting an unexpected behavior using a datatable with dynamic:true.

When scrolling with the mouse at the end of the datatable, the last X (can be the last row only, but can be the last 3 rows too, it’s random) rows are not showing data, i.e rows are visible but empty, and html is

<div role="gridcell" class="webix_cell"></div>

rather than

<div role="gridcell" aria-rowindex="XXX" aria-colindex="Y" class="webix_cell">column value here</div>

I already checked API requests/responses, everything is fine (lasts records are loaded as expected, got pos/total_count attributes …). I ended up my debugging with a clean datatable config with default values, i.e the datatable config only contains dynamic:true and api url, i don’t play with rowHeight, adjust(), css or something.

Server side, i double check that default sorted column values (something like a date, “lastUpdated”) is not changing between 2 scroll movement while debugging, because it can explain empty row (btw i don’t see how to handle this properly)

I can’t use webix snippet right now because (of the API). Does a snippet having datatable+dynamic+local data able to handle local data load + scroll already exist ?

The initial unexpected behavior which make me investigate this scroll problem was

  • blank rows too, but not only at the end. Can be anywhere but we can see that’s it’s after receiving new data. Not all the time, refreshing page make blank rows too, but not at the same position.
  • when scrolling to the bottom to load all data, blanks rows appear, but when i scroll up, then down again, values of the latests rows was changing … wired

Any suggestions ?

#2

looks like you have inconsistent data.

  1. check if you have repeating ids
  2. check if total_count is wrong (bigger than real)

wrong behavior is reproduced here

#3

Thank you @intregal

  1. ids are unique id from mongodb, i can not have duplicate. Except if the sorted order change between 2 scroll (default last update date sorting by example), but i double check this case in this troubleshoot and i always have the same sort order coming from the server between 2 scroll movement.

  2. total_count is OK too, i triple check that.

Any other suggestions ?

#4

Note my previous message:

And a last point too:

Depending of the number of record, sometimes, the last empty row make ajax request looping.

Question. Does the “pos” attribute returned by the server should be equal to “start” request parameter or start+1 ?

#5

pos is 0 based. normally it should be equal to start.
and looping request is occured when total received items count is less than total_count

#6

bug
bug1217×602 150 KB

Btw i’m using this to transform mongodb id to webix id.

webix.DataStore.prototype.id = function (data) {
  data.id = data._id || data.id;
  delete data._id;
  return data.id || (data.id = webix.uid());
}

And this, to get total_count, pos from the “meta” attribute api response

webix.DataDriver.mongoresult = {
  getInfo:function(data){
    this.meta = data.meta;
    return {
      size:(data.meta?.total_count||0),
      from:(data.meta?.pos||0),
      parent:(data.parent||0),
      config:(data.config)
    };
  },
  getMeta(attr) {
    if (!this.meta) return;
    if (typeof this.meta[attr] === 'undefined') return;
    return this.meta[attr];
  }
};

webix.extend(webix.DataDriver.mongoresult, webix.DataDriver.json);
#7

Mhhh. Mongodb side, when sorting with a date and if multiple records have the same date, then order of results is random !

“looks like you have inconsistent data.” => true

#8

Thank you @intregal with your “strict” first response.

Solved by adding a double sort mongodb side: sort by date AND _id

1 Like