Is there a possibility to collapse a group in Unitlist?

If not, any pointers how could I try making a custom component with such functionality? Or maybe a grouplist which collapses vertically?

No, I can suggest only an approximate solution: you can use a custom tree from our adminApp demo with grouping:

http://webix.com/snippet/b2f18295