It seems like cooking in-process work items will use as much CPU as possible. So if I put some complex calculations in onCookTask() callback, it will freeze Houdini UI sometimes. Can it be limited by the scheduler?

Not yet, but it's on our RFE list. The workaround for now is to use a feedback loop to limit execution to be done in serial.