LMB to increase distance doesn't work. Nor does shift+LMB to change the step.
Bug or a problem on my side?

No problem in my 411

Yea, I've found the problem, but the solution is not one that delights me. I've copied a keymap from a previous version (17.5 I think) and I was calling the old bevel with my custom hotkey and it worked as expected. However, calling the new bevel from shelf made it go haywire.
The reason I'm not stoked about the solution, is that the point of having a keymap is kinda negated in this case if other problems appear. I hope this problem surfaces only when some tools are replaced with a newer version. Remains to be seen.