If you're in the stage viewport, looking through a camera with the camera handle activated, middle mouse sometimes pans the camera rotation only. It seems like it might also rotate around the geo under the mouse, but it seems random when it decides to.
Looks like it works under the object viewport too, but if you click on the right side of the viewport, it goes into crazy mode. The left side seems to do what you want though.
