Click the
Remove Objects from Simulation button on the Drive Simulation tab.
Select the dynamic objects to remove from simulation and press Enter to confirm your selection.

In this example, the torus has been removed from the simulation and the sphere is still simulated.