I'm trying to get a glued wall to levitate and then get hit and destroyed. very basic setup. I attached a file, if you run it for a minute you can see the pieces are doing weird stuff. Any idea whats the issue here?

As soon as I plug in collision geo to the rbd solver the wall isn't falling anymore but it sticks in the air. Thats weird. It starts hearing to gravity when I unplug the geo.

Not entirely sure what you're trying to achieve but there were a number of issues in the setup.
One big issue was the animation was done early on in the graph (a simple xform in Y) which makes the whole setup unnecessarily time dependent.
The main reason things aren't behaving as expected is you are then applying a transform pieces to the render geo only (not the constraints or the proxy geo).
The easiest way to fix the setup and improve performance is to move the animated transform just before the RBD Bullet Solver, and have it transform all 3 streams (render geo, constraints and proxy geo) at the same time by RBD Pack'ing the output of the RBD Configure SOP.
Here's an annotated version of your hip file that goes through a few of the issues and has the wall move upwards before getting hit by the collision sphere.

Generally, a good way to check the sim if it's not doing what you expect it to, is to visualize the sim geometry. On the RBD Bullet Solver > Visualization > Show Collision Shape. That shows the geometry Bullet uses in the sim (after the collision padding and shrinking is applied) and you would notice in your case it isn't moving upwards like the render geometry is.