Hello.

I'm trying to simulate a boat floating on a flat tank or a wave tank and I convert the boat to RBD hero object so it can float. I successfully made it floating like I want it but fluids starts to penetrate to the boat from its' bottom as soon as the boat sinks a little.
So my goal is to prevent that from happening and I did check all the collision settings and guides on RBD node inside dope network. I use both GEO and VDB collision sops and also tried transforming the object to a bigger one which helped a little but still the fluids are there.
Thanks.

the most important is to check the FLIP collision field, which has checkbox on FLIP Object/Guides/Visualization
(since that's what FLIP sees, so even if your boat VDB and RBD volume representation is looking good, it is ultimately resampled at the FLIP collision grid resolution)

if you se holes in this visualization and don't want to decrease FLIP particle separation you can also override collision separation independently

or if you are already happy with your sim just don't want to have water in the boat you can create vdb sdf of the whole enclosed boat shape and remove any particles within post sim (Group SOP by Bounding Volume) or directly in Particle Fluid Surface SOP (Collision Objects input)