How to rebuild underlay with new gazebo plugins

If I’ve added or changed my gazebo plugins, how do I rebuild the underlay since the files come from /opt/ros/humble

I don’t understand the question.

If you have built Gazebo from source e.g. in /opt/gazebo_garden/ per these instructions, and then made some changes e.g. in gz-sim/src/systems/<some system>/,
Then just recompile:

cd /opt/gazebo_garden
colcon build --merge-install
source /opt/gazebo_garden/install/setup.bash

Or, if you’re developing an own plugin in a separate workspace, refer to these instructions.

But it is not clear to me what you mean with “the files come from /opt/ros/humble/”.