That’s probably intended behavior, because most likely HDMI connector comes out of NoVidya card.
Good idea, but keep a note that unlike NoVidya one, Ryzen might not support some top-tier HDMI features like 8k 60 FPS and stuff like that, in case you need it