Yeah, Microsoft Remote Desktop is relatively slow but works for most workflows. For more demanding/responsive stuff with higher framerates (games), try Parsec (https://parsec.app/), it's great. It has controller pass-through as well, at least for sony. I'm not sure about other brands.
Too salty. Drink more water.
If you don't run new builds contantly, that means you aren't doing any actual work if you are on the engineering side.
This is not true. There are higher-end engineering positions that orchestrate everything for the other engineers, amongst other things.
I'll be explicit in regards to my other comment about remote working for others without gamedev experience. Compiling/generating/deploying game-builds is done locally at the studio, but the process is controlled remotely by the game devs.
Yes, you can use Parsec for that.
The devkit software (has its own proprietary video streaming/control) would be on the on-site PC you are controlling remotely, using Parsec from anywhere, which would then control the PS4. Or you could use VPN from anywhere with the devkit software and connect remotely to the devkit directly.
The first scenario is ideal because you're only using Parsec (or RDP) from your personal device, whereas the second scenario would require installing the devkit software on your personal PC, which in my opinion it's not good to mix work/personal hardware. Your work might instead provide a computer to remote with, but I would rather keep work devices outside of my home network.
Parsec is very smooth and responsive, and it takes only 50mbps or less IIRC.
Dev kits are accessed remotely. High-end PC 's are accessed remotely. Everything is done remotely. Hardware stays on-site. Source? Am game dev who works remotely full-time, using 4K streaming.