Software management for Windows Server

I have 15 VM's running for clients and I'm looking for a way to keep the tools up to date without having to connect to each server and do it manually. A few examples are WinDirStat, Firefox, SSMS, Filelocator, etc.

We have expanded recently and I'm at the limits of doing this manually. These servers are not domain joined and are in separate virtual networks.