This is the way! I’m using it too with a piped instance. Also configured the RSS feed to a Slack channel with yattee://watch?v={videoId} which works awesome. When I press the link, when a new video is posted, it opens in Yattee. Basically I have a privacy focused YouTube with no ads and with subscriptions.
Funnily enough we got this message even in enterprise environment, where licenses were paid (and damn they are expensive). I’m convinced it’s just some process in the software itself. Have you tried looking through every process? Gotta find that one file that starts this pop-up and block it. I think that’s why your firewall rule didn’t help.