Self hosting voice assistant

Hi all, looking for my next major project/frustration. I've been forcing myself to learn the new AI tools and I think I'm ready for the next step. I'm familiar with image generation and I dabbled in a bit of chat bot stuff, but I think I'm ready.

I've read a few blogs but I want to find something that could work with my existing setup. My dream setup would be:

A voice assistant that runs locally, preferably dockerized, backup linux, and final option would be Windows, that can run a decent model and preferably let me train a custom voice for it.

I currently have:

• Home Assistant set up already, I've seen the OpenAI integrations but would like to migrate off of those
• Google Minis laying around, I'm willing to sacrifice one of them if it means I can use my own stuff
• Spare 1650GTX GPU, I know not the best but hopefully enough to get it off the ground before deciding to go in on a larger GPU that would be dedicated to this

Needs/wants/nice to haves would be:

• Basic chat functionality, what's the weather like
• Play music from my plex or jellyfin server
• HA integrations so I could say stuff like "Turn off the lights"

Sorry for dumping all of this, like I said I've seen blog posts around, some are doing parts of this, but I wonder if anyone has done something like this. I'm sure people have tried. Guides, jumping off points, even githubs/projects you know of would be helpful.

Thanks all!