You're running a 405b param model on 24gb of VRAM, no shit it's not gonna work
--lowvram
Yeah I'm sure that's how they got it to run at all lol, luckily they've fixed a lot of the issues with earlier versions of model runners, I had blue screen running 7b models back then. One of this size might've literally started a fire on my computer
You're running a 405b param model on 24gb of VRAM, no shit it's not gonna work
--lowvram
Yeah I'm sure that's how they got it to run at all lol, luckily they've fixed a lot of the issues with earlier versions of model runners, I had blue screen running 7b models back then. One of this size might've literally started a fire on my computer