Palworld server costs near $500K per month as network engineer is ordered to 'never let the service go down no matter what'

nanoUFO@sh.itjust.worksmod to Games@sh.itjust.works – 193 points –
Palworld server costs near $500K per month as network engineer is ordered to 'never let the service go down no matter what'
pcgamer.com
43

You are viewing a single comment

This is a software development business, which is a positively bananas trade no matter what’s getting written. And the smaller the business, the more hats network guys wear. We work with everything from the server app down to the coffee machine fueling the devs. And 100% uptime isn’t the most crazy demand I’ve heard. I’m sure Chujo is busier than a one-armed paper hanger with jock itch.

At least he’s got money to throw at his hosting company. Scaling up would have been much slower in the old days.

I'm not versed in videogame network infrastructures, but wouldn't be enough just having a load balancer and a couple of instances to ensure "100% uptime"? At least before all instances and the load balancer itself decide to join a suicidal pact, but more instances mean less chance of a critical event happening, no?

At a press level, sure, and the same for the average user. Legally speaking these numbers do have significance, though. Amazon Web Services (at least at one time) offer a guarantee of 99.99% uptime for their infrastructure. That 0.001% covers things like once a year outages that make the news. A 10000th of a year is actually a tangible amount of time and not even Amazon is confident enough to ignore it.

Depends on the cloud provider. AWS, as an example, have up to three “availability zones” within a single data center. If the customer needs HA, they are encouraged to run their applications in separate availability zones. It means different subnets within the VPC, redundant LBs spread across those zones, and more.

There is also probably DNS-based global load balancing across different data centers.

That’s just the hosting infrastructure. I’m sure Chujo works on the office LAN as well. He might wear the infosec hat also, which means he’s up to his eyeballs in firewall policy.

I don’t envy my brethren in software development orgs. Been there, done that, got that t-shirt long ago.