Why the data need to be copied in each instance?

winterwulf@lemm.ee to Fediverse@lemmy.ml – 14 points –

Could not it be just like a embedded? or something that works like that?

Seems a bit wasteful to have the same content stored in hundreds (potential thousands) of servers.

11

because it reduces traffic and, logistically, it would be insane to try and build a standard forum tree from say, the 50 different servers that might be involved for every single page load.

it would be crazy not to.

ill add, the content is almost pure text. highly compressible, designed for high speed transmission and storage. its not that big a deal. .. imaging is a whole other can o worms

I believe image proxy and caching is in the works, but no proper duplication. Same for vids.

Wow this is cool! Would work similarly to peertube?

No, peertube is way more complex and as the name suggests, peer to peer. This still relies entirely on the server.

And if the origin server goes down, then so does the content. As it stands today, it behaves like a CDN which is what you’d want for many reasons.

Imagine a post from a tiny instance running on the cheapest VPS going viral and now all of the top Lemmy instances are trying to embed it from the tiny instance. It would immediately blow up.

That would increase the barrier to entry massively, and all that to save a few GBs of disk space. It's small enough it wouldn't even fill up half of my phone's storage that I'm carrying in my pocket all the time.

What's wasteful is the MBs of JavaScript you load on most modern website infested with all the video ads.

What Lemmy does is also very effective because each instance acts as a cache for the other instances. If any given post is viewed more than once from a remote instance, it ends up cheaper in CPU and bandwidth.

On bigger instances, the cost of federation is very small compared to the load of serving the instance's users.

It also allows instances to have their own sorting algorithms, discovery algorithms, you name it. You have the data, you can crunch the numbers however you want for your users. You can develop your own spam filters and tools.

Even on my small 5 users instance, that's 4 users worth of traffic that never hits the bigger instances. Probably more because I refresh the home page a few times a day whenever I open up my app to scroll a bit.