[Done] Brief outage for Pictrs upgrade today Oct 2nd 18:30 CEST is finished

lwadmin@lemmy.worldmod to Lemmy.World Announcements@lemmy.world – 157 points –

Edit : OK so the outage was less brief than expected. The upgrade of the 3GB pictrs database took over 1 hour, and the version 0.4 database is now 14.5 GB... But anyway, it seems to be working alright now!

We will upgrade pictrs today, from 0.3.1 to 0.4.4. This will enable us to switch to S3 storage later on, and is needed for more anti-CSAM tools. Outage should only involve picture uploads, and should be brief.

26

Awesome that pict-rs has postgres and full s3 support now. Lemmy is getting really scalable!

I think the postgres support will come in 0.5?

How will the postgres support look like? They are not storing actual images in the database I hope?

No. Just like now , they store the metadata there, the images are on disk or S3

Ah, right. I was aware of the ability to use S3 for 0.4, but this post made me go check out Asonix's Forgejo. The fact that you can get rid of sled in 0.5 and go fully stateless/scalable is a game changer.

Yeah we can upload pictures again!

OK, the pictrs database upgrade is taking it's time... please wait.. ;-)

pictrs_1     | {"timestamp":"2023-10-02T16:31:44.746467Z","level":"WARN","fields":{"message":"new"},"target":"pict_rs::repo","span":{"name":"Migrating Database from 0.3 layout to 0.4 layout"},"spans":[]}

how's it faring? 👀

It's building a 0.4 database, which is already twice as big as the 0.3 one

It's grown from 3.1 GB to over 8GB and still running...

thats a lot of pictures

That's not the pictures... The picture take up 1.2 TB.. it's just the database with the metadata about the pictures. (11.8 GB now...)

I'm curious, how much storage in total is needed for a Lemmy instance like this one currently?

1.2 TB for pictures and 100GB for Postgres

For reference I run a pretty small instance. I'm at 34GB for the main database, and about 120GB of media. (Doesn't count backups)

My instance is much, much smaller though. A few hundred users, and probably only a couple dozen actually active.

I know. Giant database = Giant × 10² amount of Pictures

Are you manually updating Pictrs or is this included in the new lemmy update?

Newest update defaults to 0.4, but the latest version is 0.4.4. I don't think there's a significant difference.

But it looks like ruud stayed on 0.3.x until now because there's a pretty serious migration between 0.3.x and 0.4.x