Lemmy v0.18.0 Release

Dessalines@lemmy.mlmod to Announcements@lemmy.ml – 545 points –
join-lemmy.org

What is Lemmy?

Lemmy is a self-hosted social link aggregation and discussion platform. It is completely free and open, and not controlled by any company. This means that there is no advertising, tracking, or secret algorithms. Content is organized into communities, so it is easy to subscribe to topics that you are interested in, and ignore others. Voting is used to bring the most interesting items to the top.

Major Changes

HTTP API instead of Websocket

Until now Lemmy-UI used websocket for all API requests. This has many disadvantages, like making the code harder to maintain, and causing live updates to the site which many users dislike. Most importantly, it requires keeping a connection open between server and client at all times, which causes increased load and makes scaling difficult. That's why we decided to rip out websocket entirely, and switch to HTTP instead. This change was made much more urgent by the sudden influx of new users. @CannotSleep420 and @dessalines have been working hard for the past weeks to implement this change in lemmy-ui.

HTTP on its own is already more lightweight than websocket. Additionally it also allows for caching of server responses which can decrease load on the database. Here is an experimental nginx config which enables response caching. Note that Lemmy doesn't send any cache-control headers yet, so there is a chance that private data gets cached and served to other users. Test carefully and use at your own risk.

Two-Factor Authentication

New support for two-factor authentication. Use an app like andOTP or Authenticator Pro to store a secret for your account. This secret needs to be entered every time you login. It ensures that an attacker can't access your account with the password alone.

Custom Emojis

Instance admins can add different images as emojis which can be referenced by users when posting.

Other changes

Progressive Web App

Lemmy's web client can now be installed on browsers that support PWAs, both on desktop and mobile. It will use an instance's icon and name for the app if they are set, making it look like a given instance is an app.

Note for desktop Firefox users: the desktop version of Firefox does not have built in support for PWAs. If you would like to use a Lemmy instance as a PWA, use use this extension.

Error Pages

Lemmy's web client now has error pages that include resources to use if the problem persists. This should be much less jarring for users than displaying a white screen with the text "404 error message here".

Route Changes

Pages that took arguments in the route now take query parameters instead. For example, a link to lemmy.ml's home page with a few options used to look like this:

https://lemmy.ml/home/data_type/Post/listing_type/All/sort/Active/page/1

The new route would look like this:

https://lemmy.ml?listingType=All

Note that you now only have to specify parameters you want instead of all of them.

Searchable select redesign

The searchable selects, such as those used on the search page, have a new look and feel. No more inexplicable green selects when using the lightly themes!

Share button

Posts on the web client now have a share button on supported browsers. This can be used to share posts to other applications quickly and easily.

Lemmy-UI Overall look and feel

lemmy-ui is now upgraded to bootstrap 5, and every component is now much cleaner.

Special thanks to sleepless, alectrocute, jsit, and many others for their great work on improving and re-organizing lemmy-ui.

Database optimizations

Special thanks to johanndt, for suggesting improvements to Lemmy's database queries. Some of these suggestions have already been implemented, and more are on the way.

Query speed is Lemmy's main performance bottleneck, so we really appreciate any help database experts can provide.

Captchas

Captchas are not available in this version, as they need to be reimplemented in a different way. They will be back in 0.18.1, so wait with upgrading if you rely on them.

Upgrade instructions

Follow the upgrade instructions for ansible or docker.

If you need help with the upgrade, you can ask in our support forum or on the Matrix Chat.

Support development

We (@dessalines and @nutomic) have been working full-time on Lemmy for almost three years. This is largely thanks to support from NLnet foundation.

If you like using Lemmy, and want to make sure that we will always be available to work full time building it, consider donating to support its development. No one likes recurring donations, but they've proven to be the only way that open-source software like Lemmy can stay independent and alive.

143

I apologize for the stability issues everyone.

People found an exploit and are using it to DDOS several lemmy instances.

Growing pains. You got popular, now you've got a target on your back.

That's what happens when you start getting popular. Lemmy will survive it :D

So Reddit has decided to fight back, eh?

Not sure, it is a coordinated DDOS attack tho. I welcome them doing these, because it only strengthens lemmy, and helps us find bugs.

How is your personal life btw? I was feeling bad for you guys because I imagine dealing with the (welcome) explosion of your project must take all your time.. Are you studying?

Its good, thanks for asking <3 . I do have a high-score of github notifications ( > 1200 ), and hundreds of people all trying to get my attention. I've settled into a rhythm now tho where I just work on what I think is important, take lots of breaks, and try to keep a regular schedule.

As in any project the few loud voices can be the most damaging. Please isolate yourself from that and understand the quiet majority greatly appreciates what your doing and recognizes the pressure you probably feel lately!

That's excellent. What do you do for a living, by the way? Because Lemmy can't be your day job, obviously!

3 more...
3 more...
3 more...

exploit [โ€ฆ] coordinated [โ€ฆ] bugs

Hilariously wrong on all three points. It's a Python script I wrote in under a minute making identical requests to the same thread fewer than 10 times a second; Lemmy's complete lack of caching isn't a "bug". That this extremely simple vector instantly brought down even the high-spec Lemmy.World instance with a low volume of requests is sad, and it's tiring to see Lemmy zealots spam my communities advertising this tankie spaghetti code jank as an alleged reddit "alternative" when it topples from this. The multiple attack vectors I've tested make it abundantly clear that Lemmy's team has been incapable of performing even the most basic possible load testing.

Won't be responding to or reading any replies, don't care.
BTW, ltree is a horrifically bad solution for comment trees, and any of the self-proclaimed "Postgres professionals" in these threads should be able to explain why if they have even the slightest bit of RDBMS knowledge.

3 more...

those fat fucks

3 more...
3 more...

Another addition that wasn't covered in the release notes, these will now automatically be linked to your local instance without having to do anything:

Edit: Forgot about kbin support

The markup is rendered as links when a valid format is detected without modifying the underlying text:

There was like 200 comments worth of discussion on this and it seems like it was implemented without much fanfare. What gives? This is one of the biggest gripes I had early on. Iโ€™m gonna shout this from the roof tops

I saw those giant threads where it was being discussed in circles and decided to take a crack at implementing it myself. Turns out it was not that difficult, I think I was just the first developer who found the threads and took action. Everyone else was focused on getting bug fixes into this release.

Updated my comment, I forgot to mention it also supports the kbin /m/ format for communities

Query speed is Lemmyโ€™s main performance bottleneck, so we really appreciate any help database experts can provide.

I have been pleading that Lemmy server operators install pg_stat_statements extension and share metrics from PostgreSQL. https://lemmy.ml/post/1361757 - a restart of PostgreSQL server is required for the extension to be installed. I suggest this be part of 0.18 upgrade. Thank you.

I think this kind of analysis can help a lot, sadly I don't see anyone interacting on your post.

Please do keep pushing for it and writing down your discoveries and explanations might help others understand the importance.

Howto with real world data (like you are doing) is helping and is quite interesting.

Does having that extension enabled affect performance at all? Just curious

Not original commenter. No performance impact that I know of but it takes a bit of shared memory (hence the restart).

My favourite feature: The collapse comment icon is on the left of the username now!

My favorite feature is that upvotes now have a spinner that lets you know they actually got processed by the server. On 0.17 I would click an upvote and it would sometimes flash between on and off state or not be very obvious that it actually went through.

i think my favorite feature is that errors while posting no longer silently fail. however, since they do show in a small popup box with a short timer, if i'm not paying attention, i'll still miss them, and it'll look like it failed silently.

Such a big improvement!

Not sure what changed but the UX feels much more responsive than it did a couple days ago. Great work!

All collapsed comments get uncollapsed when I upvote comments (lemmy.ml on Firefox Nightly for Android)

This is a known issue on the web ui - we're looking into it.

Thanks for linking! It is 'closed as completed', but I still get it every time?

When a GitHub issue is closed as complete it means someone has fixed the bug in the source code (or the developers have decided it's not a bug). It doesn't necessarily mean the fix has been shipped in a new version or that the server you're connecting through has been updated to a version with the fix. In this case, they haven't shipped a new version with this fix in it. I'm not sure what the delivery cadence will be but I assume a new version will go out in the next week or so and then once your server admin upgrades their server the problem will be fixed for you.

Great. I saw that Lemmy.ml been on v.0.18.0 for more than a day. I love the subtle changes they made. Keep up the good work ! Itโ€™s becoming to be my favourite site day by day

Thanks! I'm super excited about all the contributions making this site and its UI better.

That's the difference between an open source project like this and privatized software like reddit; we can add features we want directly.

Holy fuck man! As someone who wants to be a Web Dev, itโ€™s really exciting to see open source projects where people develop without any financial incentives! Itโ€™s mind boggling how much this community is getting together to improve the experience of everyone!

Thank you for your hard work โค๏ธโ€

1 more...
1 more...

Wow, can't wait until my instance upgrades. Great work! I don't think I will miss that alien site.

yay! thank you all!

i have made a not-so-quick-but-dirty Dockerfile to build on arm64

FROM rust:1.70.0
WORKDIR /app

COPY . .

RUN echo "pub const VERSION: &str = \"$(git describe --tag)\";" > "crates/utils/src/version.rs"
RUN cargo build --release

RUN apt update
RUN apt -y install libpq5
RUN cp /app/target/release/lemmy_server /app/lemmy

CMD ["/app/lemmy"]

later I am planning to improve it a bit, to make the image smaller if i can

Proposals:
Bring the .16 themes back
Make the site denser and more compact
Remove flourishy animations

Agreed, I know there are workarounds with custom css extensions but it's a bit of a pain on mobile. A theme that's oled black and compact would be fantastic.

andOTP is no longer maintained, so it's probably best to not suggest it!

Oh, that's sad to hear. It's such a great app! Surely it can be used a little longer though. Unmaintained does not equal buggy or insecure.

It's been over a year since the last commit and it's security software, so being unmaintained is dangerous. And it's not like there's no other options. My transition to Aegis was very smooth.

Bitwarden has support for 2FA tokens

Personally, I like to keeping my password manager and OTP separate but I do love Bitwarden!

super reasonable. The 'I probbbbbably shouldn't be doing this' thought has occured

So, the auto update is no longer active for the feeds. That is a good thing! However, I think Lemmy now needs a refresh button.

When I move away from my defaults (eg to check "new"), it isn't easy to update the new feed (when I refresh I need to reselect new)

how is that a good thing?

If you were reading something the page would jump up by itself.

i'd rather have that than have to constantly refresh sites

You're very much in the minority, then; it's absolutely terrible from a usability perspective.

Good that websockets are going. Lemmy sucks up battery like nobody's business and for ultimately a negative payout

The auto update was awful for usability. Hate my page randomly jumping around when I'm trying to read.

it was good for usability actually, i wish more sites would implement that (i might just go on GH and fork .16 so that I can turn .16 into a client that never gets new versions)

By Odin's beard!!! That's an absolute LIST of impressive and impactful enhancements!
I feel like I owe you several drinks after reading all of that!

Iโ€™ve been noticing significantly less errors when using the API. Thanks for all the hard work I know you guys are putting in. Much appreciated to both of you and the other guys contributing to the project.

Hot diggity, these new changes are amazing! I can definitely see the difference in performance on lemmy.ml. Feed content sorting is also way better without the live updates and bugs with Hot and Active. Great work!

Thanks! I'm really glad we could get this out before the end of the month (when reddit kills 3rd party apps).

Also now we should be able to do a lot more frequent bugfix releases.

Wow lemmy.ml feels a lot faster now. Compared to lemm.ee on 0.17.4 the difference is huge and lemm.ee was already very snappy. Excellent work!

Damn, big update! I see a lot of people are contributing, that's really the biggest gift from Reddit's poor decisions (well, presuming contributions did increase, which is a fair assumption).

That's why I love open source software, it's truly a community effort!

Has anyone gotten 2FA to work? I tried it, made a qr code locally, scanned it in authy, and wasn't able to get my instance to accept the code. It would say 'incorrect_totp token'. .. Also can the webui generate the totp qr code in the future?

1 more...

Hello Devs. I'm not sure if you've noticed but some times the are discrepencies between different instances, for example, updated Display Names doesn't federate, profile descriptioms, edited post/comments, deleted post/comments are not reflected in some instances. Does this update fix these issues? If not, when will these problems be fixed? And will the fix also retroactively sync changed that were made before the update has been applied?

Edit: fixed spelling

I've been noticing on lemmy.ml that clicking Next in the news feed pagination seems to cause it to scroll to a random point instead of back to the top of the feed, at least on mobile.

Is this a known bug?

I have the same issue but itโ€™s only because my Chrome on iOS hard-cached 0.18-rc instead of 0.18. The official 0.18 release has this bug fixed (I fixed it).

Try clearing your site data on your mobile browser or just wait for the cache to expire I guess. ๐Ÿค”

Will there be an ARM docker? I was hoping to start an instance on my RPi thatโ€™s laying around.

Thank you

There's a PR in draft to add these, but they still need some work.

Great, it seems that there's way less error message on Jerboa now with this update.

Painless upgrade for docker setup, congrats on the huge release! After using for a while its definitely faster!

The image I uploaded got rotated on lemmy:

https://lemmy.ml/post/1466468

Pictrs probably didn't honor EXIF rotation metadata, or strip them entirely for privacy reason.

It was exactly that, I removed it and now it's correct, thanks

Thank you for your hard work! I'm a big fan of both systems. What's the alternative for systems relying on captchas keeping out bots though?

nice, I really like the new UI changes!

So I'm trying to get a local instance of Lemmy 0.18 running on my LAN with docker. I don't have a domain name (yet) and would not be able host lemmy as a Proxmox Container primarily to get acquainted with hosting and administering an instance before eventually hosting an instance on a VPS. Has anyone successful hosted lemmy on their LAN? How did you go about setting it up?

I haven't tried it yet, but I'm sure the folks on the support forum and matrix channel (listed under upgrade instructions on the original post) would love to help you, especially if you're willing to document your experience for others.

The change was so drastic and I'm still finding little changes here and there. Amazing work, y'all! <3

Did the testing go smoothly?

Found lots of bugs in the process that were fixed, yep.

Hi,

I've tried to install Lemmy using the Docker image and it keeps failing. There seems to be typos and other errors in the configuration files. It's really frustrating.

Compared to Lemmy, installing Mastodon is dead easy.

I've never worked with Ansible before but should I be using the Ansible Playbook method instead?

WOOO!!!

Sounds like a great collection of needed changes, and a few QoL thrown in for good measure.

Thank you devs! โค๏ธโค๏ธโค๏ธ

Regarding

Note that you now only have to specify parameters you want instead of all of them.

That doesn't seem to apply to the search page. For example, putting https://lemmy.ml/search?q=test&sort=New will lead to The server returned this error: couldnt_find_object. This may be useful for admins and developers to diagnose and fix the error

Copying the URL of a search (without stripping parameters) and opening it in a new tab so that it doesn't contains any referers also returns the same error.

Also great work and congrats on the release ๐Ÿ™‚

Absolutely phenomenal work! I love how fast the community is reacting to the scale changes. So many great things are in store for this platform.

What donation method gets the most of the funds to you guys? Crypto?

I just want as much of it going to you as able if I choose a recurring monthly donation.

Thank you SO MUCH for the work you're doing to help us all!

It broke Jerboa. I couldn't get jerboa to change to view all or change sort. On my lemmy.ml or lemmy.world accounts. So I assumed the jerboa v0.0.36 update broke it. I Uninstalled jerboa and tried to sideload 0.0.35 but no matter what apk I download it acts like 0.0.36 and says the server is out of date and won't log in. Now I can't log in to lemmy.world because they are on 0.0.17 (since the update broke their server).

Thanks for the work, but rolling out two updates simultaneously makes diagnosing issues impossible.

I am trying to enable 2FA. Tapping the link opens Microsoft Authenticator but the new and security key fields are empty. How do I fix this?

Awesome update.

On iOS, after adding lemmy.ml to the homescreen as a PWA from Safari, I can't log in because the keyboard doesn't pop up when tapping the username or password fields.

this is all great and all but how can i stop this post from being at the top of my feed every single day.

And still no RTL support.

Did you submit a PR yet?

Its there almost a year ago on github and we spoke about this a lot.