How can we fix Lemmy codebase so Wayback Machine archives not allowing replies to comments being viewed?

Nix@merv.news to Lemmy@lemmy.ml – 54 points –
i.imgur.com

Currently if you archive a Lemmy thread you can see all the posts and comments but all replies to all comments are hidden in a “X# more replies ->” which if clicked it tries opening it forever and the replies never load.

Edit: updated title and body so its more clear im asking about how lemmy can be updated so its easier for the wayback machine to also archive the replies to comments

Example: https://web.archive.org/web/20230801065606/https://lemmy.world/post/2398000

14

TL;DR: the code/servers could be changed to use SSR, but that's more expensive to run.

Lemmy is written more as a web app than as a traditional webpage. This means that the website sends a partial page plus the code+resources needed to finish building the page and the browser builds ("renders") the final page.

This has advantages in that the server can send less data over time, cache more of that data, and overall has to do less work, plus also makes the site feel more snappy for the user, because their browser only needs to download the data that's changed (instead of a whole new page).

The disadvantage is that the browser needs to be more powerful, and older/simpler browsers (like IE6, some text-only browsers and some web spiders) won't apply the extra work to finish the page off.

The normal solution is called "server-side rendering" (SSR) where the server renders the full page, sends that over, then also sends over the code+data needed to run things more dynamically ("hydrating" the static site into an app-like experience). This means the server has to do a lot of work, but is often the best of both worlds; search engines see the proper page (good for SEO) but users get to have a nice experience (once that longer initial load is complete, anyway).

There’s two archive-friendly solutions Lemmy could take to solve this tho:

  1. Detecting archive services and doing a full render for them
  2. Instead of using api requests and adding them to the DOM, having them link to a separate page that has the comment thread (like how Reddit does it)

I'd rather see the second option - having a JavaScript-free solution is definitely more resilient than trying to detect and whitelist every archive service. As long as it works for wget/curl then it works for almost everyone.

When you click more reply, the link takes an action to make a web service call, in the case of lemmy.world, to the lemmy.world server(s). If the archived link has not changed from when it was archived and today, and the content is still accessible (e.g. not deleted or not made private somehow), then the call to load the new data should work… unless the link is being prepended by the wayback machine’s servers (I’m not very familiar with how wayback machine actual works). If the former, it’s not loading due to lemmy.world not hosting it anymore; if the latter, it’s not loading because Wayback Machine doesn’t host website’s services that it archives. I hope that helped.

The screenshot is a lemmy thread. The replies on the lemmy thread are all still being hosted (its a post from a few hours ago)

Im asking how Lemmy’s codebase could be updated so its easier for wayback machine to properly archive threads including the replies to the comments.

The view all comments button, does it add a param to the query string? Could be worth a try.

I don’t see a vote all comments button (atleast on mobile)

When you click from a reply in your mailbox, you only see that post, you can click View all comments and it makes a request to the backend with max_depth=8. I thought there would be a way to use that parameter on the main post, but unfortunately, it doesn't seem to work like that right now. It's probably worth raising this as a bug on their issue tracker and seeing if they consider this.

theres now a github issue open on lemmy's github

There's a massive gap in your understanding of how computers work. The 10k character limit of posts on Lemmy will not be enough to explain it.

Having said that - go down that rabbit hole - it is absolutely amazing!

I dont really understand what your comment is implying lol. I’m asking if there’s something admins or ideally the devs can change in the codebase so its easier for the wayback machine to properly archive lemmy threads.

There likely is something that could be done. You could try raising this as an issue on their GitHub project page or wherever the devs tend to hang out.