Lemmy posts are starting to pop up on search results with Google (+ other search engines)

Otter@lemmy.ca to Fediverse@lemmy.world – 1326 points –
lemmy.ca

To see the original discussion, you can see this thread: https://lemmy.ca/post/8488573

To open the post on your instance you can go to !lemmybewholesome@lemmy.world and see the recent top posts, or use an app/frontend/ browser extension (ex. !instance_assistant@lemmy.ca)

Alternatively, here is the screenshot from the post.

I also wanted to share this tip for how you can filter for Lemmy posts when searching:

  • Search using site:home_instance. So if I wanted to find recommended phones, I could go site:lemmy.ca recommended phones. Since every instance has its own collection of posts, you will be getting results from all over Lemmy. The limitation is that you won't see content from instances that aren't federated with yours, but you probably didn't want to see that stuff anyway since you picked your instance for a reason. You can also put any instance into the search if you wanted different results.

Question to everyone, what does Lemmy need to make it easier for people to find content? What are the implications of the Fediverse on how people might find content in the future?

One thing is that people are more likely to get posts from the larger instances, likely because more people are linking to them and opening those links? Another thought was the common complaint about how our post links aren't community specific. While I can search for posts using the method above, I can't search within a specific community like I can with Reddit (ex. I can't search site:lemmy.ca/c/Vancouver recommended restaurants

EDIT: The issues for it are here, looks like the devs are good with it now and someone just needs to implement it:

83

We did it, Reddit

Watch spez squirm every time lemmy improves

spez wants to remove Reddit from Google actively, so at least for this specifically I sincerely doubt he minds. Of course, the main reason he wants thatis only that he wants to inflate Reddit's perceived investor value, so not sure what that even helps. But eh.

Reddit is talking about hiding Reddit from Google. I hope they do that because it will let Lemmy start to replace Reddit as the go to source for non-SEO, real-human answers.

Except each instance has its own URL meaning ranking for ANYTHING is extremely hard since each domain’s rank will always be weak in the sea of others. Each is even being penalised by the algorithm if there are duplicate content mirrored between different URLs. It’s the weakness of the fediverse if we are to follow how search engines have worked in the last decades. Maybe it will lead to new search engines (I hope so) but right now it is not going to work well to replace for example Reddit … or rank well in general at all.

I wonder if there’s a way around this that we can create, instead of doing nothing or hoping google adapts.

Like a dummy instance that catalogues everything on all instances (but also links to the original posts) for the purpose of showing up on google search.

Since this instance isn’t for posting but for search engine indexing, there may be some otherwise undesirable micro-optimizations that can help improve its chances of showing up.

Yes, this would be possible (and not too hard technically either). But all instances would have to agree to link this instance as canonical.

You'd also want to add a feature where you can set you home instance where this canonical instance would redirect you (perhaps even automatically). Home Assistant does something like that.

What pisses me most about Lemmy is that each instance has its own post IDs which means that crosslinking and switching instances based purely on URLs is impossible.

IMO posts should have random GUIDs for IDs; that would help a ton with these kinds of issues. It'd then be trivial for Google to detect same content (if they wish) this way

At first I was thinking a GUID might be impossible because of federation, but a simple implementation might be to use the post ID from instance the community lives on

So something like

The point of random GUIDs is that there are so many that it's effectively impossible to generate duplicates just by random chance. They'd be perfect for this.

The initial instance picks it, and then the federated instances use it.

Hi there! Looks like you linked to a Lemmy community using a URL instead of its name, which doesn't work well for people on different instances. Try fixing it like this: !cats@lemmy.ca, !cats@lemmy.ca/1234444

This bot needs a "that was intentional, delete your comment" option

Would another option be like having every post have some kind of anchor of invisible text or something with "lemmy" so when I search for "best washing machine lemmy" it'll show all posts across instances or something. Idk if that's how SEO works entirely though.

Each post refers to the poster's home domain as the canonical URL, regardless of which instance you're viewing it on specifically to avoid duplication SEO concerns

Thanks, I didn’t know that and never bothered to look. But then you still have the (SEO) issue of all the domains vs one in the case of Reddit or Quora or Stack Overflow. But yeah, a few very large Lemmy instances will probably start to rank well once they have enough good content.

Didn't know that's the case, that's neat though it doesn't solve the redirect back to your home instance.

It'll also probably lead to centralization because if you're more likely to find a particular instance through search and decide to join Lemmy you're probably going to do so on that instance.

1 more...

If the Fediverse gets big enough search engines wil probably optimize for it e.g. prioritizing the instance of the community...

One of the main reasons reddit mega turned to shit was due to far too many people joining and using it, granted this is due to mobile phones but is it really worth it to attract more and more people? These instances are run by average people not corps with money they can easily collapse under tuw burden of to many

This anti user attitude is so lame. It's some real hipster nonsense.

It's really not, when it's been proven time and time again that more people doing something ruin that something it should become obvious that lots and lots of people is a detriment

The reddit front page is a classic example of that, the general state of the internet proves it too, beaches/music festivles are both great examples too

That's not even thinking about the cost of everything and corporate meddling either

It is so fascinating to me there are people out there who really mistake their own subjective experiences as iron objective truths. Just a complete lack of self awareness. It's like how babies lack object permanence.

Its been proven? Really. Do tell.

Yes like all the people using rice for food has totslly ruined rice

Dont even get me started on indoor plumbing. Was so much better before it got popular.

The cool thing about the fediverse is that if grows to much where and the moderation turns to crap on some instances, you can always defederate from those problem instances and avoid the trouble makers entirely. Instance admins can always turn off sign ups when they feel like they have reached their limits for moderation as well. And users can always self host their own instances and only let the people they trust sign up. The fediverse is more resilient than traditional corporate run social media

Reddit is not moderated by paid corporate employees. It's all volunteer labor.

Moderation is not the issue, the sheer cost to host the tremendous amount of data is very likely to be a reason an instance goes down, thats what I'm getting at

1 more...

I use duckduckgo and lately have been adding Lemmy at the beginning (like I did Reddit) and it really works. Search results come back if there is a discussion going on about the topic.

I've not tinkered around with it too much yet, but how has your experience been with actually viewing the results? I would imagine that most results are statically likely to be for an instance other than the one you have an account on, requiring a few extra steps to load the post in your home instance if you wanted to vote/comment on it.

This is one of those UX things that I think is still holding Lemmy back from more mainstream adoption, imo.

You can make that a bit less painful with browser extensions. 'Instance Assistant for Lemmy & kbin', for example, will always show a button to open a page in your home server. Not sure if there's anything exactly like it here yet but FediAct for Mastodon lets you like/boost/follow/comment on other servers directly, more or less eliminating the other-servers problem.

It's hit and miss most of the time. I just tried Lemmy memes and it redirects to lemmy.ml and the memes community.

Then I tried Lemmy how do I wash my kitten and there was an answer. Again for Lemmy ml. I think it depends on whether there is a post answering the question and if people commented on it.

Swag. The more we show up in search, the more people will be asking "what the heck is Lemmy?" Some of 'em will join.

Well then. Here. We. Go.

I have been regularly sharing shit with friends that I see in Lemmy, and they always said to me why my links always have weird names and domains and shit... so I proceed to explain and we get to nowhere.

Anyway this is people that weren't even into Reddit, so that people are the harder to get, IMHO.

It really was silly of Lemmy to not have community specific links, it's even more confusing that way. Now it's just a bunch of

{weird-domain}/post/{number}

You never know what it is unless you have link previews

Yeah hopefully its possible to change that without breaking anything

without breaking anything

That wouldn't be the Lemmy experience, no hating here 😀

I’ve had success describing it as “imagine you owned the server your Facebook info was stored on but could still interact with all other Facebook servers”. It’s a little simplified, but it usually gets the point across.

Guys... I finally understood the Fediverse! I made a Threads account!!!! /s

Sounds like a "them" problem, you keep doing what you're doing. Maybe they'll eventually get it, maybe not. Unless you give up sharing content with them entirely, of course, but that's your choice.

It would help a lot more with SEO-friendly URLs. https://lemmy.ml/post/7417691 is not very SEO friendly at all.

would the suggestion be to have the post title in the URL?

That would be ideal, you'd have some sort of slugged title in the URL, yeah.

Good! I love it here and I think others will too :)

I have a question, I have posted with my lemmy.zip acccount to a community hosted on feddit.ro, and when I search the title only lemmy.world appears, even tho i didnt post it there, any idea why?

It means many people are using lemmy.world instance to view your post instead of lemmy.zip or feddit.ro.

Every instance stores what are essentially copies of everything it's users are subscribed to. So when you post "to" feddit.ro, you're posting to the copy on lemmy.zip. Similarly, Lemmy.world has their own copy (that for whatever reason is ranking higher in search) because someone there is presumeably subscribed to the community.

Adding a bit more to the other comments

The post should exist on all 3 instances (as well as any other instance federated with feddit.ro). However lemmy.world is bigger and so more people are likely linking to /from that instance, which mean Google is indexing it with higher priority.

The other instances should probably show up over time?

Hello from kbin... (federated here too)
But this poses an interesting dilemma for Google, potentially to top 100 results could end up just being the same post observed on many Lemmy and kbin instances.

I think Google tries to prevent that, which is interesting because I don't know how that will work

That's cool. I seem to have issues finding this post, or the top post of the day (https://sh.itjust.works/post/8365139) by searching: site:sh.itjust.works After watching the 2nd episode of 11th season of Futurama And I'm not getting the correct result. Am I doing it right, but there's something else affecting the results (top of day for lemmy isn't as popular as needed to show up on google, bing, nor DDG)? or am I making a mistake somewhere?

I don't think the instance is the issue like the other comment is suggesting (although I don't quite understand the specifics of how it works). I'm playing around with it myself right now

If I search site:sh.itjust.works Hello and set it to 24 hours, I do see posts. So timing should be ok too

Update: So I think what's happening is that the post needs to go through a few stages when it's on a different instance

  1. Someone posts on a foreign instance (https://foreign.example.com/post/123444
  2. Someone on your instance views the post, which generates a link on your instance for that post (ex. https://example.com/post/135799)
  3. Google indexes your home instance and grabs that post

So we're probably between steps 2 and 3 right now?

Looks like it's appearing now! Just needed some time I guess

It's on lemmy.ca, which is a different instance than sh.itjust.works, surprisingly. That's probably your mistake.

Did the same thing, now the linked thread is shown and not the discution

One big issue that Lemmy has because it's a distributed service is the dilution of results.

For example, there is only one Reddit domain (that people use to access the service) but there are hundreds/thousands of Lemmy domains and the dilution will continue to increase as Lemmy's popularity increases. It's either that or there will only be a couple of Lemmy instances that will dominate all of Lemmy.