The Man Who Killed Google Search
Edward Zitron has been reading all of google's internal emails that have been released as evidence in the DOJ's antitrust case against google.
This is the story of how Google Search died, and the people responsible for killing it.
The story begins on February 5th 2019, when Ben Gomes, Google’s head of search, had a problem. Jerry Dischler, then the VP and General Manager of Ads at Google, and Shiv Venkataraman, then the VP of Engineering, Search and Ads on Google properties, had called a “code yellow” for search revenue due to, and I quote, “steady weakness in the daily numbers” and a likeliness that it would end the quarter significantly behind.
HackerNews thread: https://news.ycombinator.com/item?id=40133976
MetaFilter thread: https://www.metafilter.com/203456/The-core-query-softness-continues-without-mitigation
Really weird to stumble onto a blog written by someone you vaguely knew in college almost 20 years ago... Anyway, nice job, Ed Zitron.
He has a podcast too called Better Offline. Just started it up a few months ago.
And has RSS, nice.
I know the feeling. I went to middle school with Hillel Wayne
I'd never heard of Ed Zitron but this is the second good blog I've seen by them this week.
Wonder how this isn’t bigger news. The story is shocking, but absolutely confirms my gut feeling that google search has gone to shit in the last few years, and was fine before
Prabhakar Raghavan and the McKinsey-inspired management class forced the real tech people out and shit all over the search engine intentionally to squeeze out more short-term profits. Google: An Enshittification Tale
Honestly, I'm surprised it took this long for Google to kill its main product over profits. This is in every big company's textbook.
anyone can tldr?
Google internal politics ousted the last of the OG Google guys and replaced him with the same person who killed Yahoo, Prabhakar Raghavan.
The general consensus is that all of the changes to Google since 2019 were driven by profit instead of trying to find things, like a search engine should. And those decisions were spearheaded by Prabhakar Raghavan, who used the training of a data scientist to run Google into the ground for short term financial gain. Sundae Prichai hired Prabhakar Raghavan directly and then promoted him from Head of Ads to Head of Search after firing the guy who had been helping guide Google Search since 1999.
Pulled a Boeing
thx.
Proof that everyone can fail upward.
I haven't read the article at all (I plan to) but this basically confirms everything I have assumed as an outsider for years now.
In this rare case, I would totally suggest you read the article. It has the perfect amount of humor mixed with shocking facts (revealed via email evidence from the Google antitrust case) and it wraps it all up in a way that's easy to understand.
Hey, I follow up your suggestion - come back and read the article. No doubt, a very engaging read. Thx.
Based on this particular comment chain and your decision to come back and read the article, i decided to read it as well. Very engaging, indeed! Learned quite a bit, def worth the time. I even subscribed to Ed's newsletter, lol
A man named Raghavan has been taken on as a major operational manager for yahoo, ibm, etc. Seems his direction of their operations lines up with a sudden collapse in quality in the areas he was at. Regardless everyone seems to discuss how he is one of the best researchers in field. The dark design, and other issues, google has been seeing an increase in, for years, is basically his direction and, while he isn't the CEO, he basically runs google.
thx.
The guy who is now in charge of Google’s search division is the guy who ran Yahoo’s search division into the ground about 10-15 years ago
Google should have improved the search with more powerful tools instead of chasing numbers and greed.
But that's not how capitalism works!
As an programmer, I want to think out loud about possible technical solutions.
I would have kept the understandable / hand-made algorithm as the core of search results. If you want to do fancy machine learning, do it on the periphery and we can include the machine output in our algorithm and weight its importance by hand. This would allow us to back out of the decision, because we could lower the weight of the machine learning output as needed.
It sounds like Google jumped strait to including the machine learning in the core algorithm though, and now with a decade of complexity in the core algorithm they are no longer able to go back without huge effort.
In general, it's important to consider "is this a decision we can easily back out of?".
Amazon (and I'm sure others) refers to this as a two way door. Good rollouts minimize impact and can be undone easily.
Exactly, and that's something my company is aggressively moving toward, even though our userbase is nothing like Google's. It's just good engineering to be able to rapidly undo an unfavorable rollout.
Google's operations are absolutely built around the idea of easy rollback. Their products, and the their entire product ecosystem, are not.
Yeah, they seem to do "easy roll-foward." Any service is subject to replacement, given a sufficiently motivated project manager. So if there's a problem in deployment, they just replace the whole thing.
Did you read the piece? This isn't a software issue, it got worse by design to push even more ads and stop suppressing the ad-ridden fake sites too.
I keep wondering who’s gonna hire away Ben Gomes after they read this.
Jeeves?
Duck Duck Gomes
It overall seems like a good article but this is why I kind of hate Ed Zirtron's reporting:
Overall the reporting is interesting, but weird comments like this show his naked disdain for everyone and everything in the tech industry which does not make him a particularly trustworthy source.
Like "oh my god, how dare a company choose an arbitrary alert system based on a quirky influential engineer's practices, what crazy psychos!"
If he sees the code yellow tank top thing as some crazy ridiculous thing that no company should do, then I can't really trust his interpretation of the rest of the emails and documents etc.
Later in the article, he boils everything down to literally "Heroes vs Villains", and maybe in this case both of them are archetypal representations of those roles, but based on his appearances on behind the bastards it feels more like he always needs to boil everything down to black and white, good vs evil, bastard vs non bastard, with nothing in between, which again, makes it hard to trust his overall interpretations of what he's read.
It seems clear to me that he hates the people that are ruining the tech industry, ripping off customers, and pumping out shitty projects for short term stuck pumps, and he takes every opportunity to shit on those people and point out their idiosyncrasies. That's pretty much every tech CEO these days.
It's also pretty clear to me that he believes in the promise of the industry, and thinks that workers deserve better than the people that they work for.
Hunter S Thompson wrote a scathing eulogy for Richard Nixon, which I think is relevant here:
"Some people will say that words like scum and rotten are wrong for Objective Journalism -- which is true, but they miss the point. It was the built-in blind spots of the Objective rules and dogma that allowed Nixon to slither into the White House in the first place. He looked so good on paper that you could almost vote for him sight unseen. He seemed so all-American, so much like Horatio Alger, that he was able to slip through the cracks of Objective Journalism. You had to get Subjective to see Nixon clearly, and the shock of recognition was often painful."
(Non paywalled link: https://web.archive.org/web/20150213034115/https://www.theatlantic.com/magazine/archive/1994/07/he-was-a-crook/308699/)
Sometimes, you need one or two journalists who are in a position to say "you know what? These people suck, and I'm sick of pretending they don't". It doesn't need to be every journalist, and it probably shouldn't be, but someone needs to say it.
Yeah, I mean that's kinda of the whole conceit of Behind the Bastards, the host is explicitly and inherently calling everyone they cover a bastard by default, but if you listen to Ed Zirtron's appearances, he always just immediately wants to boil them down to a bastard as the root cause of their actions, when the literal entire point of that show is to examine what factors and backgrounds turn someone into a bastard.
Or again, I just can't understand why he would be flabbergasted by a company naming their alert system after an early engineers' tank top colour. Does he think all quirkiness and whisky should be outlawed from the workplace?
Yes, there's value in calling people bastards and scum and villains, but Ed Zirtron does it immediately, every time, which makes his judgement of them untrustworthy. There's the old adage that "if everything hurts when you poke it your finger is broken", in Ed's case given that everyone is always a bastard or a hero, it seems more plausible to me that he has some pathological need to boil everything down to simple binary systems.
There's quirkiness and [whimsy?], and there's needless obfuscation. 'Code Yellow' meaning 'Code Red' is dumb. Like I get it, it probably started as an equivalent to 'Code Wayne' and subverting expectations is funny, but it's a punchline from an old adult swim show more than anything. I get that Google HQ isn't a Hospital or the military, but sometimes clarity is important. More now because they're actively doing contracts for governments and militaries, not a scrappy startup. They became a trusted resource and are now cannibalizing themselves for short term gains.
Whimsy at the top of a company while their workers are protesting their actions isn't great.
If someone at a company tells you "code yellow" do you stop what you're doing and follow your drilled into memory code yellow training from school, or do you say "hey, what does code yellow mean?". They're not obfuscating anything, they've just got a company procedure with a quirky name.
Shitting on that just shows that you are looking for things to shit on them for, rather than being a thoughtful critic pointing out valid flaws.
I feel Iike the correct application of this analogy here is "if everyone you examine is a bastard, you're the bastard."
It’s an interesting piece and starts in the traditional journalism mold, but moves much more into opinion and blog. Like going from NewsHour to Last Week Tonight. That’s not to say it’s not an interesting read or he’s not supporting his argument, but it is about persuading, not just reporting. Of course, I haven’t actually gone through all his references to see if they’re mischaracterized or taken out of context.
I agree with both your comments, but there's something so satisfying about reading vitriol about a type of person you fucking hate. I kinda liked that he doesn't hide his bias or disdain for these people.
If I hear code yellow, I assume I need to grab a mop and bucket.
And an illustrated book about birds.
I'd disagree - what this shows is only disdain for everyone who's fucking up technologies for the sake of profit. And I'm with him there, I found it refreshing to read an accurate account of what pieces of shit work behind the scenes in the industry. Not that I am surprised, but the account of what seems to have happened in detail and in that sequence was new to me.
Well you can disagree all you want but I don't see how you can read his snarky comments and think that.
His criticism of the code yellow is not because anyone involved in the code yellow procedure, invention, or naming deserves anything. He just hates everyone in tech so much that a whimsical name must be a bastard move, and not just people at their job trying to make the most of it.
Yeah, cause you're accepting his characterizations of everyone as bastards at face value despite not knowing them and despite knowing that Ed Zirtron thinks everyone is a bastard because it makes his world simpler. Yes it is "refreshing" to stop thinking about complex chains of actions and consequences and just think "he's an evil bastard man and it's all his fault".
"[considering] everyone as bastards" is a strawman argument. Furthermore, the people described are assholes by the evidence provided, assuming the evidence is noy falsified.
No, far from it. Noone involved with the naming of the code yellow name has any evidence of bastardry presented at all.
It’s like a reverse Kara Swisher. Which, though I hate her work and her complete lack of integrity, I don’t want. I totally get and agree with your take.
[Warning: "ideas guy" tier babble]
It's somewhat clear that search engines are too prone to go to shit, either due to malice or something worse (like stupidity).
Based on that, I wonder if a user-run, free-as-speech and open source decentralised search system wouldn't work. Roughly in the spirit of torrents - where anyone can use the system but if you're using it you're expected to contribute with it too.
You just described the categories pages many search engines had before Google. Or proto Web 2.0 bookmark sharing sites like del.icio.us. Sites like Metafilter also existed as a kind of Internet index before everyone was adding reddit.com to their Googling. It's a laudable idea, but these systems all seem to fall prey to market manipulation in much the same way that SEO helped kill Google.
It's interesting that you mention MetaFilter, because they're literally in the process of transitioning fully to a non-profit organization.
https://metatalk.metafilter.com/26430/MeFi-Nonprofit-Update-March-26-2024
They're the only aggregator that still isn't flooded with ads and has pretty decent moderation policies.
There's absolutely a reason I linked to the discussion over there: because it's quality, and it's the first place I saw the article pop up.
Wow, that's really neat.
Thanks for letting me know about MetaFilter and its transition to NPO. This really seems like a great move for the site.
I've heard of the site before, but haven't had the chance to try it before. Guess a bit late is better than never, right? :D
I was thinking on something slightly different. It would be automatic; a bit more like "federated Google" and less like old style indexing sites. It's something like this:
It would be vulnerable to SEO, but less so than Google - because SEO tailored to the algorithm being used by one server won't necessarily work well for another server.
Please, however, note that this is "ideas guy" tier. I wouldn't be surprised if it's unviable, for some reason that I don't know.
I think you could do it in Lemmy itself combined with RSS feeds. The mods would curate a list of RSS feeds, and use the keywords to pick the ones for a bot to automatically post (which means if a programming blog did a post about windsurfing, it wouldn't show up as long as the meta keywords didn't match). Mods could take suggestions each week for feeds to add or remove.
There was (is?) the yacy project which used a distributed index, and the individual nodes would contribute to the index.
A hybrid of original Yahoo! and Google is probably the best option. Sites submit themselves, they get reviewed, and an algorithm catalogs the contents. So curation and automatic indexing together.
https://en.m.wikipedia.org/wiki/YaCy
That exists, it's called Searx, and in my experience it isn't great
No that still uses traditional search engines in the background
Searx is a meta-engine, as bdonvr mentioned.
since i changed the instances i use, it works great for me
I've been using foss.family for a little while with good results.
You have to spend a bit of time selecting and using different instances before you find one that suits you. I tried about three before this one and I don't think I'll need to change again.
One part of this (which isn't really covered in the article) is that Google historically had a give-and-take relationship with people gaming search engine results. SEO has been a thing for a long time, and it's impossible to make it go away. However, Google used to punish sites that took it too far. It wasn't necessarily ideal, but it worked well enough to keep egregious spam out of the top level results, and companies could still direct users to their site when they had something they were actually looking for. SEO consulting companies sprang up who knew Google's rules well, and that arguably meant a bunch of grifters being overpaid, but at least the results stayed relevant.
Google seems to have given up on enforcing many of those rules.
There's also some minor discussion in the MeFi thread about "federated search" as well.
Self-hosted search also seems like a strong possibility.
The problem that I see with self-hosting is that it isn't a practical reality for most people, due to different tech expertises and machine capabilities. Instead I think that a better system would allow you to simply install some software, and contribute as much as you can while you use it.
I'm not informed on MetaFilter. From your other comment it seems that it's also an indexing site (besides being a community - from their "About" page). Is this correct?
Yes, it's got a really old-school layout, because it's been around since 1999. To me, the fact that they've managed to avoid advertising for 25 years while having the main indexing site as well as things like Ask MetaFilter, IRL meetups, and even a jobs board, it means they've been pretty darn good at managing their finances and figuring out how to support the site long-term without ads. They're also in the process of becoming an actual non-profit organization. They pay their moderators a living wage, because it's a job. That's... pretty amazing.
The comment section takes a bit to get used to, because it's just chronological order of comments, no sorted threads. Very, very old school web ethos. However, if you can get used to it, some really amazing discussion can happen in there.
One of the benefits of the ways MeFi posts work is often you have users doing massive amounts of research and providing literally mountains of links and analysis, you can get pretty lost in the weeds on some posts.
It's been the source of high quality discussions for a long time and there's some really interesting professionals on there who have been staples of the community for a long time. Think hackernews and how many people it has from the industry, but instead of it all being tech people (MeFi has it own share of techies) it is thoughtful and sometimes expert opinion from a large variety of disciplines, as well as first person accounts from people of all walks of life.
It's also where I first found this link (The Man Who Killed Google Search) and decided to post it here.
Seems like something the public library system should be doing. That and hosting websites for the community not for profit but as a public service.
While I'm on wish list tangent, post offices should be municipal banks and be a free email domain provider.
"emails"
It's like saying "trafficks" or "firmwares".
It still is.
The amount sadness for the loss of Google Search accuracy due to ad infiltration the author writes here shows how much of a corporate brand dick rider a lot of people are.
These corporations do not give a fuck about you, so mourning their loss is so pathetic.
No one cares Google sucks now. If you do, go get a fucking life. Move on and use something else for fucks sake. They won't care if you're dead, why do you cry when these corporations die?
it's not that the company died, it's that collective progress was sacrificed for greed.
Not everyone’s got the capability to make up for the lost utility in the tool themselves. Should they just go fuck themselves?
Yes, SloppySol, they should indeed go fuck themselves.
Ah. Well.
There it tis.
Maybe because it's hard to find stuff now? I don't care about the company.
I'm not sad that Google turned out to be evil because I care about Google. I don't care about Google. I'm disappointed in no longer being able to search for and find the things online on any search engine.
Right, so with all me very specific troubleshooting questions I should go where exactly?
Ecosia? Very limited search results
Yandex? More obscure results, probably not what I'm looking for
Bing? Ok on general stuff, not great on very specific questions
Yahoo? Never tried it, heard the enshittification has become bad
Duckduckgo and similar? Proxying Google
Edit: apparently it's proxying Bing and not Google. Idk if that's better but I got that wrong.
There is no way to get around Google. Everything else is either highly specialized, very limited or unusable in general.
Also feel free to chime in with your experience, I'm so down to hear what everyone has to say.
I'm pretty sure Duckduckgo proxies Bing, not Google.
I've been enjoying Kagi, although it also proxies google and others, and you have to pay for it, and I was dismayed to read on Lemmy recently that the CEO may be a sea lion. So yeah, the search for good search continues I suppose
As a concept, paid search engines is actually a good idea. It incentivize the company to produce great result so their users won't search over and over (which reduce their profit), unlike google which incentivized to reduce search quality so their users have to search over and over and see more ads (per the article). If it's not kagi, I hope other paid search engines start to appear in this space. Indexing the web is expensive, and after seeing what happened with google, it's clear that free ad-suported search engine is not the way to go now.
There’s an awful lot of things where if the incentives were to keep paying users happy instead of keeping advertisers happy we would see very different results from the service. Unfortunately, for an awful lot of these services people don’t want to pay for them, or at least don’t want to pay what it costs to make them financially viable.
The high cost of housing is squeezing people all over the globe and we're seeing a spike in homelessness in first-world countries from USA to Australia, where the affordability of housing is out of control, on top of explosive inflation of food costs.
It may not be that they "don't want to pay" but simply not enough people have enough discretionary income to pay enough to make the business financially viable.
I mean, that's what happened to Beeper and while I was a very early on their sign up list I decided to never give them any money. When it became clear they weren't able to keep things going on how much money they were making from paying users: Micigovsky sold to a larger company.
I think it's an issue that the services they're offering actually cost more than the market is actually effectively able to bear and they're trying to hide that fact with advertising and data sales to cover operating costs.
More simply put: Consumers don't actually have enough money anymore to be able to support a business, and businesses essentially now must rely on other businesses as customers to be able to functionally exist financially. Only other businesses have the finances to support new business.
https://hackers.town/@lori/112255132348604770
For folks not understanding the sealioning reference.
https://d-shoot.net/files/kagiemails.txt
The author is probably weren't aware that their blog post get a huge engagement on hacker news and the ceo got a lot of flak there, which was probably why he felt the need to reach out and "correct" the author.
Searx exists and is decentralized although as for the quality of results that's up in the air
DDG proxies Bing you silly fuck
Use SearXNG
It’s not just google that sucks. All of the rest have tanked/sucked ass to begin with.
Dude, no. Having good search results matter. People are directly influenced by what comes out at the top of search results. Finding a good reference makes the difference between a well sourced claim and just talking out of your ass. It absolutely has an effect on public discourse at large.
It doesn't have to be Google, but Google was so good at it for so long that we're now kinda lost.
Then either adapt or die. Move on to another search engine, host your own, use an AI LLM or go to the fucking library.
Complaining to a corporation doesn't do shit unless you affect their bottom line. And so far all these articles and message boards with losers complaining about this have done nothing to slow it down or reverse Google's trajectory.
You say that because it's clear you have no fucking clue how difficult a problem this is. This isn't something you can do overnight, and I'm not even sure a self-hosted solution is possible.
You say that, but it's clear you have no fucking clue how easy a solution is.
https://yacy.net/
Commercial options:
https://solr.apache.org/
https://www.meilisearch.com/
No, you just haven't thought through the implications more than a single step.
The real trick is SEO. These systems will be gamed. Google used to handle this by using its monopoly on search to enforce rules. It wasn't perfect, but it kept the worst spam from being in the top five results for the most part. Doing this self-hosted would mean a million users having to agree to do the same thing to punish spam results, and that does not work.
And then there's the problem of crawling and storing the entire web. Doing this for specific topics is doable. The entire web is not. Not for a home user with limited budget. YaCy's P2P mode might be a way around that, but it's also not really "self-hosted" anymore.
Microsoft dumped tons of money into making the second best search engine, and it's a bit of a joke. This is not an easy problem.
It used to help me greatly at my job (software development). I'm using mostly DDG as a replacement but it just isn't even close to what Google used to be years ago.