We unleashed Facebook and Instagram’s algorithms on blank accounts. They served up sexism and misogyny
How do the algorithms of Facebook and Instagram affect what you see in your news feed? To find out, Guardian Australia unleashed them on a completely blank smartphone linked to a new, unused email address.
Three months later, without any input, they were riddled with sexist and misogynistic content.
Initially Facebook served up jokes from The Office and other sitcom-related memes alongside posts from 7 News, Daily Mail and Ladbible. A day later it began showing Star Wars memes and gym or “dudebro”-style content.
By day three, “trad Catholic”-type memes began appearing and the feed veered into more sexist content.
Three months later, The Office, Star Wars, and now The Boys memes continue to punctuate the feed, now interspersed with highly sexist and misogynistic images that have have appeared in the feed without any input from the user.
I mean it‘s the exact same if you visit Youtube without an account or cookies. The Internet has become a swamp of right wing and neoliberal populism that kicks down on minorities and people with lower than average income in general. The insane amount of completely made up rage bait stories that you get recommended is just unfathomable.
I think it‘s gotten to a point where it needs to be regulated how many lies a site can throw at you at the same time and I don‘t say this lightly. I just see no other way to get this mind eating populist machine under control.
Youtube shorts are the worst. I am transgender so my historic is really not right wing
30mins on shorts and I end up in Shapiro's Dreamland. It's a nightmare
Pirate Software has the only shorts on YT worth watching imho
My favourite thing about his shorts is trying to guess if it’s going to be a wholesome one or one where either him or his chat bestow cursed knowledge onto each other
The only political thing I watch is Behind the Bastards and the algorithm keeps trying to get me to watch right wing shit. I installed a third party channel blocker and open suspicious channels in private tabs and it still serves me right wing chuds complaining about being suppressed by "woke google". I would troll the comments section if it wasn't for the fact that that counts as "engagement".
I despite all my lgbt content got served some 1 mil subs catholic lady saying how she loves red pill, pro life or something like this. Such women always have this crazy light in the eyes, it creeps me out.
You can feel that she is in some kind of state of altered reality, like someone on drugs. High on something.
It has either gotten better or just improved its suggestions for me over time. I basically never get right wing content anymore. There's plenty of garbage, but it's stupid garbage rather than dangerous garbage.
Regulation won't work, because regulation moves slowly, and these companies find workarounds fast. And as long as the cost of breaking the rule is less than the benefits of doing so, it'll be "just the cost of doing business."
A simple way to do it is to stop considering them as platform providers but editors and it shou'd be done in my opinion because by their recommandation systems, they are making editing choices.
I have to hard disagree on this outmost pessimistic outlook because it reads like any regulation we already have is pointless so we can just scrap regulations and rules altogether across the board. That's similar to the neoliberalist rethoric I loathe to see pushed into my recommendations and it's simply not true. In reality we do see that regulations sometimes do the trick. It's just that they likely won't regulate them as harsh as I proposed, but that's a different argument. Regulation as an instrument does work.
Or maybe you just live in a swamp of left wing and radical socialism?
That's just the Netherlands.
still capitalist ✓
Then how is them live in a swamp of left wing and radical socialism?
He doesn't. He makes false claims.
Fuck off.
Well, that just evidences that the only good way to do capitalism is with a wealth tax and unlimited paid sick leave.
Or maybe capitalism works well, if it runs on top of a good framework?
In which country is capitalism working well? Because I can't think of one that doesn't have a desperately poor underclass. Just ones that have a good non-capitalist government safety net to help them.
Nah, the capitalism parts of the NL are the most often criticised ones. It's a tax haven for corps - did you know IKEA is a Dutch charity? - and the healthcare system is only getting worse because of copying US patterns of private ownership.
That can change.
own goal
Capitalism, like every other form society takes, is inherently flawed. Power pools, and every system eventually falls into oligarchy. The only way to prevent this, is strong social welfare programs enacted with regularity. This is proven mathematically, here.
The only reason capitalism works there is because of their strong social welfare programs.
You're wrong.
I can't take this model seriously - it assumes that economy is a zero sum game. If an economy actually was the zero sum game, then where all the wealth came from???
To be clear, I absolutely agree with the title - inequality is 100% unavoidable, but for completely different reasons.
are you being serious right now, is this a real question? In the age of cryptocurrency you're asking this question.
I'm trying to point out the massive hole in the reasoning behind linked paper
Cryptocurrencies are barely relevant here
Edit: Wait. Are you a university student? What did they tell you about where the wealth is coming from? I'm genuinely curious to know.
you're wrong. I'm not interested in explaining why you're wrong, maybe someone else will.
I was explaining this to my daughter not long ago when she told me she kept getting recommended videos about something that offended her (I can't remember what, but something Republicans would be in favor of) on YouTube that the algorithm doesn't care whether or not you agree with the videos. It only cares about whether or not you'll watch them. And if you're willing to hate-watch, which many people are, you'll get served the same videos as the people who enjoy it.
And, of course, the more controversial the better because you'll get a whole lot of both groups. So if you post something sexist and hateful, you'll get a huge number of redpill viewers and the like and then all the other people who go to that post to argue with them. Which means the algorithm learns that those are the best things to push on new accounts too.
You engauge with it, you must like it.
That’s just not true. Many people engage with things they don’t like because of education and curiosity.
For instance, I don’t like your comment but I still engaged with it to point out that you’re wrong. I like Lemmy, in general, though.
I took that comment to be summarizing the platform’s perspective, rather than their own. I think it’s common sense that people will watch/engage with things they don’t like, but the algorithms don’t care about how you feel or why you watched something; they see engagement and they give you more of that thing to drive more engagement. As far as the unfeeling numbers go, engagement might as well be liking; they don’t need to distinguish a difference.
Sure, I saw that side of it too as I work in marketing but I took the comment at face value since it wasn’t specific about that intent. I’m happy to retract my comment if that’s what they meant.
That is what I meant - engage to facebook means you like it and want to see more.
As someone who used to sit around at a TV station for hours waiting for news to happen so I could go shoot it, I can tell you for a fact that you don't have to like a show to watch it. You just have to be bored and it's in front of you.
That said, I did find out that American Ninja Warrior was amusing.
Howard Stern's entire career was famously based on that fact.
You engaged with this post, you must like it.
No, they think "You engage with it, that makes us money."
That's what happens if you only care about engagement. Chauvinism of any kind is liked by a certain amount of people and despised by the rest of us - both positions drive engagement.
That’s why a format like lemmy is the only way. No desire for profit means we let the content do what it does organically.
I think they vastly underestimate how many things Meta tracks besides ad tracking. They're likely tracking how long you look at a given post in your feed and will use that to rank similar posts higher. They know your location, what wifi network you're on and will use that to make assumptions based on others on the same network and/or in the same location. They know what times you're browsing at and can correlate that with what's trending in the area at those times, etc.
I have no doubt that their algorithm is biased towards all that crap, but these kinds of investigations need to be more informed in order for them to be useful.
Well it also looks at stuff like the other devices connected from the same IP, other devices near where you connect from and then based on this also hones the served content.
So if the author/some colleague or even the neighbors are red pilled/MGTOW/Chriso-Fachists etc this also makes sense. Possibly even their research into these subjects slanted the results. Let alone what happens if they spun up a VM at a cloud farm and used it from there.
I'm in no way surprised that "social" media corps serve up vile shit for profit ik just not convinced by some random let's see what happens.
Edit: I'd be for a law that required targeted ads to have a small "why you see this" and if you click it the company is required to show you the selection criteria that caused this ad to be served to you in an easy to understand format. (Leaving out all the irrelevant criteria).. ie.
You where selected by the following criteria:
It's also probably looking at scroll speed. So if the people conducting the experiment tended to linger longer examining content they disliked, that could result in getting more of it.
Would need to see a more detailed explanation of the methodology. Ideally the scrolling was done in an automated way, at a consistent speed.
Yep, it’s called dwell time and it is 100% one of the metrics used by the algorithms that decide what content to serve up.
When I first joined Internet communities as a preteen, I just followed forums that interested me and got exposed to whatever people happened to be talking about on those forums.
Why, oh why, has the world decided that we need recommendation algorithms at all?
The algorithms aren't there to improve the user experience they're there to increase user engagement. People engage with things positively and they engage with things negatively. The algorithm doesn't care.
Why is every third Reddit post someone "accidentally misspelling" or otherwise humorously butchering a post title? Because people comment on it.
to make money
And many people are refusing to leave Twitter, Facebook, Reddit, etc. due to those very same algorithms are not being present on Mastodon, Lemmy, etc.
Corporations realized infinite growth is unnatural and had to engineer a way to keep themselves marketable for rabid investors. Lo, and behold.
I have a ~3year old fb account because I have to use messenger. I don't use FB almost at all (I even sparingly accept friend requests) and have turned off ~anything that provides targeted content (I live in EU).
Since about last year (or possibly even before), my feed is about 35% Ikaria ads (a Greek island, I'm from Greece), 10% porn, 10% sexist-misogynistic stuff, 15% sexist-misogynistic porn, 15% christian stuff and the rest random stuff.
At least this might confirm that turning off targeted content works..🤷
I use Facebook because my relatives are spread out across the world and that's the way I know to stay in touch and also my brother, who is neurodivergent, mostly likes to communicate to the family as a whole that way and I want to be able to stay in touch with him too. On top of that, I'm stuck in a town I hate with no friends here and some old friends from my hometown are there so I can talk to them.
Anyway, ever since my brother started talking about how he was taking various hallucinogenic substances and calling himself a psychonaut (he's almost 60, he got into it very late), most of the ads I see are for shroom gummies, ketamine and boner pills. I've done my "psychonaut" stuff back when I was in my teens and twenties. I'm not interested in the former two and the latter is, thankfully, not necessary yet.
The funny thing was that maybe 4 or 5 years ago, I kept getting shown an ad for a wooden hurdy-gurdy kit. Like the medieval instrument. I have no idea why. I have never expressed an interest in playing the hurdy-gurdy, listening to the hurdy-gurdy or building anything out of a wooden kit. It became a joke with me and my friends for a while.
You know, you can just use Messenger on its own, on desktop web or mobile. I also need to use Messenger in some limited capacity and I have no idea what my feed looks like :)
You mean without a fb account? I think you're right, but I think this wasn't possible when I made the account ~3years ago.
About desktop web, I'm only using the website on my pc (and I have isolated the app on work profile on my android, while turning off ads with an lsposed module).
Oh, no, I mean I have an account, but I never (...exceedingly rarely) go on Facebook itself. messenger.com is just the messenger with no feed or other features, and there's a standalone mobile app called Messenger as well, same idea. I use those when I need to interact with someone over Facebook so that I'm not exposed to most of the crap.
I don't know anything about using it totally without any account.
Ahhh, yeah. I too use the messenger app on mobile and the messenger website on pc.
I just happen to open facebook (I have an open source front end client) from time to time out of boredom, because it's kinda amusing how bad my feed is :)
Ah okay, lol. Sounds like you've got it under control.
So these algos they time how long you look at things when scrolling. So to begin, sure there's no user input, but the mere act of looking unless controlled to be exactly the same on everything is in fact input.
Getting a sexist thing and lingering on it for an extra 5 seconds tells the algo you engaged with the sexist thing so send them more sexist things.
Edit: it shouldn't be serving it up at all, but that's how this can happen.
I wonder if they did any other Internet activities on the phone at all too... I don't know for sure if it matters, but I wouldn't be surprised if Facebook can track where you go or have been on the Internet to some degree.
Absolutely, that's the whole tracking ID thing.
Apple makes that required to opt in now, I don't think that's the case in all Android devices yet though?
Or so it said.
One of the rules of the Internet is "unsubscribe from the defaults."
Of course, young men were starting to be much too peaceful and open-minded.
This is the best summary I could come up with:
On Instagram, while the explore page has filled with scantily-clad women, the feed is largely innocuous, mostly recommending Melbourne-related content and foodie influencers.
Nicholas Carah, an associate professor in digital media at the University of Queensland, said the experiment showed how “baked into the model” serving up such content to young men is on Facebook.
She praises the federal government’s Stop it at the Start campaign, which includes an “Algorithm of Disrespect” interactive depicting what a young man may encounter on social media.
The federal government has also funded a $3.5m three-year trial to counteract the harmful impacts of social media messaging targeting young men and boys.
The social services minister, Amanda Rishworth, says combatting misogynistic attitudes and behaviour in the online and offline world will help achieve the national plan to end violence against women and children in one generation.
“Around 25% of teenage boys in Australia look up to social media personalities who perpetuate harmful gender stereotypes and condone violence against women - this is shocking,” she says.
The original article contains 1,154 words, the summary contains 170 words. Saved 85%. I'm a bot and I'm open source!
The two examples of misogynistic memes in this article are really tame IMO. It's the type of humor that teenage boys have always had, long before the existence of social media. Don't get me wrong, I definitely think social media algorithms are having a major negative impact on society. But I don't think content like this is the problem.
Holy shit, so you mean you made accounts posing as men and got content targeted mostly at men, presumably based on what other men have interacted with in the past? Say it ain't so. Why would the algorithm do this?
Holy shit, were you born yesterday?
Social media was not a stew of shit a decade ago. The algorithm recommending this crap is not a natural phenomenon, it's a practice that the companies running things have come to adopt. They could adopt a different practice. Source: was a man on social media a decade ago and was not constantly bombarded with toxic shit.
I quit Facebook a decade ago because it was toxic shit but I get your point. There was a moment in the past where it was good and connected you to distant people in your life and little else.
I'm able to keep facebook okay, but only by constant blocking all from. I wish I could turn off all shared content. The good stuff on facebook is people I personally know but see rarely - but that only takes a few minutes to catch up on and they need me to doomscroll for hours to make money.
Lol imagine being on social media for 10 years and still complaining about the algorithm.
Here's a post from 11 years ago explaining how you were being targeted and how engagement was measured back then:
Source (actual fucking source, as in an article written Aug 13, 2013, instead of your personal experience from the last decade): https://buffer.com/resources/understanding-facebook-news-feed-algorithm/
So, in those 10 years you've been on social media (congrats on the milestone btw, maybe you'll get a clue about the fucking world soon) what they've been doing has not changed, it's merely been perfected. But yeah, sit there and tell me all about how your rose-tinted glasses are ackshually great and don't distort your view, and you're not just a mindless cunt that's been zucking the zuck's dick for the last 10 years, scrolling through shitty posts meant to make you click them. "things were better back in the day" lol gtfo dumbass it's always been the same, it just took you 10 years to notice.
What an actual waste of my time.
Fine, I'm a middle aged fuck who thinks the late 2000s is 10 years ago. So sue me for moving the goalposts to 15-17 years ago, around the time Obama was getting elected. Yes engagement was a thing but fuck no it was not a constant deluge of far right propaganda.
Uh ... sorting posts into video, pic and text only and then by votes and comments is not what we're talking about when we say algorithm. Though yes, I guess that technically is a very simple algorithm.
It's the same algorithm. It has the same purpose and the same result. It has simply been updated and improved. And it most likely still relies on certain markers that can measure how much a post will be engaged with and who will do it, except those markers are now less primitive and harder for us to define.
We're arguing semantics. This has always been the purpose of a social network: to keep you addicted to it. To keep you interacting with it. They don't make money if you're not there to click their ads, to look at their sponsored videos, to be marketed towards. Did they do it as well 10 years ago as they do it today? No, of course not. But you were still being targeted with posts that would "do well" with your gender, your age group, your location etc. They haven't changed one little bit of their business model.
So what are we talking about here? Some guy discovered 10 years down the road that a company wants you to keep using their website/service/app/whatever, but he thinks 10 years ago that company was - what? More scrupulous? More genuine?
Nah, man. It was always the same. They just got better at their jobs. And - fuck me, it sounds like they were pretty good at their jobs even 10 years ago: they managed to keep Joe Slow scrolling for an entire decade.
If the algorithm was "updated and improved" then it is not the same algorithm.
My bad, I didn't know we were building our own 'ship of Theseus' argument. I'm out, smarter people than me can discuss that one.
If it doesn't work in the same way anymore, using the same variables, then it's not the same ship. Your argument is like saying a submarine and an aircraft carrier are the same because they're both intended to fight ocean battles. Pretty much nonsense.