GMail is Breaking Email

Jo@readit.buzz to Technology@beehaw.org – 61 points – GMail is Breaking Email
igregious.com

Email is an open system, right? Anyone can send a message to anyone... unless they are on Gmail! School Interviews uses two email servers t...

70

And this is how you kill an open standard. Good resource to share with people cheering for Meta to adapt ActivityPub etc.

Thinking about starting my own personal email server, but to use it seriously I’ll have to weigh the pros and cons. If anyone has anything on this to share I’d appreciate it.

Short answer: Don't.

Long answer: It is a massive amount of work, not just to setup, but also to maintain. On top of the fact that the big email providers block smaller email servers like crazy. Even if you had business class Internet service at home, the IP range is most likely already in their block lists. And if you have it on a VPS, the amount of time and effort it takes to get the security and filtering going properly is nightmarish.

It really sucks, but it's a fait accompli.

Would agree.

Even when done 100% by the book and correct. Companies like Google and Microsoft, in particular, will just randomly send the email to spam.

I gave up after years of fighting the good fight and went to googles free tier. That is now over and I probably just need to move to some other service.

Also dont use a gTLD or if you do, have a backup .com or .us as well. Many forms dont recognize things like .email as legit.

Additionally, these days the sheer amount of flak that a self-hosted mail server gets are enough to make a lot of providers ask you to either shut it down or go somewhere else. Probably 80-90% of the server's inbound network traffic will be bots trying to brute force access (usually over POP3 or IMAP4, though occasionally SSH) to use it as a spam relay as well as relatively dumb bots just assuming that your server is an open relay and trying to send garbage through it. That kind of traffic hogs a lot of bandwidth and the hosting provider will have to do something about it to keep their infrastructure stable. Also, figure that you'll be spending about as much CPU time on the server for anti-spam processing on a 24x7 basis.

I have to agree with other commenters, it's just not worth the hassle and kinetic pattern baldness these days.

Even if you set up everything perfectly you encounter email providers that only have allow lists and you have to jump through hoops to be allowed to send emails to them (like publishing your whole name and address). I loved the fact that I had a mail server but in the end it didn't make sense.

Yeah, I think this is done to provide the illusion of choice. The rate limits are high enough to allow personal emails through, but for any mass emails or corporate emails this forces you to use Google. Unfortunately a standard corporate strategy, it's why corporate office suites are so generic and tend to be from one of the big companies.

When I went to the DMV my independent mail server was immediately filtered into spam when I tried to email them my proof of insurance. It was no trivial thing for them to get it out of the spam filter, either

Anyone know a decent alternative at a reasonable price though? What if I have an @gmail today, and I want to move my storage elsewhere and have that just forward?

I switched to ProtonMail and have really enjoyed it. I was using my own domain with Gmail so my email address didn't even change.

For those considering Proton Mail: There is one great benefit or disadvantage, depending on how you see it. As all traffic is encrypted, Proton Mail does not support standard IMAP or POP3. It's therefore best used with the official Proton Mail app rather than third party apps. On desktop, you can use your favourite email client (Thunderbird et al) only if you install a "bridge" which decrypts incoming emails before forwarding them to the client: this bridge is, in turn, only available to paying subscribers.

That said, it's a great service, and the fact that they have a viable business model which doesn't depend on selling out their users might be a good thing.

IMAP supports TLS, what’s Proton’s excuse for enforcing their own delivery protocol?

Proton is end-to-end encrypted - they don't have the keys themselves. With TLS, encryption is between you and the server, but the information can be decrypted on the server side.

At least that's my understanding of it. If you want Proton's own words, they wrote an explanation on their website. :)

Any advice or hints on how to switch over? I wanted to do it years ago but I dread having to change my main mail address on everything, from apps, tools and games to bills or RL document-related stuff, it sounds like a horrible mess and ton of work

My recommendation (assuming you have a normal @gmail addy and not a custom domain like I had) would be to use email forwarding. So you can leave your Gmail as is, but set it up (in the settings) to automatically forward all your email to your new protonmail address. Then you can gradually change the important contacts/sites to your new email at your leisure.

I do highly recommend buying a domain and setting up your own email address though, it gives you a lot more portability going forward. You can actually do a lot with your own domain, and it helps you maintain trust better.

Anyway, enough preaching lol, protonmail also maintains a guide to help people switch: https://proton.me/easyswitch

If you're recommending setting up a forward/IMAP collection from a Gmail account, don't forget to mention deleting the messages from the server as well! Emails left on a server for more than 30 days are considered "abandoned property" for the purposes of warrantless search.

Do you have a source for this?

The most I could find is that the Electronic Communications Privacy Act allows for warrants to be issued for emails less than 180 days old. I've found vague references and snippets from articles no longer available that seem to claim some acts that have passed since then allow for simple subpoenas instead of full on warrants for said emails, but 180 days is the only threshold I've found and again, it's for less than 180 days that's at danger.

My recommendation for everyone is to use Fastmail and a custom domain.

Fastmail is extremely reliable, and since they charge money they also offer customer support. A few years ago I lost a lot of emails due to a client bug, and Fastmail support was very helpful recovering them from backup.

Use a custom domain so you can change providers in the future so you're not locked into your provider and can change if you aren't happy with them anymore.

I'm also using fastmail and I'm happy with them. Their native android email client is a little clunky but I still use it and I have the option to use other mail clients too.

allow me to second proton mail. First they're in the EU, so fall under their privacy protection rules. Secondly, the service is technically an encrypted email service. They operate on a fremium basis, letting you have basic email for free, but also,if you upgrade, you can do lots of things (like they'll manage your email server if you happen to have a domain)

I feel like step 1 is just buying a domain so you can have control over your e-mail address, and then you can switch providers whenever you want (or host it yourself).

If you already pay for extra iCloud storage you can use a custom domain for e-mail with iCloud... Many people are already paying for this, and if not it's only $1/mo. Apple's still a pretty big e-mail provider, so maybe that doesn't address all of your concerns, but it's a really cheap way to use a custom domain that more people should take advantage of imo.

I host my own e-mail and it's pretty care free these days (I don't send bulk e-mails, though, so I don't contend with rate limits at all). Honestly, more people should do it instead of buying into all of the fearmongering about e-mail... It's a little tricky to set up right, but the impossibleness of the situation is somewhat exaggerated. The best defense for self-hosted e-mail is if more people actually do it... Otherwise you're just capitulating to the large (and slightly less large) mail providers.

+1 on having your own domain. I was using gmail for a long time, and recently switched to my hosting provider's included-with-purchase email. Having my own domain made the move transparent to everyone, and relatively painless.

I use posteo.de which is a german provider. It costs 1€ per month. Did not have any problem with them and I've been using them for years by this point

Anyone got a different site covering this? This site's HTTPS certificate is invalid or sth which doesn't inspire confidence

It's hosted on blogspot, it's a google issued certificate, to me seems valid

But the posted link is http:// not https://, so browsers will mark it as insecure.

Ah I assumed that Google had enough engineers to set automatic https redirection on blogspot...

And this is happening after SPF, DKIM and DMARC provided a solution to the spam problem.

Any mail system can remove practically all spam by insisting messages conform to those three standards

But that is not true at all. Spammers can easily send mail with all proper SPF, DKIM and DMARC records and signatures. A lot of spam is and will be sent like that. Those extensions do not make spam impossible, they just make it easier to track and block.

But this does not change the point of the article – in this case it is a specific domain sending very specific non-spam messages. SPF/DKIM/DMARC prove it is not someone else – GMail has no ground for blocking these (unless were are not told something).

And GMail has been breaking mail for years now. E.g. I hate them for breaking message threading by ignoring threading headers and forcing own view on how messages should be grouped.

How does one send a spam email that passes SPF and DKIM if one doesn't have access to the DKIM private key, or the DNS server to edit the SPF or DKIM records?

  • Open a Gmail account, send spam.
  • Buy a domain, setup SPF and DKIM, send spam.
  • Hack an SMTP server, send spam.

In my experience Office 365 is even harder to deliver to. The email envelope can be in perfect shape and sent via sendgrid (their recommended partner) and it will still silently drop mails for no obvious reason and if it does deign to deliver them it will often mark them junk.

I’m only sending low volume transactional emails. The amount of time I have to spend tweaking the email content just to persuade Microsoft to deliver the mail is absurd.

In my experience Office 365 is even harder to deliver to.

Yep, this is my experience as well.

I've had some issues with google, but at least they tend to tell me the messages are being rejected.

Microsoft will give me a 250 message, and then route the message to /dev/null.

That's contrary to the RFCs, and really annoying. Since it doesn't end up in Junk, the receiver can't say 'not junk', and since it doesn't bounce, the sender thinks it has been sent.

I'm signed up for Microsofts junk mail reporting, and when this happens the UI shows no issues with my ip, and doesn't admit to any e-mail filtering. The only way I can detect it is by sending messages to my test accounts, or waiting for users to yell.

Fwiw, anyone else who runs in to this scenario, expect your first support ticket with microsoft to be rejected. Keep responding to it. On the second or third try they might end up removing the silent ban.

In this position I would already be putting warning on the order page to say to explain to Google/Microsoft users that they will receive a confirmation email, but that their email provider may delay the email to up to 12 hours before they can see it in their inbox.

It would be informative to customers, avoid some support calls, and would give credit where credit is due.

I'm not really all that surprised. Google, Microsoft, and Yahoo aren't incentived to design for a precise spam detection system, just one accurate enough to prevent spam from being a big issue.

The term you're looking for is 'Horizontal Conduct' and it's illegal. The hard part of course is making that claim against the team of lawyers that Google would be able to field.

Confirmation emails when they create a booking being marked as spam but is that the only email they send out. Do they send out unsolicited ones to folks like me who mark them as spam? Are bots making booking and using my email. Its great they are doing everything right but im skeptical that something else they are doing is not causing the issue. I get plenty of good email in my gmail spam but the ones that do I can bet get marked as spam for good reason. Like from my alma matter which Im sure has a good and trusted endpoint but they can be a bit annoying with their emails and some folks likely mark them as spam. I don't but I could see them getting a reputation of sorts.

I had delayed messages, kicked back messages, and messages lost in the ether. I finally gave up and went to a small hosting company that seems ok so far.

Could someone please ELI5 this? I get the overall concept but I don't really understand why doing this is convenient for them.

They stand to benefit by having a de facto monopoly over email by exploiting their position as one of the biggest email providers in the world

"Monopoly" may not be the right word since Microsoft and Yahoo! also seem to be engaging in similar anti-consumer practices, but basically they seek to end the days of independently hosted email providers, which is obviously not good for us

Here's another fantastic article related to this. It's about someone who's had to give up on selfhosting.

https://cfenollosa.com/blog/after-self-hosting-my-email-for-twenty-three-years-i-have-thrown-in-the-towel-the-oligopoly-has-won.html

It's actually far worse than the Igregious article makes it look.

I wish they'd gone into a bit more detail about the issues they had, where they hosted, how they tried to fix their ip reputation, which providers blocked them, etc.

I've experienced the same issues in the past, but didn't find any of the insurmountable.

Though admittedly mine is more 'small business' than 'self-hosted', so I can afford to buy a small IP block and run on dedicated hardware.

I appreciate the frustration, you're doing everything right. But solving the problem of spam at Gmail's scale is hard. Gmail should prioritize user's experience and I can see the balancing act between timely delivery and reducing spam. Are these two new mail servers?

Spam shouldn't really come from folks that meet the standards they're at though. It makes spam not cost effective to meet all of that. In fact, they'd likely be more effective following those standards than making up their own, which does fail frequently enough to notice.

I disagree on this point. Spam is a customer service and if a company will face the consequence if users are upset by to much spam which actually has been a reason folks used to abandon emails in the passed before spam filters got good which was done at the product level initially. The standards came later out of what was being done at each place independently.

I don't understand your rebuttal. Nothing actually counters anything I said other than you saying you disagree. The rest of the post isn't contradictive. I agree spam is bad. I agree it's important to stop it. My point is that spam servers can't meet the requirements listed above. The only behavior that Gmail is calling out is the amount of emails being sent, but they're still sending them. That's not countering spam. That's pushing people to use their own system which is identical but isn't free.

yeah I think I misunderstood you. I thought you were saying there was not reason to block if they met all the standards. Which just turns sending spam into the get your link high ranked type or thing.

Email is a relict of a bygone era and needs to die. It's not designed for the modern Internet, and no patching like DKIM and DMARC can fix that.

Do share the alternative with us, that's universally supported and not owned by a corporation.

Already responded to someone else asking here.

So the article is about unwarranted 12 hour delays and your solution is to use a federated platform where delays are built into the system between server syncs? The fact that people cant see your other post yet because their local servers have not synced your post to this thread should be the first sign that this might not be a good solution to this email problem...

Email also is a federated platform that syncs. It's just a matter of getting this working, it's a solved problem on a conceptual level.

For example, all mail servers come with an outbound queue for mails to retry for at least a day until the mail goes through. Lemmy simply discards the message after a single try. This is a result of being beta-level software that just hasn't been fully finished yet.

That's the first time I've heard anyone says that. What would replace it?