Gmail SPF Softfail 0.0.0.0 when sending between accounts on the same server.

[DigitalComunic]

Hello all.

I'm facing a problem what is news to me, when users send messages to other users on the same server. Messages sent to other servers are working fine. After received some custommers complains, I did testes and I noticed the problem. The 3 scenarios bellow I tested using the same sender and same recipient in test 1 and 2. In test 3 it is a different recipient.


Situation 1, sending via webmail - Received with problems.

When a user sends by their webmail to a recipient account on the same server, and the recipient receives the message in Gmail, the message does not go to spam, but SPF registers IP 0.0.0.0 and DKIM and DMARC are not registered.

SPF: PASS with IP 0.0.0.0

Authentication-Results: mx.google.com;
spf=pass (google.com: did not find external ips, assuming domain of [email protected] as allowed sender) [email protected]
Received-SPF: pass (google.com: did not find external ips, assuming the domain of [email protected] as allowed sender)

Situation 2, sending via local Outlook - It went to spam.

When the same user sends through their local outlook and the recipient receives it in Gmail, the message goes to Spam and a softfail is registered. SPF registers IP 0.0.0.0 again and DKIM and DMARC are not registered.

In this case, IP xxx.xxx.xxx.xxx shows the local connection IP of the user and not the IP of the server.

SPF: SOFTFAIL with IP 0.0.0.0

Authentication-Results: mx.google.com;
spf=softfail (google.com: transition domain [email protected] does not designate xxx.xxx.xxx.xxx as allowed sender) [email protected]
Received-SPF: softfail (google.com: transition domain [email protected] does not designate xxx.xxx.xxx.xxx as allowed sender) client-ip=xxx.xxx.xxx.xxx;

Situation 3, sending via Outlook or local Webmail to another server - Received without any problems.

When the user sends via local Outlook or webmail to another server (in this test a Gmail) the message is received perfectly without any problems. SPF, DKIM and DMARC are registered correctly.

In this case, xxx.xxx.xxx.xxx shows the server IP and not the user's local IP.

SPF: PASS with IP xxx.xxx.xxx.xxx
DKIM: 'PASS' with domain xxxxx.com.br
DMARC: 'PASS'

ARC-Authentication results: i=1; mx.google.com;
dkim=pass [email protected] header.s=default header.b=jB7ZS2WC;
spf=pass (google.com: domain of [email protected] designates xxx.xxx.xxx.xxx as allowed sender) [email protected];
dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=xxxxx.com.br

I never had this problem in years, maybe something changed with last Cpanel updates? Or Gmail changed something to use it to read messages in Cpanel servers? I did some checks on the DNS settings and they are all fine. Tested by emailing Newsletters spam test by mail-tester.com, I get a 10/10 rating, perfectly. The problem only happens if the message is sent to the same server, but if it is sent by webmail, message forwarding, and if it is sent by outlook, the message goes to spam.

Any advice to find the cause of this problem would be great.


Regards.

 

[cPRex]

Hey there!

When a user sends by their webmail to a recipient account on the same server, and the recipient receives the message in Gmail

Can you clarify this part a bit for me? If they are sending the message to an account on the same server, what mechanism gets the message to Gmail? Is there a forwarder or mailing list in place?

 

[DigitalComunic]

Hello cPrex. Yes sure, sorry for any misconception in explaining. No forwardings or lists, but pop3. I notice the issue with users who host their domains and emails on my server but download (from my server) and read their emails in gmail.com via pop3 (Gmail settings > Accounts & Import > Check emails from other accounts).

 

[cPRex]

If you do a search on Google for their own "SPF: PASS with IP 0.0.0.0" message, you only get two results, both of which aren't helpful. Since the SPF record checks out everywhere else, you may want to contact Google directly to see if they can offer more specific details on that behavior as I'm not finding anything on my end.

 

[vikins]

This issue is the same as this issue.

SPF Softfail From Client Ip

I'm getting an spf softfail from google. spf=softfail (google.com: domain of transitioning [email protected] does not designate 1.2.3.4 as permitted sender) Now, I fully understand that I can add 1.2.3.4 to the spf record and it should all work. The part I'm having trouble with is that 1.2.3.4...

forums.cpanel.net

Maybe an admin can merge all of these and we can make some headway?

 

[cPRex]

I'm not able to merge a thread that's five years old, but I'm happy to talk about the issues here.

 

[vikins]

Thanks. Did you remove my reply on the other thread?

SPF Softfail From Client Ip

I'm getting an spf softfail from google. spf=softfail (google.com: domain of transitioning [email protected] does not designate 1.2.3.4 as permitted sender) Now, I fully understand that I can add 1.2.3.4 to the spf record and it should all work. The part I'm having trouble with is that 1.2.3.4...

forums.cpanel.net

I replied there last night and attached an image.

 

[cPRex]

I did remove that reply as the thread was five years old.

 

[cPRex]

@vikins - I thought I actually merged it into this thread but apparently that didn't work! Can you post that again?

 

[vikins]

It was the appropriate place to post because the issue is still happening and that cPanel thread was the most informative and well documented.

The most well documented thread anywhere I can find about the issues is this one:

Gmail SPF fail based on client IP

Gmail is failing SPF check based on the client IP. These are the relevant headers: Received-SPF: fail (google.com: domain of [email protected] does not designate 164.77.240.58 as permitted sender)

serverfault.com

The reason I'd like to talk about it here is because this issue has been known for years, Google must know about it. They are not doing anything about it. So I think it must be addressed on the server side, and that means involving exim / cpanel and adding headers of some kind.

The point I was making in the other cPanel thread that got deleted is that it is no longer just that it quietly does an spf softfail, it actually puts a huge obnoxious warning inside the email and/or it goes directly to gmail spam. This is a NEW twist on the issue that was not happening before. I've attached what the google warning looks like.



This happens when people make use of gmail feature "Check mail from other accounts". You can config a gmail account to POP mail from any legit POP mail account and it will pull it into your gmail and process the mail. It will not pull in emails with suspicious attachments though and I've never seen it have a false positive. But unlike when you simply forward mail to gmail, it rewrites the headers in a very strange way and ends up doing an spf check on the client IP address of the original email sender. As in, their home IP address of their wifi router, which makes no sense whatsoever for spf.

The key thing here is that this doesn't happen with all mail. This only happens with mail that is delivered locally on the server. For example, when a customer sends an email to another customer on the same shared server. That email will process internally and not go through SMTP. It is in these cases that the gmail feature "Check mail from other accounts" will rewrite the header and then it will fail spf and produce this warning.

It is a very specific kind of case, but nonetheless it does happen more than you'd think.

Since google isn't going to fix this anytime soon, there are suggestions out there that adding a header using exim can resolve the issue. I'd like to discuss how this can be accomplished.

 

[cPRex]

Thanks for that - I spoke with our email team and they thought it would be best to submit a ticket to our support team so we can see this in action on the server-side to see if there is anything we can do on our end.

 

[vikins]

Ticket # 94440007 has been created.

 

[cPRex]

Thanks - I'm following along with that on my end as well.

 

[Metro2]

@vikins - the same thing you talk about in your post above Gmail SPF Softfail 0.0.0.0 when sending between accounts on the same server. started happening with my servers / customers just back in early February, after over a decade of no issues whatsoever between us and Gmail. Not-so-coincidentally, Gmail started marking ALL emails that pass through my servers to Gmail / Google Workspace recipients as spam. I knew something was wrong at Gmail's end, because all of my IP/SPF/DKIM/DMARC headers were showing as "PASS" in the "Show original" option of the Gmail messages, and all my mail server IP addresses domains were (and still are) 100% clean, not on any blacklists anyway, and had/have a Reputation of 100 on SenderScore.org . Didn't matter if they were retrieved by users using the "Check mail from other accounts" and directly POP3 downloading in their Gmail from their domain, or to any Gmail user / recipient. All regular messages were suddenly marked as spam by Gmail, and emails for some customers using Gmail to "Check mail as" POP3 for their domain email were getting that ominous "Be careful with this message" banner that you posted. (In fact some still are).

So beginning back in February I purchased Google Workspace & set it up for one of my personal domains for testing, and so that I could actually take a shot at speaking with someone in Google Workspace Support. I was able to get a case opened on their end, as Workspace technicians confirmed that they could see no reason why suddenly all messages through my servers were getting tagged as spam by Gmail. The case dragged on for almost 2 months while I stressed over it every waking moment (which was basically 24/7 because you can't sleep when all of your customer's messages to Google recipients are getting marked as spam and you're inundated with tickets / calls over it). Finally on March 24th, after 5 weeks of seemingly canned responses, I received an email update from Google Workspace support stating that they had resolved the issue on their end. And despite my best efforts to obtain details from them in regard to why this happened and what Gmail engineers discovered / how they fixed it, their support remained / remains tight-lipped about it, stating that is "confidential data" with absolutely zero explanation.

Even though I was relieved that they resolved the problem on their end and that finally after 5 weeks of sweating it out, (almost) everything went back to normal and all messages to Gmail recipients through my servers / domains / customer's domains were no longer being incorrectly marked as spam and finally going right to Inbox as normal, I still wanted to know the reason this suddenly happened and I kept digging. And while I don't dare post the links here, I ended up finding a couple of tech blogs that had reported that they discovered Gmail had made a change to it's AI / filter algorithms back in February. I can't confirm the sources, and Gmail is not admitting anything, but it made sense and I was able to find out that it wasn't just my servers or my customers, but other hosts and businesses had been hit with the same situation.

And now to this day, although almost everything is fine again between Gmail and my servers / domains, there are still a number of legitimate messages getting marked by Gmail as either Spam or Dangerous - and those are a portion (about 10%) of the emails generated by server alerts and notices (such as some high load alerts and the like). Also, there are still a couple of customers who get that scary yellow caution banner that you posted an example of when even just sending an email from their own domain addy to their own domain addy but using their Gmail account to "Check mail as".

Bottom line - been serving Gmail / G Suite users for over a decade, and never had those issues until February 2022, Workspace techs confirmed all was good on my end and opened an investigation, then in late February a few tech blogs reported that Google had tweaked their spam filters earlier in Feb, then in late March the problem was 90% resolved but with no explanation from Workspace engineers as to why / what / how. Take that, plus the fact that no other similar services (Hotmail / Outlook / etc...) were marking normal messages as spam or dangerous, and the math becomes slightly more simple - Gmail changed something in late January / early February that had this effect on a lot of hosts and businesses, regardless of how pristine their email reputations were, and they're not coming forth about what they did or how they are resolving it.

It's not just you.

 

[vikins]

@Metro2 It sounds like you resolved a part of the issue, possibly by getting your spf / dkim / dmarc properly aligned, which can resolve the gmail warning message for most situations. But I think the 10% of messages that you mention that are still having an issue is due to the same issue I'm seeing. These are messages are ones produced by the server and/or messages between users on the same server, meaning messages that are handled internally on the server. Those that go though normal SMTP channels are not affected by this issue. Happy to keep working together to solve this last 10% of the issue.

 

[Metro2]

@vikins - I actually didn't solve anything. I know my message had a lot of words to skip through, but I mentioned that even before the problems started, SPF/DKIM/DMARC were ALREADY correct. The Workspace technicians I chatted with (and their supervisors) were baffled. But yes, that is in regard to normal mailing situations. Indeed some of the ones that are automated through server notifications (and things like help desk scripts) that may identify as either 0.0.0.0 or no obvious actual sender verify, even if they pass DKIM in some cases, are getting tagged as spam or dangerous by Gmail still. For example - 1 out of every 8 "high 5 minute load alert" messages I receive to an actual POP3 domain account that I have set up in a Gmail account as "Check mail from other accounts" is still getting marked as spam and even some standard server alerts & messages between users on their own domain send email directly to each other (but using Gmail as their POP3 client) are getting marked with the "Caution" banner and suspect as phishing. For no identifiable reason whatsoever. It's going to take a lot of people reporting this situation directly to Google for anything to get done, and even then... expect a long wait.

 

[vikins]

@Merto2 sorry if I missed that. Have you read the ServerFault post?

Gmail SPF fail based on client IP

Gmail is failing SPF check based on the client IP. These are the relevant headers: Received-SPF: fail (google.com: domain of [email protected] does not designate 164.77.240.58 as permitted sender)

serverfault.com

That post concentrates on the specific issue I'm talking about. It can be assumed for this sake of what I'm talking about here that IP reputation, message content, spf / dkim / dmarc, etc are all correct and in compliance. This is more a functional issue with mail originating or being delivered locally on the server and then pulled into gmail using the "Check mail from other accounts" feature. So, we are talking about the same thing in general, but I'm fully confident that I'm not having any other weirdness with gmail marking messages as spam or giving a warning other than in this very specific case. These message *either* end up in spam or are delivered to the inbox with that huge warning message attached. If they end up in spam and I open the message it does not show the yellow warning. If I manually mark the message as "not spam" it is placed into the inbox and in that case it also does not show the yellow warning. The yellow warning only shows up when mail like we've been talking about is delivered directly into the gmail inbox.

 

[Metro2]

Yes indeed, and it is a conundrum to me that Gmail bases it's algorithm on only the IP of the sender even if they're authenticated when the server sends local messages directly to a domain account. It would be wonderful if Google had a support division that was dedicated to working with other providers, but apparently that's only possible if you're already too wealthy to have to work this crap every day like a slave.

 

[Volt55]

This has been happening to me too over the last few months, with communication between my clients and myself on the same server being flagged as Softfail too (using Gmail POP3/SMTP). Very annoying as there seems to be no solution.
Could a possible fix be for cPanel to include an option to never send email locally across accounts on the same server?
What are the repercussions of setting all accounts to 'Remote mail server' in the existing setup?
I have SPF, DKIM and DMARC set up on all accounts.

 

[DigitalComunic]

I appreciate the way people in this thread can contribute to solving the problem.
Based on @vikins' answer, I did tests today by changing the SPF.

First I add the sender's local IP address in the domain's SPF record (domain.com). Didn't work, softfail happens in Gmail.

Second, I add the sender's local IP address to the hostname's SPF record (server.domain.com). It worked, the softfail did not happen.

So for my clients who have fixed IP address at their location, I am putting their IPs in the hostname SPF records. This works as a partial server-side solution to bypass Gmail's SPF checker. Of course, this doesn't solve it for all my clients, as some of them use dynamic IPs, but it minimizes the problem for me at this point until a complete solution appears.