smtp delivery error not shown

[csb0730]

Added a new chat with new email address manally. Sent first message and got no response.
If I examine INBOX by other email app later there is a mail with subject "Mail delivery failed: returning message to sender".
Into that mail wrong "to: address" is shown and error reason is described.
DC doesn't show anything about that error to user. User is not aware that sent message never arrived!

DC should show every error message somehow to user. In best case error message is shown into chat to which it belongs to.

[r10s]

thanks for the post, yes, this could be improved.

did the delivery report pop up in "contact requests"? (which is a rather bad name in this context, e-mail would be much better here)

had the idea of parsing the reports and marking the messages as failed or so in #39

afaik there are also servers that give feedback directly when sending is not possible by them (seen this on my server but did not investigate in that)

[csb0730]

Unfortunately even in Contact Requests there is no message to see.
But in logring-buffer I find a hint:

xx:xx:15 Receiving message INBOX/18384
xx:xx:15 Cannot create Message-ID
xx:xx:15 1 mails read from "INBOX"
xx:xx:15 0 mails read from "INBOX"

All in one second and time is matching to the error message. Maybe this helps to find why even in Contact Requests nothing is shown?

It would help or even sufficient to show these error messages in Contact Requests in full text. So user can see what's going wrong (+ alert of new mail in Contact Request !).

[adbenitez]

My memory sucks, but I think I have already seen error emails from servers on contact request view, but well were redirected from a gmail account to my account...

[csb0730]

I examined the error message and found no Message ID header in it. Maybe a reason for not handling it correctly?

Here the anonymized original .eml file.

Mail delivery failed: returning message to sender-anonymisiert.eml.txt

[r10s]

just tried to send a message to an address of the german telekom - quota exceeded :)

unfortunately, the mail is not easily assignable to the failed message, however. the german telekon unfortunately does not use a standard delivery report but just adds a X-Failed-Recipients header plus some not standard-structured information in the body.

but, also here, the subject starts with Mail delivery failed, so this could be used as a strong hint indeed.

[csb0730]

I have next incident. Provider is "t-online.de".
Added a new chat manually with new email address. Provider sends a reply with subject "Mail delivery failed ..."
In body the text

The following address(es)
failed:

[email protected]:
SMTP error from remote server for RCPT TO command, host: mx01.t-online.de (194.25.134.72) reason: 550-5.1.1 user unknown
550 5.1.1 Unknown recipient.

is shown.

This message is not shown in contact requests!

Headers:

Return-Path: <>
Received: from mout-bounce.gmx.net ([212.227.15.46]) by mx-ha.gmx.net
 (mxgmx115 [212.227.17.5]) with ESMTPS (Nemesis) id 1N2lBU-1gxy5u0JJ6-0130VP
 for <[email protected]>; Fri, 29 Mar 2019 15:51:22 +0100
Received: from localhost by mout-bounce.gmx.net id 0MD8qC-1hAwdx49LY-00GXuq
    Fri, 29 Mar 2019 15:51:21 +0100
Date: Fri, 29 Mar 2019 15:51:21 +0100
From: "GMX Mailer Daemon" <[email protected]>
To: [email protected]
Subject: Mail delivery failed: returning message to sender
Auto-Submitted: auto-replied
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-UI-Out-Filterresults: unknown:0;V03:K0:LpHJWJTvsi8=:UafFcV5jS+P2bOv6ZwnNdg
 ==
Envelope-To: <[email protected]>
X-GMX-Antispam: 0 (Mail was not recognized as spam); Detail=V3;
X-Spam-Flag: NO
...

[csb0730]

still existant in v0.200.0, core 0-42-0-dev

[csb0730]

If user setting "Show all emails in contact requests" is activated we should see this error message.
But it doesn't.

[testbird]

In your example cases, would these checks allow to match the error-message to a sent-message?:

https://support.delta.chat/t/make-contact-requests-more-visible/196/41

[csb0730]

From: contains text "Mailer" and "Daemon" ==> ok
Subject: contains "failed" ==> ok

And in body somewhere you should see "smtp" and "error" and an email address which is known.

As r10s mentioned: There are some reply formats which are not standard so a common detection is somewhat difficult, but not impossible :-)

But nevertheless: If "show all emails" is selected I should see all messages from inbox. So even the error messages should appear. But this is currently not the case.

[testbird]

The actual tests from the linked post, to identify the affected message:

• sender: From: MAILER-DAEMON@
• And the text body contains at least this header data from the affected, original message on different lines:
  • From:
  • To:
  • Subject:
  • Date:

Which should already be sufficient to associate it to a affected, recently sent message.

Only some failure receipts do additionally contain:

• an X-Failed-Recipients: header.
• or contain the original Message-ID in the text body

[csb0730]

@testbird
We should not come to far away from real issue: see #673.
I think that's it (!)

[testbird]

I think #673 would make errors appear as an email (old "contact requests") only for those that have enabled this, and not show any error (state, symbol or icon) for sent but failed chat messages.

But "smtp delivery error not shown" will require and should work based on the 5 filter rules to actually show an error for failed chat messages.

[csb0730]

For me it would be ok to see it in "contact requests".

But ok, the ideal way for most users would be to see it in the causing chat or so ...

[csb0730]

Core v0.43.0
Option "show all emails" is on.

I have a next delivery error from arcor.de ==> Message is shown.
Header "Message-Id" is existing.

Next delivery error from aol.com ==> Message NOT shown
Header "Message-ID" NOT existing.

It seems that many providers are not sending a message Id in error messages and DC
will suppress all mails which do not have a message Id in headers? @r10s ?

[csb0730]

"Cannot create Message-ID" is the important information.

See void dc_receive_imf(...)

if (rfc724_mid==NULL) {
				rfc724_mid = dc_create_incoming_rfc724_mid(sent_timestamp, from_id, to_ids);
				if (rfc724_mid==NULL) {
					dc_log_info(context, 0, "Cannot create Message-ID.");
					goto cleanup;
				}
			}

If no message ID is created all further workflow of this message is stopped and is not shown.

I think that's it.

[csb0730]

dc_create_incoming_rfc724_mid(sent_timestamp, from_id, to_ids) is the reason for this issue.
If the parameter to_ids is NULL, this function is not creating a Message-ID. This stops any further handling and prevents DC to show the message.
I adapted the function as follows and now it works :-)
The else part is the interesting one.

	size_t   i = 0;
	size_t   icnt = dc_array_get_cnt(contact_ids_to);
	uint32_t largest_id_to = 0;

	if (contact_ids_to!=NULL && dc_array_get_cnt(contact_ids_to)!=0) {
		/* cs: only do standard procedure if contact_ids_to are set */
		for (i = 0; i < icnt; i++) {
			uint32_t cur_id = dc_array_get_id(contact_ids_to, i);
			if (cur_id > largest_id_to) {
				largest_id_to = cur_id;
			}
		}
	}
	else {
		/* cs: set var to a reasonable value to build Message-ID.
		 * current timestamp should be secure enough (in the moment!) */
		largest_id_to = (unsigned long)time(NULL);
	}

@r10s maybe this part should be updated in rust core in a similar or same way too ?!