[mrgluek]

Fixes the LMTP part of chatmail/relay#1064.

Problem

For multi-recipient deliveries LMTP saves the message for the first recipient, then copies that saved mail (first_saved_mail) for every other recipient so the maildir files can be hardlinked. Delta Chat clients expunge messages right after they fetch them, and with remove-500ms-idle-debounce.patch they are woken up instantly. So the first recipient's copy is often gone before LMTP finishes the list. Every remaining recipient then gets

451 4.2.0 <…> Message was expunged (received-date) (in reply to end of DATA command)

and Postfix defers it. On chatmail.uk this was ~1.4% of all local deliveries (40.7k deferrals in 16 days). Delivery of group messages was delayed by minimal_backoff_time (5 min by default).

Fix

debian/patches/lmtp-retry-expunged-first-saved-mail.patch, src/lmtp/lmtp-local.c:

• When a delivery from first_saved_mail fails with a temporary error and the source mail is expunged, lmtp_local_default_do_deliver() returns without replying.
• lmtp_local_default_deliver() retries that recipient once from local->raw_mail and logs Source mail was expunged, retrying delivery from the raw message.
• lmtp_local_deliver_to_rcpts() then uses raw_mail for the remaining recipients. I adjusted the user cleanup condition for this case: once first_saved_mail exists, recipients that saved from raw_mail still deinit their own mail_user.

Without the race nothing changes, including hardlinking. Other errors (quota, rejects, …) are replied to as before. The cost when the race does happen: the remaining recipients get full copies instead of hardlinks.

The changelog entry bumps to +chatmail3. Change it to whatever version suits you.

Testing

Built for bookworm amd64 (the same patch on top of the 2.3.21+dfsg1-3+chatmail2 release tag, with nocheck) and running on chatmail.uk since 2026-10-22 23:07 UTC. Overnight (23:39 → 06:30 UTC):

• The race happened 33 times, and all 33 recipients were delivered by the retry in the same transaction.
• No postfix Message was expunged deferrals, and no other save failed errors.
• Deferred local deliveries in the UTC hours 00–05 went from 222 (out of 32,405 delivered, unpatched, the night before) to 0 (out of 34,327).

Example transaction:

23:39:09.864  R1 saved                          <- first_saved_mail
23:39:09.927  R2 saved
23:39:09.961  R3 save failed: Message was expunged (received-date)
23:39:09.962  R3 Source mail was expunged, retrying delivery from the raw message
23:39:10.059  R3 saved                          (from raw_mail)
23:39:10.173  R4 saved                          (from raw_mail)
23:39:10.226  R5 saved
postfix: 5 × status=sent, delay=0.38

I haven't added a unit test or run the full CI test suite locally, so this PR's CI run is the first one.

Upstream

Dovecot main (2.4) has the same logic in lmtp-local.c. I'm submitting this here first because relay deploys 2.3.21 from this repository and this exact version is tested in production. It can reach all chatmail relays without waiting for upstream review. A port for dovecot/core main will follow as a separate PR, since the surrounding code differs in 2.4 and needs its own build and test run. I'll link it here.

[mrgluek]

Upstream PR for main (2.4): dovecot/core#338

While building a reproducer for it I found the missing condition: the copy only fails with compression enabled (zlib_save = gz here, mail_compress in 2.4). The hardlink fails with ENOENT, and the fallback copy has to look up the received date. With a compressed stream there's no fd to fstat(), so that lookup does stat() on the file that's already gone, which gives Message was expunged (received-date). Without compression the fallback copies from the still open file and nothing fails, which is probably why this wasn't noticed elsewhere.

The reproducer delivers to 50 recipients while doveadm expunge loops on the first recipient's INBOX, 10 rounds of 50 recipients:

	2.3.21+chatmail2	dovecot/core main
without the patch	482 / 500 deferred	482 / 500 deferred
with the patch	0 deferred	0 deferred

The config and script are in the upstream PR description.