Postfix ignores -o settings in master.cf

Question

I'm configuring a Postfix with three services in master.cf. So far I learned from @anx (thank you so much!) that smtpd-based services in master.cf do not understand transport-related options. Only trivial-rewrite-based services support transport options so a custom trivial-rewrite service needs to be created and then the smtpd-based service can be bound to it with option rewrite_service_name:

smtp    inet    n       -       n       -       -       smtpd
  -o syslog_name=postfix/smtp-in
  -o content_filter=
  -o mynetworks=/etc/postfix/exo_networks
  -o smtpd_client_restrictions=permit_mynetworks,reject
  -o smtpd_recipient_restrictions=reject_non_fqdn_sender,permit_mynetworks,reject
  -o smtp_tls_security_level=encrypt
  -o smtp_tls_mandatory_protocols=>=TLSv1.2
  -o smtp_tls_mandatory_ciphers=high
  -o rewrite_service_name=smtp-in-rewrite

submission      inet    n       -       n       -       -       smtpd
  -o syslog_name=postfix/submission
  -o content_filter=
  -o smtp_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_sasl_type=dovecot
  -o smtpd_sasl_path=private/auth
  -o smtpd_sasl_security_options=noanonymous
  -o smtpd_sasl_tls_security_options=noanonymous
  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
  -o rewrite_service_name=submission-rewrite

2525    inet    n       -       n       -       100     smtpd
  -o content_filter=
  -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks,no_milters
  -o smtpd_helo_restrictions=
  -o smtpd_client_restrictions=
  -o smtpd_sender_restrictions=
  -o smtpd_recipient_restrictions=permit_mynetworks,reject
  -o mynetworks=127.0.0.0/8,10.10.10.0/24,10.10.20.0/24
  -o smtpd_relay_restrictions=
  -o rewrite_service_name=smtp-dmz-rewrite

smtp-in-rewrite   unix  -       -       y       -       -       trivial-rewrite
[...]

submission-rewrite    unix  -       -       y       -       -       trivial-rewrite
  -o syslog_name=postfix/submission-rewrite
# default_transport was a mistake, should be transport_maps
#  -o default_transport=hash:/etc/postfix/transport_to_exo_domains
  -o transport_maps=hash:/etc/postfix/transport_to_exo_domains
  -o sender_dependent_relayhost_maps=hash:/etc/postfix/transport_to_senders_relay
  -o relayhost=[3.3.3.1]

smtp-dmz-rewrite   unix  -       -       y       -       -       trivial-rewrite
[...]

In main.cf I've added a common relayhost to check if overriding it in master.cf works correctly:

relayhost=[4.4.4.1]

# /etc/postfix/transport_to_exo_domains
mydomain.com        relay:[1.1.1.1]
@mydomain.com        relay:[1.1.1.2]
*       relay:[1.1.1.3]

# /etc/postfix/transport_to_senders_relay
@mydomain.com       relay:[2.2.2.1]
mydomain.com relay:[2.2.2.2]
*       relay:[2.2.2.3]

If I sent via the transmission service an email from testa@mydomain.com to testa@mydomain.com, I'd expect the transmission-rewrite service to find a match in transport_to_exo_domains and therefore return 1.1.1.1 or 1.1.1.2 as next hop. That works (now, thx to @anx). But then, already close to the finish line, smtp jumps in and sends the email to the main.cf-relayhost 4.4.4.1 instead of using the rewritten next hop 1.1.1.1. Why does this happen and how can I reliably overwrite main.cf's relayhost? As far as I read the documentation of the smtp command, it doesn't support the relayhost option, so creating a custom smtp service won't work.

Jul  7 23:49:38 localhost postfix/submission/smtpd[100828]: warning: dict_nis_init: NIS domain name not set - NIS lookups disabled
Jul  7 23:49:38 localhost postfix/submission/smtpd[100828]: connect from me.dynamic.kabel-deutschland.de[188.194.X.Y]
Jul  7 23:49:38 localhost postfix/submission/smtpd[100828]: Anonymous TLS connection established from me.dynamic.kabel-deutschland.de[188.194.X.Y]: TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384
Jul  7 23:49:40 localhost postfix/trivial-rewrite[100833]: name_mask: ipv4
Jul  7 23:49:40 localhost postfix/trivial-rewrite[100833]: inet_addr_local: configured 2 IPv4 addresses
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: process generation: 7 (7)
[...]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: dict_open: hash:/etc/postfix/transport_to_exo_domains
[...]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: dict_open: hash:/etc/postfix/transport_to_senders_relay
[...]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: match_list_match: transport_maps: no match
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: maps_find: transport_maps: hash:/etc/postfix/transport_to_exo_domains(0,lock|no_regsub|fold_fix|utf8_request): * = relay:[1.1.1.3]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: wildcard_{chan:hop}={relay:[1.1.1.3]}
[...]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: maps_find: address_verify_transport_maps: hash:/etc/postfix/transport_to_exo_domains(0,lock|no_regsub|fold_fix|utf8_request): * = relay:[1.1.1.3]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: wildcard_{chan:hop}={relay:[1.1.1.3]}
[...]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: submission-rewrite socket: wanted attribute: address
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: input attribute name: address
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: input attribute value: testa@mydomain.com
[...]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: send attr address = testa@mydomain.com
[...]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: submission-rewrite socket: wanted attribute: sender
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: input attribute name: sender
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: input attribute value: (end)
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: submission-rewrite socket: wanted attribute: address
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: input attribute name: address
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: input attribute value: testa@mydomain.com
[...]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: match_list_match: mydomain.com: no match
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: maps_find: sender_dependent_relayhost_maps: "<>": not found
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: maps_find: sender_dependent_relayhost_maps: <>: not found
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: mail_addr_find: <> -> (not found)
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: maps_find: transport_maps: testa@mydomain.com: not found
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: maps_find: transport_maps: hash:/etc/postfix/transport_to_exo_domains(0,lock|no_regsub|fold_fix|utf8_request): mydomain.com = relay:[1.1.1.1]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: mail_addr_find: testa@mydomain.com -> relay:[1.1.1.1]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: `' -> `testa@mydomain.com' -> (`relay' `[1.1.1.1]' `testa@mydomain.com' `4096')
[...]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: send attr transport = relay
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: send attr nexthop = [1.1.1.1]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: send attr recipient = testa@mydomain.com
[...]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: submission-rewrite socket: wanted attribute: address
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: input attribute name: address
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: input attribute value: double-bounce
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: submission-rewrite socket: wanted attribute: (list terminator)
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: input attribute name: (end)
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: `local' `double-bounce' -> `double-bounce@smarthost.mydomain.com'
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: send attr flags = 0
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: send attr address = double-bounce@smarthost.mydomain.com
[...]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: submission-rewrite socket: wanted attribute: address
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: input attribute name: address
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: input attribute value: testa@mydomain.com
[...]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: `local' `testa@mydomain.com' -> `testa@mydomain.com'
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: send attr flags = 0
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: send attr address = testa@mydomain.com
[...]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: submission-rewrite socket: wanted attribute: sender
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: input attribute name: sender
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: input attribute value: testa@mydomain.com
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: submission-rewrite socket: wanted attribute: address
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: input attribute name: address
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: input attribute value: testa@mydomain.com
[...]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: maps_find: sender_dependent_relayhost_maps: testa@mydomain.com: not found
[...]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: match_list_match: mydomain.com: no match
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: maps_find: sender_dependent_relayhost_maps: hash:/etc/postfix/transport_to_senders_relay(0,lock|no_regsub|fold_fix|utf8_request): @mydomain.com = relay:[2.2.2.1]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: mail_addr_find: testa@mydomain.com -> relay:[2.2.2.1]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: maps_find: transport_maps: testa@mydomain.com: not found
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: maps_find: transport_maps: hash:/etc/postfix/transport_to_exo_domains(0,lock|no_regsub|fold_fix|utf8_request): mydomain.com = relay:[1.1.1.1]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: mail_addr_find: testa@mydomain.com -> relay:[1.1.1.1]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: `testa@mydomain.com' -> `testa@mydomain.com' -> (`relay' `[1.1.1.1]' `testa@mydomain.com' `4096')
[...]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: send attr transport = relay
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: send attr nexthop = [1.1.1.1]
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: send attr recipient = testa@mydomain.com
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: send attr flags = 4096
Jul  7 23:49:40 localhost postfix/submission-rewrite/trivial-rewrite[100833]: master_notify: status 1
Jul  7 23:49:40 localhost postfix/submission/smtpd[100828]: 640A34A99A: client=me.dynamic.kabel-deutschland.de[188.194.X.Y], sasl_method=PLAIN, sasl_username=mfp1@mydomain.com
Jul  7 23:49:40 localhost postfix/cleanup[100834]: 640A34A99A: message-id=<eddd4a91-3acf-fb28-a662-ab10c5801db2@mydomain.com>
Jul  7 23:49:40 localhost postfix/qmgr[100822]: 640A34A99A: from=<testa@mydomain.com>, size=860, nrcpt=1 (queue active)
Jul  7 23:49:40 localhost postfix/submission/smtpd[100828]: disconnect from me.dynamic.kabel-deutschland.de[188.194.X.Y] ehlo=2 starttls=1 auth=1 mail=1 rcpt=1 data=1 quit=1 commands=8
Jul  7 23:49:45 localhost postfix/submission-rewrite/trivial-rewrite[100833]: connection closed fd 128
#
# So far everything looks promising now, but then smtp uses the default relayhost from main.cf instead of the correct next hop 1.1.1.1
#
Jul  7 23:50:10 localhost postfix/smtp[100836]: connect to 4.4.4.1[4.4.4.1]:25: Connection timed out
Jul  7 23:50:10 localhost postfix/smtp[100836]: 640A34A99A: to=<testa@mydomain.com>, relay=none, delay=30, delays=0.14/0.01/30/0, dsn=4.4.1, status=deferred (connect to 4.4.4.1[4.4.4.1]:25: Connection timed out)
Jul  7 23:51:25 localhost postfix/submission-rewrite/trivial-rewrite[100833]: idle timeout -- exiting
Jul  7 23:53:00 localhost postfix/anvil[100831]: statistics: max connection rate 1/60s for (submission:188.194.X.Y) at Jul  7 23:49:38
Jul  7 23:53:00 localhost postfix/anvil[100831]: statistics: max connection count 1 for (submission:188.194.X.Y) at Jul  7 23:49:38
Jul  7 23:53:00 localhost postfix/anvil[100831]: statistics: max cache size 1 at Jul  7 23:49:38
Jul  7 23:59:04 localhost postfix/qmgr[100822]: 640A34A99A: from=<testa@mydomain.com>, size=860, nrcpt=1 (queue active)
Jul  7 23:59:34 localhost postfix/smtp[100852]: connect to 4.4.4.1[4.4.4.1]:25: Connection timed out
Jul  7 23:59:34 localhost postfix/smtp[100852]: 640A34A99A: to=<testa@mydomain.com>, relay=none, delay=595, delays=564/0.01/30/0, dsn=4.4.1, status=deferred (connect to 4.4.4.1[4.4.4.1]:25: Connection timed out)

Once up and running, we'll deploy further changes through a git-Ansible-process and there will be a four eye maintenance to avoid misconfiguration. We haven't decided about all monitoring metrics, loop detection and bouncing directions yet.

Thank you!

Answer 1

If you want to override global defaults in master.cf, you have to specify the options for the service that actually processes them.

Refer to the manuals of the individual services to find out which options go where, in this case the trivial-rewrite manual tells us the sender_dependent_default_transport_maps and relayhost features are handled there.

Your smtpd service is ignoring your options just like it would ignore any other global options not meant for the smtpd service: it is talking to a separate service that has been started without these options. Try with these modifications:

# instead of overriding options for the smtpd service, reference a customized variant of the service
[.. ] smtpd
 -o [..]
 -o rewrite_service_name=local-custom-rewrite
# duplicate this service ..
rewrite unix - - y - - trivial-rewrite
# .. under a new name and add your options there
local-custom-rewrite unix - - y - - trivial-rewrite
  -o sender_dependent_default_transport_maps=/etc/postfix/relay_transport_out
[..]

If you search for examples overriding cleanup service options, you will find more analogous configuration using the cleanup_service_name for the case where you want to override options of the service for specific smtpd instances.

---

Postfix services also accept the option -v (or multiple, to increase verbosity further), add that (like the -o options; at least for smtpd and trivial-rewrite) to see the fetched rewrite/resolve results.

Recommend you add options like -o syslog_name=postfix/submission to services started in multiple variations, otherwise it will be difficult to tell from logs which one sent a given log line.

---

If you suspect a problem with the lookup, test it isolated from the whole postfix service configuration, call postmap -q "testa@mydomain.com" hash:/etc/postfix/transport_to_senders_relay (see the TABLE SEARCH ORDER paragraph in the man 5 transport for which queries are tried).

input attribute name: transport input attribute value: hash

I am not entirely certain I understand what you want the end result to behave like, but you need to pass transport:nexthop (or just transport:) to the _transport parameters, and type:table lookup to the *_maps parameters. I understand the log lines as the result of a type confusion, you must have attempted to specify a lookup where a direct transport specification was expected.