Sendmail: getting mailertable, smarttable and virtusertable to play nice together

Intro
Remember when I posted about some obscure but cool sendmail features? Well in that posting I mentioned having trouble with a catch-all mailertable entry co-existing with a smarttable. I recently got an incentive to work through that and I am sharing my results here. Plus use of virtusertable thrown in for good measure.

The setup
I have a Secure Mail Gateway (SMGW). Some of my users use it, most do not. So for the few who do I wish to divert my outgoing mail, if it sent by one of them, to the SMGW1. Everything else that’s outbound gets forwarded to an Internet-facing relay2. And for inbound mail I also want to divert their email over to it so it can do S/MIME or PGP decryption3. All other inbound emails should go to my native email system4.

So consider the outbound stream, requirement 1. Sender-based routing. That’s the normal thing in sendmail. That’s why you use smarttable, which Andrzej Filip has developed. See my references for more details on that. But my mailertable has a catch-all entry like:

.              relay:emgw.drj.com

I found if I got rid of this entry my smarttable entries began to work, but they stopped working if I put it back. I rolled up my sleeves and tried to understand ruleset 0. I became pretty convinced that mailertable runs before smarttable, and that if mailertable made a successful lookup of the recipie4nt domain that’s it. You’re done. Then I read a very brief comment in Andrzej’s writeup of smarttable. He bascially said mailertable runs first, and I guess form that brief comment you’re supposed to know that this setup I am trying will never work. And yet I have a colleague with a similar setup who says it does work, which got my competitive juices flowing – if the other guy can do it, then I know it can be done and I can do it as well! I knew I had to eliminate the catch-all entry, but still needed a catch-all feature. What to do? Fudge DNS to the point where all TLDs have a fake MX entry pointing at my external mail server?? Sounds too kludgy.

I was reading here and there in the Sendmail book and the cf/README file when SMARTHOST caught my eye. A smarthost can be defined to deliver all email to a specified relay. I always viewed it as an alternative to the mailertable, where you can specify much more specific delivery rules. But maybe they could co-exist? Mailertable for domain-specific delivery instructions, and smarthost for everything else? Yes, you can indeed do that! And with the catchall entry gone from mailertable does smarttable begin to work? Yes! It does! So our outbound stream is in good shape and all requirements are met. Each user of the SMGW is entered in the smarttable:

user1@drj.com    relay:smgw.drj.com
user2@drj.com    relay:smgw.drj.com
etc.

The mc configuration for smarthost looks like this:

dnl smarthost: to take care of the everything-else delivery case - DrJ 2/28/14
define(`SMART_HOST',`internet-facing-relay.drj.com')

Inbound
For inbound we can use virtusertable to rewrite the user domain to a fictional domain, then a mailertable entry which describes that this fictional domain should be routed over to the SMGW! Like this.

Virtusertable

user1@drj.com   user1@drj.com.smgw
user2@drj.com   user2@drj.com.smgw
etc

and

Mailertable

drj.com   relay:native-mail-system-gw.drj.com
.smgw     relay:smgw.drj.com

Although this sounded good on paper, I found it was not enough by itself. In addition I needed to throw in a virtuser domain file which included the domain drj.com.

The mc configuration for virtusertable and virtuserdomain look like this.

dnl
dnl add virtusertable to do recipient re-writing to accomodate SMGW routing -DrJ 3/3/14
FEATURE(`virtusertable',`hash -o /etc/mail/virtusertable')dnl
dnl virtuser domain file location - DrJ 2/27/14
VIRTUSER_DOMAIN_FILE(`/etc/mail/virtuserdomains')dnl

virtuserdomains is a standard text file which contains drj.com.

Testing
It is helpful to know how to debug this stuff. Smarttable is probably the hardest. Here’s how to see the before and after effect that smarttable has:

$ sendmail -Csendmail-test.cf -bt<<END
> 3,0 addr@gmail.com
> .Dfuser1@drj.com
> 3,0 addr@gmail.com
> END

The 3,0 lines show the result of running rulesets 3 and 0, I guess. The .Df…line defines the sender, so the last 3,0 line shows how delivery to the sender is now altered if the defined sender exists in the smarttable table.

Regular addresses can be tested with the same -bt switch if you want to see the gory details, or simply -bv:

$ sendmail -Csendmail-test.cf -bt< 3,0 regularaddr@gmail.com
> END

$ sendmail -Csendmail-test.cf -bv regularaddr@gmail.com

It’s assumed you put your sendmail config in sendmail-test.cf to not interfere with production.

Even if all the tests succeed, what I found is that smarthost did not take effect dynamically. I needed to re-start sendmail.

Conclusion
By digging into the innards of sendmail we learned enough to see how things should work together and found that it is indeed possible for smarttable, virtusertable and mailertable to peacefully co-exist, but only with a helping of smarthost and virtuserdomains!

References
I describe smarttable here.
Andrzej’s smarttable page is here.

This entry was posted in Admin, Internet Mail, Linux and tagged , , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *