Intro
My sendmail binary built for SLES 10 SP 3 was not working well at all on SLES 11 SP1. It became apparent that libraries were not compatible so it was time to re-compile. I’ve documented that journey here. There were a few pitfalls along the way so I felt it was worth a blog post should anyone else ever need to do this.
February 2013 update
And now I’ve repeated the journey on SLES 11 SP2 – and ran into new problems! I’ll put that story in the appendix below.
Why Build?
Why build sendmail when you can find a package for it? For security it’s a good idea to run the latest version. It’s easier to defend during an audit. So when I look via zypper, I see it proposes me sendmail v 8.14.3:
# sudo zypper if sendmail Refreshing service 'nu_novell_com'. Loading repository data... Reading installed packages... Information for package sendmail: Repository: SLES11-SP1-Pool Name: sendmail Version: 8.14.3-50.20.1 Arch: x86_64 Vendor: SUSE LINUX Products GmbH, Nuernberg, Germany ... |
I go to sendmail.org and find that the latest version is actually 8.14.5. And that’s fairly typical. The distributed release is over a year old.
Where this can really matter is when a vulnerability comes out. If you can roll your own you can be the first on the block with that vulnerability fixed – not putting yourself at the mercy of a vendor busy with hundreds of other distractions. And I have seen this phenomenon in action.
Distributing Your Build
I’m mixing up the order here.
Once you have your sendmail built, what’s the minimum set of things you need to put it on other servers?
For one, you gotta have a database package with sendmail. For historical reasons I use sleepycat (I think formerly known as Berkeley db). Only it was gobbled up by Oracle. I don’t have a feel for what the future holds, though I fear decline. Sleepycat provides db. I grabbed the RPM from rpmfind.net:
db-4.7.25-1rt.x86_64.rpm.
This package has to go on each server where you will run sendmail if you are using db for your database. First issue in trying to install this RPM:
# sudo rpm -i db-4.7.25-1rt.x86_64.rpm
file /usr/bin/db_archive from install of db-4.7.25-1rt.x86_64 conflicts with file from package db-utils-4.5.20-95.39.x86_64
file /usr/bin/db_checkpoint from install of db-4.7.25-1rt.x86_64 conflicts with file from package db-utils-4.5.20-95.39.x86_64
file /usr/bin/db_deadlock from install of db-4.7.25-1rt.x86_64 conflicts with file from package db-utils-4.5.20-95.39.x86_64
file /usr/bin/db_dump from install of db-4.7.25-1rt.x86_64 conflicts with file from package db-utils-4.5.20-95.39.x86_64
file /usr/bin/db_hotbackup from install of db-4.7.25-1rt.x86_64 conflicts with file from package db-utils-4.5.20-95.39.x86_64
file /usr/bin/db_load from install of db-4.7.25-1rt.x86_64 conflicts with file from package db-utils-4.5.20-95.39.x86_64
file /usr/bin/db_printlog from install of db-4.7.25-1rt.x86_64 conflicts with file from package db-utils-4.5.20-95.39.x86_64
file /usr/bin/db_recover from install of db-4.7.25-1rt.x86_64 conflicts with file from package db-utils-4.5.20-95.39.x86_64
file /usr/bin/db_stat from install of db-4.7.25-1rt.x86_64 conflicts with file from package db-utils-4.5.20-95.39.x86_64
file /usr/bin/db_upgrade from install of db-4.7.25-1rt.x86_64 conflicts with file from package db-utils-4.5.20-95.39.x86_64
file /usr/bin/db_verify from install of db-4.7.25-1rt.x86_64 conflicts with file from package db-utils-4.5.20-95.39.x86_64 |
I don’t really know if any of these conflicting files are used by the system. I also don’t know how to install “all but the conflicting files” in RPM. So we’ll try our luck and simply overwrite them:
# sudo rpm -i –force db-4.7.25-1rt.x86_64.rpm
Now no errors are reported.
We gotta make another kludge. sendmail, or at least the version I compiled, needs this db library in /usr/lib64, but the RPM puts it in /usr/lib. So…
# cd /usr/lib64; sudo ln -s ../lib/libdb-4.7.so
By the way, how did I decide that db has to be brought over? I fired up sendmail and got this error:
# sendmail sendmail: error while loading shared libraries: libdb-4.7.so: cannot open shared object file: Error 40 |
Now with the sym link made I get a more pleasant application error:
# sendmail can not chdir(/var/spool/clientmqueue/): Permission denied Program mode requires special privileges, e.g., root or TrustedUser. |
Continuing my out-of-order documentation(!), I create an init script in /etc/init.d and make sure sendmail is going to be started at boot:
# sudo chkconfig -s drjohnssendmail 35
I like to put the logs in /maillog:
# sudo rm /var/log/mail; sudo mkdir !$; sudo ln -s !$ /maillog |
I like to have the logging customized a bit, so I modify syslog-ng.conf somewhat:
# DrJ attempt to define filter based on match of sm-mta
filter f_sm-mta { match("sm-mta"); };
filter f_fs-mta { match("fs-mta"); }; |
and
...
#destination mail { file("/var/log/mail"); };
#log { source(src); filter(f_mail); destination(mail); };
destination mailwarn { file("/var/log/mail/mail.warn" perm(0644)); };
log { source(src); filter(f_mailwarn); destination(mailwarn); };
destination mailerr { file("/var/log/mail/mail.err" perm(0644) fsync(yes)); };
log { source(src); filter(f_mailerr); destination(mailerr); };
#
# and also all in one file:
#
#
# and also all in one file:
#
destination mail { file("/var/log/mail/stat.log" perm(0644)); };
log { source(src); filter(f_sm-mta); destination(mail); };
log { source(src); filter(f_fs-mta); destination(mail); }; |
Followed by a
# sudo service syslog restart
Going Back to Compiling
So how did we compile sendmail in the first place, which was supposed to be the subject of this blog?
We downloaded the latest version from sendmail.org and unpacked it. Then we read the INSTALL file in the sendmail-8.14.5 directory for general guidance about the steps.
To make our ocmpilation configuration portable, we try to encapsulate our idiosyncracies in one file, devtools/Site/site.config.m4, which we create. Mine looks as follows:
dnl DrJ config file for corporate mail delivery dnl I am leaving out the ldap stuff because we stopped using it dnl dnl which maps we will support - NEWDB is automatic if it finds the db libs dnl in Linux NDBM is really GDBM, which isn't supported. NEWDB support is not automatic define(`confMAPDEF',`-DNEWDB -DMAP_REGEX') dnl Berkeley DB is here... dnl this doesn't work, exactly - put the db libs directly into /usr/lib APPENDDEF(`confLIBDIRS',`-L/usr/local/ssl/lib -L/usr/lib64') dnl libdb-4 looks like the sleepycat library on Linux APPENDDEF(`confLIBS',`-ldb-4') APPENDDEF(`confINCDIRS',`-I/opt/local/include -I/usr/local/ssl/include') dnl where to put smrsh and mail.local programs define(`confEBINDIR', `/opt/mail/bin') dnl where to install include files define(`confINCLUDEDIR', `/opt/mail/include') dnl where to install library files define(`confLIBDIR', `/opt/mail/lib') dnl man pages define(`confMANROOT', `/opt/local/man/cat') dnl unformatted man pages define(`confMANROOTMAN', `/opt/local/man/man') dnl the sendmail binary goes into MBINDIR define(`confMBINDIR', `/opt/mail/bin') dnl programs only executed by root go to sbin define(`confSBINDIR', `/opt/mail/sbin') dnl shared library directory define(`confSHAREDLIBDIR', `/opt/mail/lib') dnl user-executable pgms, newaliases, mailq, hoststat, etc define(`confUBINDIR', `/opt/mail/bin') dnl TLS support APPENDDEF(`conf_sendmail_ENVDEF', `-DSTARTTLS') APPENDDEF(`conf_sendmail_LIBS', `-lssl -lcrypto') |
I’ll explain a bit of this file since of course it is critical to what we are trying to do. I arrived at its current state from a little experimentation, so I don’t know the full explanation of some of the settings. What it’s saying is that we use the NEWDB, which uses that db we spoke of earlier for our maps. I like to install the binaries into /opt/mail/bin. We like to have the option to run TLS.
With that set we run the compile:
# cd sendmail; sh ./Build
and it spits out some errors to the screen which indicate we’re missing some SSL headers. We get them with:
# sudo zypper source-install openssl
Now with any luck it will fully compile.
At this point it helps to create some of the target directories:
# sudo mkdir -p /opt/mail/{bin,sbin} /opt/local/man/cat{1,5,8}
And we create a user account, smmsp, with uid and gid of 225, and a group with the same name.
And then we can install it:
# sudo sh ./Build install
The install should work, mostly. But makemap doesn’t get put in /opt/mail for some reason. So you have to copy it by hand from sendmail-8.14.5/obj.Linux.2.6.32.12-0.7-default.x86_64/makemap to /opt/mail/sbin, for instance. You really need to have a makemap.
Finally, I suggest to recursively copy the cf directory:
# cp -r sendmail-8.14.5/cf /opt/mail
This way you have a pretty relocatable set of files under /opt/mail.
Appendix: Building on SLES 11 SP2
I thought this would be a breeze. Just look at my own blog posting above! Not so fast – that approach didn’t work at all.
You have to appreciate that under SLES 11 SP1 we needed a few key packages that aren’t very common:
– libopenssl-devel-0.9.8h-30.27.11
– zlib-devel-1.2.3-106.34
We pulled them from the SDK DVD. Well, turns out there is no SDK DVD for SLES 11 SP2! Novell, probably in one of those beloved cost-saving measures, no longer releases an SDK DVD. What to do?
Well, I found what not to do. I began copying key files from my SLES 11 SP1 installation, like libcrypto.a, libssl.a, /usr/include/ssl. This all helped to reduce the number of errors. But at the end of the day there was an error I couldn’t chase away no matter what:
/usr/src/packages/BUILD/openssl-0.9.8h/crypto/comp/c_zlib.c:235: undefined reference to `inflate' |
Meanwhile the resourceful sysadmin found those development packages. He told me about SUSE Studio, which allows you to build your own distribution. He looked for those development packages in the distribution, found and installed them:
$ rpm -qa|grep devel
libopenssl-devel-0.9.8j-0.26.1 zlib-devel-1.2.3-106.34 zlib-devel-32bit-1.2.3-106.34 |
Then my build went through fine. Whew!
Conclusion
I could go on and on about details in the setup, but the scope here is the compilation, and we’ve covered that pretty well.
Once you get over the pain of compilation setup, sendmail runs great and is a great MTA.