Categories
Apache CentOS Hosting Service Web Site Technologies

Compiling Apache 2.4 on CentOS

Intro
This is a tale of one thing leading to another. I’ll probably either continue this post or delete it altogether if I find I’m headed down a wrong path.

The details
I suspect that to get better marks for my server’s SSL implementation I probably need apache 2.4. There is an RPM for apache 2.4 but it is almost two years old! So I decided to bite the bullet and compile the darn thing myself. Easier said than done. My current production version is 2.2.15.

Now if you just want to compile a recent version of apache 2.4 then this guide is much, much better than mine: https://jasonpowell42.wordpress.com/2013/04/05/install-apache-2-4-4-on-centos-6-4/. My guide, where I’ve hit just about every conceivable error and powered through, is more for timid folks like me who want to keep their current apache 2.2 running while trying 2.4. In spite of what you read elsewhere this is possible to do, but you need patience and perseverance.

Getting the source is easy enough. Then you configure it:

httpd-2.4.16$ ./configure −−prefix=/usr/local/apache24

checking for chosen layout... Apache
checking for working mkdir -p... yes
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
configure:
configure: Configuring Apache Portable Runtime library...
configure:
checking for APR... configure: WARNING: APR version 1.4.0 or later is required, found 1.3.9
configure: WARNING: skipped APR at apr-1-config, version not acceptable
no
configure: error: APR not found.  Please read the documentation.

What version of apr do we have?

$ sudo rpm −qa|grep apr

apr-util-devel-1.3.9-3.el6_0.1.x86_64
apr-util-1.3.9-3.el6_0.1.x86_64
apr-util-ldap-1.3.9-3.el6_0.1.x86_64
apr-1.3.9-5.el6_2.x86_64
apr-devel-1.3.9-5.el6_2.x86_64

Drat. No wonder we’re having trouble. Guess we could compile apr ourselves, but perhaps there’s a suitable version out there somewhere we can simply download?


Warning: this approach to apr shown below was a dead end for me. Further down I show a successful approach.

$ sudo yum search apr

Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: linux.cc.lehigh.edu
 * epel: mirror.us.leaseweb.net
 * extras: mirror.rackspace.com
 * updates: mirror.es.its.nyu.edu
========================================================== N/S Matched: apr ==========================================================
...
httpd24-apr-debuginfo.x86_64 : Debug information for package httpd24-apr
httpd24-apr-devel.x86_64 : APR library development kit
httpd24-apr-util-debuginfo.x86_64 : Debug information for package
                                  : httpd24-apr-util
httpd24-apr-util-devel.x86_64 : APR utility library development kit
httpd24-apr-util-ldap.x86_64 : APR utility library LDAP support
httpd24-apr-util-mysql.x86_64 : APR utility library MySQL DBD driver
httpd24-apr-util-nss.x86_64 : APR utility library NSS crytpo support
httpd24-apr-util-odbc.x86_64 : APR utility library ODBC DBD driver
httpd24-apr-util-openssl.x86_64 : APR utility library OpenSSL crytpo support
httpd24-apr-util-pgsql.x86_64 : APR utility library PostgreSQL DBD driver
httpd24-apr-util-sqlite.x86_64 : APR utility library SQLite DBD driver
httpd24-apr.x86_64 : Apache Portable Runtime library
httpd24-apr-util.x86_64 : Apache Portable Runtime Utility library
...

I singled out the promising looking ones. After all it’s apache 2.4 that’s driving the need for this version so the httpd24 versions of apr should suffice.

So I installed these:

$ sudo yum install httpd24-apr-util.x86_64
$ sudo yum install httpd24-apr-util-devel.x86_64

Now how do we tell the configurator where our new apr package is?

httpd-2.4.16$ ./configure −−help|grep −i apr

  --enable-hook-probes    Enable APR hook probes
  --with-included-apr     Use bundled copies of APR/APR-Util
  --with-apr=PATH         prefix for installed APR or the full path to
                             apr-config
  --with-apr-util=PATH    prefix for installed APU or the full path to

The with-apr switch looks promising. Now we guess as to exactly what we should put for the path. Here’s what happens when we guess wrong:

httpd-2.4.16$ ./configure −−with-apr=/opt/rh/httpd24/root/usr/lib64 −−prefix=/usr/local/apache24

checking for chosen layout... Apache
checking for working mkdir -p... yes
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
configure:
configure: Configuring Apache Portable Runtime library...
configure:
checking for APR... configure: error: the --with-apr parameter is incorrect. It must specify an install prefix, a build directory, or an apr-config file.

I’ll spare you the guesswork. Here is the path correctly specified:

httpd-2.4.16$ ./configure −−with-apr=/opt/rh/httpd24/root/usr −−prefix=/usr/local/apache24

...
configure: Configuring Apache Portable Runtime Utility library...
configure:
checking for APR-util... yes
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... gcc -E
checking for gcc option to accept ISO C99... -std=gnu99
checking for pcre-config... false
configure: error: pcre-config for libpcre not found. PCRE is required and available from http://pcre.org/

So we finally got past the apr error and are onto the next one : (. I’ll try to install pcre-devel to see if that helps:

$ sudo yum install pcre-devel.x86_64

Wow! Got lucky that time. That cleared up that error and the configure went all the way through!

Oh, no. It doesn’t compile! It begins to, but it can’t compile export.c:

httpd-2.4.16$ make

...
gawk -f /usr/local/src/apache24/httpd-2.4.16/build/make_exports.awk `cat export_files` > exports.c
/usr/lib64/apr-1/build/libtool --silent --mode=compile gcc -std=gnu99  -pthread      -DLINUX=2 -D_REENTRANT -D_GNU_SOURCE     -I. -I/usr/local/src/apache24/httpd-2.4.16/os/unix -I/usr/local/src/apache24/httpd-2.4.16/include -I/opt/rh/httpd24/root/usr/include/apr-1 -I/usr/include/apr-1 -I/usr/local/src/apache24/httpd-2.4.16/modules/aaa -I/usr/local/src/apache24/httpd-2.4.16/modules/cache -I/usr/local/src/apache24/httpd-2.4.16/modules/core -I/usr/local/src/apache24/httpd-2.4.16/modules/database -I/usr/local/src/apache24/httpd-2.4.16/modules/filters -I/usr/local/src/apache24/httpd-2.4.16/modules/ldap -I/usr/local/src/apache24/httpd-2.4.16/modules/loggers -I/usr/local/src/apache24/httpd-2.4.16/modules/lua -I/usr/local/src/apache24/httpd-2.4.16/modules/proxy -I/usr/local/src/apache24/httpd-2.4.16/modules/session -I/usr/local/src/apache24/httpd-2.4.16/modules/ssl -I/usr/local/src/apache24/httpd-2.4.16/modules/test -I/usr/local/src/apache24/httpd-2.4.16/server -I/usr/local/src/apache24/httpd-2.4.16/modules/arch/unix -I/usr/local/src/apache24/httpd-2.4.16/modules/dav/main -I/usr/local/src/apache24/httpd-2.4.16/modules/generators -I/usr/local/src/apache24/httpd-2.4.16/modules/mappers  -prefer-non-pic -static -c exports.c && touch exports.lo
exports.c:1244: error: redefinition of ‘ap_hack_apr_allocator_create’
exports.c:198: note: previous definition of ‘ap_hack_apr_allocator_create’ was here
exports.c:1245: error: redefinition of ‘ap_hack_apr_allocator_destroy’
exports.c:199: note: previous definition of ‘ap_hack_apr_allocator_destroy’ was here
exports.c:1246: error: redefinition of ‘ap_hack_apr_allocator_alloc’
exports.c:200: note: previous definition of ‘ap_hack_apr_allocator_alloc’ was here
exports.c:1247: error: redefinition of ‘ap_hack_apr_allocator_free’
exports.c:201: note: previous definition of ‘ap_hack_apr_allocator_free’ was here
exports.c:1248: error: redefinition of ‘ap_hack_apr_allocator_owner_set’

This could be tough! Maybe impossible for me to get past. I’ve never encountered this kind of error. OK. Got it. Not so tough. I had two versions of apr installed – the old one needed by my apache 2.2 and the new one installed as shown above. I didn’t want to completely blow away the old one as I feared that it is dynamically linked by Apache 2.2, so I did the following:

$ cd /usr/lib64; sudo mv apr-1 drjapr-1
– then change to my apache24 root directory and run configure again; then run make

And it went through this time!

Only modules installed
make install however only installed modules, not the httpd binary.

The problem seems related to my original apr libraries. They look like this:

$ sudo rpm −qa|grep ^apr

apr-util-devel-1.3.9-3.el6_0.1.x86_64
apr-util-1.3.9-3.el6_0.1.x86_64
apr-util-ldap-1.3.9-3.el6_0.1.x86_64
apr-1.3.9-5.el6_2.x86_64
apr-devel-1.3.9-5.el6_2.x86_64

I tried to move them all to a temporary directory but then the compiler cannot find libtool which is normally supplied by apr-devel.

I considered removing apr-devel, but boy there are so many dependencies that my other packages have on it that I did not feel comfortable doing that. PHP, apache2.2 and a whole lot more depend on it.

End of dead end approach to apr


New approach needed
My new approach is to try to use the APR from apache itself by downloaded the Unix sources for APR and apr-util from http://apr.apache.org/download.cgi. Yes, this worked best of all. I even put back all the apr files I had moved in the previous failed effort.

It’s not very clear what they mean by unpacking apr and apr-util in srclib. I created symlinks in my srclib directory such that apr -> apr-1.5.2 and apr-util -> apr-util-1.5.4. For the inexperienced the command format is like in this example:

$ ln −s apr-1.5.2 apr

Of course you first have to download the source tarball to your srclib directory and unpack it:

$ tar zxf apr-1.5.2.tar.gz

It Compiles and Installs
So after all those misfires I finally got a version that compiled and installed in its entirety. That process starts with this configure command:

$ ./configure −−with-included-apr −−prefix=/usr/local/apache24

Then the usual make and sudo make install.

Modules problem
I inherited a configuration that had a mods-avalable and a mods-enabled directory which is how my old apache 2.2 was set up. After tweaking the modules path using the replace command, something like this

$ cd /etc; cp −pr apache2 apache24; cd mods-avalable
$ sudo replace /usr/lib/apache2 /usr/local/apache24 −− *.load

I still could not start my new server:

Starting apache24: httpd: Syntax error on line 203 of /etc/apache24/apache24.conf: Syntax error on line 1 of /etc/apache24/mods-enabled/authz_default.load: Cannot load /usr/local/apache24/modules/mod_authz_default.so into server: /usr/local/apache24/modules/mod_authz_default.so: cannot open shared object file: No such file or directory
                                                           [FAILED]

I looked at all my configuration files and don’t see anything that relies on this module so I deleted the reference to it in mods-enabled.

Starting apache24: httpd: Syntax error on line 203 of /etc/apache24/apache24.conf: Syntax error on line 1 of /etc/apache24/mods-enabled/cgi.load: Cannot load /usr/local/apache24/modules/mod_cgi.so into server: /usr/local/apache24/modules/mod_cgi.so: cannot open shared object file: No such file or directory
                                                           [FAILED]

Now I do like to run CGI programs on occasion so this one can’t be so easily brushed aside. It could be that we should be using mod_cgid.so instead.

Then it’s onto this error:

Starting apache24: httpd: Syntax error on line 203 of /etc/apache24/apache24.conf: Syntax error on line 1 of /etc/apache24/mods-enabled/php5.load: Cannot load /usr/local/apache24/modules/libphp5.so into server: /usr/local/apache24/modules/libphp5.so: cannot open shared object file: No such file or directory
                                                           [FAILED]

I use php so I may have to investigate this one in some detail. simply trying to update the link to where the old libphp5.so resides under apache2.2 brings up this different kind of error:

Starting apache24: httpd: Syntax error on line 203 of /etc/apache24/apache24.conf: Syntax error on line 1 of /etc/apache24/mods-enabled/php5.load: Cannot load /usr/lib/apache2/modules/libphp5.so into server: /usr/lib/apache2/modules/libphp5.so: undefined symbol: unixd_config
                                                           [FAILED]

Wow. I’m reading various things and it looks like I’ll now have to compile php5 as well. This is getting hairy. This site, although old, seems to explain it most clearly. And of course I’ve got php 5.3 which you can’t even find source for on the php web site, www.php.net

So I downloaded php5.4.43, which is the oldest one I could find on the php web site!

To configure it I used this long list of options, some of which are determined by my choices of location for my apache24 files:

$ ./configure ‐‐with‐apxs2=/usr/local/apache24/bin/apxs ‐‐with‐mysql ‐‐prefix=/usr/local/apache24/php5 ‐‐with‐config‐file ‐path=/usr/local/apache24/php5 ‐‐disable‐cgi ‐‐with‐zlib ‐‐with‐gettext ‐‐with‐gdbm ‐‐with‐curl ‐‐with‐openssl

2017 update for php
I finally needed to update some WordPress packages and found my only transport is ftp. I think my command-line compile options for php5 above leave something to be desired. I think I need to add curl and openssl like so:

$ ./configure ‐‐with‐apxs2=/usr/local/apache24/bin/apxs ‐‐with‐mysql ‐‐prefix=/usr/local/apache24/php5 ‐‐with‐config‐file ‐path=/usr/local/apache24/php5 ‐‐disable‐cgi ‐‐with‐zlib ‐‐with‐gettext ‐‐with‐gdbm ‐‐with‐curl ‐‐with‐openssl

but I get these errors:

ext/curl/.libs/interface.o: In function `php_curl_option_url':
/usr/local/src/php5/php-5.4.43/ext/curl/interface.c:180: undefined reference to `core_globals'
ext/curl/.libs/interface.o: In function `_php_curl_setopt':
/usr/local/src/php5/php-5.4.43/ext/curl/interface.c:1821: undefined reference to `core_globals'
/usr/local/src/php5/php-5.4.43/ext/curl/interface.c:1804: undefined reference to `core_globals'
ext/curl/.libs/interface.o: In function `curl_progress':
/usr/local/src/php5/php-5.4.43/ext/curl/interface.c:1113: undefined reference to `executor_globals'
ext/curl/.libs/interface.o: In function `curl_write_header':
/usr/local/src/php5/php-5.4.43/ext/curl/interface.c:1264: undefined reference to `executor_globals'
ext/curl/.libs/interface.o: In function `curl_write':
/usr/local/src/php5/php-5.4.43/ext/curl/interface.c:1038: undefined reference to `executor_globals'
ext/curl/.libs/interface.o: In function `curl_read':
/usr/local/src/php5/php-5.4.43/ext/curl/interface.c:1187: undefined reference to `executor_globals'
ext/curl/.libs/streams.o: In function `php_curl_stream_opener':
/usr/local/src/php5/php-5.4.43/ext/curl/streams.c:320: undefined reference to `file_globals'
/usr/local/src/php5/php-5.4.43/ext/curl/streams.c:406: undefined reference to `core_globals'
/usr/local/src/php5/php-5.4.43/ext/curl/streams.c:414: undefined reference to `core_globals'
ext/curl/.libs/streams.o: In function `on_data_available':
/usr/local/src/php5/php-5.4.43/ext/curl/streams.c:68: undefined reference to `executor_globals'
ext/standard/.libs/info.o: In function `php_info_print_request_uri':
/usr/local/src/php5/php-5.4.43/ext/standard/info.c:97: undefined reference to `sapi_globals'
ext/standard/.libs/info.o: In function `php_print_gpcse_array':
/usr/local/src/php5/php-5.4.43/ext/standard/info.c:213: undefined reference to `executor_globals'
ext/standard/.libs/info.o: In function `php_print_info':
/usr/local/src/php5/php-5.4.43/ext/standard/info.c:918: undefined reference to `executor_globals'
collect2: ld returned 1 exit status
make: *** [sapi/cli/php] Error 1

Here the problem seems to be that since I had already compiled php5 and left it around, it was using the old parts.

You need to do a make clean first! Then it compiles.

Now I’m down to this apache error:

$ sudo service apache24 start

Starting apache24: AH00526: Syntax error on line 55 of /etc/apache24/apache24.conf:
Invalid command 'LockFile', perhaps misspelled or defined by a module not included in the server configuration
                                                           [FAILED]

I’m going to just try to comment out that pesky Option LockFile…. I’ve found this apache page which is helpful for this upgrade: http://httpd.apache.org/docs/trunk/upgrading.html OK, next error:

Starting apache24: AH00526: Syntax error on line 145 of /etc/apache24/apache24.conf:
Invalid command 'User', perhaps misspelled or defined by a module not included in the server configuration
                                                           [FAILED]

Here the advice is to load module mod_unixd. I don’t even have anything like that so I’m looking into it now. OK. It’s in the apache24/modules so I just need to load it in. Next error:

Starting apache24: AH00526: Syntax error on line 161 of /etc/apache24/apache24.conf:
Invalid command 'Order', perhaps misspelled or defined by a module not included in the server configuration

Wow. That comes from this pretty standard line:

<Files ~ "^\.ht">
    Order allow,deny
    Deny from all
    Satisfy all
</Files>

This is a helpful document: http://httpd.apache.org/docs/trunk/upgrading.html. So at their recommendation I replaced all that with a

Require all denied

That leads to the next error:

Starting apache24: AH00526: Syntax error on line 166 of /etc/apache24/apache24.conf:
Invalid command 'Require', perhaps misspelled or defined by a module not included in the server configuration

It means Require is not even found. I needed to load some new modules, names authz_code and unixd:

LoadModule authz_core_module /usr/local/apache24/modules/mod_authz_core.so
LoadModule unixd_module /usr/local/apache24/modules/mod_unixd.so

Next error:

AH00526: Syntax error on line 20 of /etc/apache24/mods-enabled/alias.conf:
Invalid command 'Order', perhaps misspelled or defined by a module not included in the server configuration

so some of my old conf files that I copied over use the old syntax. The alias.conf file looked like this:

Alias /icons/ "/var/www/icons/"
 
<Directory "/var/www/icons">
    Options Indexes MultiViews
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>

Again looking at http://httpd.apache.org/docs/trunk/upgrading.html they suggest to replace the Order… and following line with:

Require all granted

Next error:

AH00526: Syntax error on line 3 of /etc/apache24/mods-enabled/deflate.conf:
Invalid command 'AddOutputFilterByType', perhaps misspelled or defined by a module not included in the server configuration
                                                           [FAILED]

But I was already loading the deflate module which defines AddOutputfilterByType. What I learned is that in apache 2.4 you also need to load mod_filter.

And the next error please:

AH00526: Syntax error on line 43 of /etc/apache24/mods-enabled/ssl.conf:
SSLSessionCache: 'shmcb' session cache not supported (known names: ). Maybe you need to load the appropriate socache module (mod_socache_shmcb?).

That’s in complaint about this line:

SSLSessionCache        shmcb:${APACHE_RUN_DIR}/ssl_scache(512000)

The standard advice for this error is to uncomment this line:

LoadModule socache_shmcb_module modules/mod_socache_shmcb.so

But I don’t have that module!

I guess I chose the wrong options when doing the initial ./configure. See the references for a proper guide that lists some good options.

I’m now trying to configure like this:

$ ./configure −−with-included-apr −−prefix=/usr/local/apache24 −−enable-php5 −−enable-so −−enable-ssl −−with-mpm=prefork

Actually I don’t know if I needed all those options such as enable-ssl. The main thing was that my apache 2.2 mods-available directory didn’t have a mention of mod_socache_shmcb.so. My apache 2.4 built with these config options definitely does. so I just need one of these LoadModule statements like this:

LoadModule socache_shmcb_module /usr/local/apache24/modules/mod_socache_shmcb.so

Well we’ve moved six lines down into that config file. I guess that’s progress! because now we’ve made it all the wy to line 49:

AH00526: Syntax error on line 49 of /etc/apache24/mods-enabled/ssl.conf:
Invalid command 'SSLMutex', perhaps misspelled or defined by a module not included in the server configuration
                                                           [FAILED]

Even apache’s upgrade guide documents this error. It’s caused by a conf file line that looks something like this:

SSLMutex  file:${APACHE_RUN_DIR}/ssl_mutex

and they say – I’m paraphrasing here – just try to comment it out and hope for the best.

Next error:

AH00526: Syntax error on line 9 of /etc/apache24/mods-enabled/status.conf:
Invalid command 'Order', perhaps misspelled or defined by a module not included in the server configuration
                                                           [FAILED]

Yeah status.conf has

    Order deny,allow
    Deny from all
    Allow from 127.0.0.1 ::1

We’ll try to replace that with this:

Require host 127.0.0.1 ::1

Now it runs through all the configuration OK but doesn’t actually start. I had set up an init.d script and I wasn’t going to go into this but I may have to:

$ sudo service apache24 start

httpd (pid 30896) already running

Remember I am trying to run this while still running the old apache 2.2 server. Process 30896 is the old apache 2.2:

root     30896     1  0 10:05 ?        00:00:00 /usr/sbin/httpd -d /etc/apache2 -f apache2.conf

This results from the byzantine way I set up to launch apache. There is a /etc/sysconfig/apache24 which doesn’t do much other than import environment variable definitions from /etc/apache24/envvars, except I had forgotten to update that path so it pointed to the old /etc/apache2/envvars.

Now it starts! But not without complaint:

Starting apache24: [Thu Aug 06 11:18:04.711658 2015] [core:warn] [pid 22911] AH00117: Ignoring deprecated use of DefaultType in line 178 of /etc/apache24/apache24.conf.
                                                              [  OK  ]

That stems from this line which tries to establish a default MIME type:

DefaultType text/plain

I also notice I cannot really get the status of my new web server:

$ sudo service apache24 status

httpd dead but subsys locked

So stopping/starting doesn’t really work either once it’s started.

What I found is that it seems happier if I have a line in /etc/sysconfig/apache24 which has an explicit PIDFILE defined – I use PIDFILE=/var/run/apache24.pid – with the same filepath as is mentioned in the apache24.conf file, where I have PidFile ${APACHE_PID_FILE} where APACHE_PID_FILE is taken from my envvars and has the value /var/run/apache24.pid. OK, my setup is very convoluted and probably unique. But the problem is common on CentOS so the main takeway is to have consistent reference to the pidfile filepath in /etc/sysconfig/httpd or whatever you are calling it as in your main config file httpd.conf or whatever you are calling it.

Home page test (I’m running on port 1443 to avoid conflict with my production server):

$ curl −i −k https://127.0.0.1:1443/

HTTP/1.1 301 Moved Permanently
Date: Wed, 05 Aug 2015 18:33:19 GMT
Server: Apache/2
X-Powered-By: PHP/5.4.43
Location: https://drjohnstechtalk.com/blog/
Content-Length: 2
Content-Type: text/html

So that looks pretty good.

A simple php test:

$ curl −i −k https://127.0.0.1:1443/phpinfo.php

Long output. Basically looks right.

OK. What about the opening WordPress page?

$ curl −i −H ‘Host: drjohnstechtalk.com’ −k https://127.0.0.1:1443/blog/

Yes. Big long output. Looks good. I don’t think this proves that the mySQL/php interface is really working however as that page could be cached since I use a pagecache plugin.

Next test I’d like to run is the Qualys SSLLabs test, but it won’t run on port 1443. Maybe the DigiCERT test will. Yes, it does allow it. And I no longer have the BREACH vulnerability.

A few words about a BREACH test
This prompted me to look at why Digicert felt I was vulnerable to BREACH in the first place. I thnk it’s related to serving compressed objects. So I thought of this simple test. Against my apache 2.2 I can run a query like this:

$ curl −i −k −−compress https://127.0.0.1:1443/blog/|head −10

Date: Fri, 07 Aug 2015 14:02:48 GMT
Server: Apache/2
X-Powered-By: PHP/5.3.3
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 30414
Content-Type: text/html
 
<!DOCTYPE html>
<html lang="en-US">

See that Content-Encoding: gzip? Yet the actual content that begins <!DOCTYPE html… is in plain text and plainly not compressed. So I really wasn’t vulnerable to BREACH at all. The server claimed to be compressing the pages it was sendnig to the browser but in reality it wasn’t. For apache 2.4 the behaviour is basically the same except there is no response header Content-Encoding: gzip returned. This is why it passes Digicert’s BREACH test with flying colors.

Moving on
Next test. Swap apache 2.2 for apache 2.4 by changing listening ports 443 for 1443. Then do the SSLlabs test. I now get an A. well, actually I get an A both before and after the swap.

WordPress test
I’m writing this using my new shiny apache 2.4. With regards to WordPress it all seems to feel the same as before. One small thing I’ve noticed is that I don’t get WordPress news any longer:

RSS Error: WP HTTP Error: There are no HTTP transports available which can complete the requested request.

Hopefully there’s nothing more serious.

php.ini missing
If you blindly copied my config options for compiling php then sooner or later (much later in my case) you’ll realize that you have no valid php.ini file! You will see an error like this when the date() function is called:

Warning: date(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone. in

So because I used the config option –with-config-fil
e-path=/usr/local/apache24/php5 I needed to put a php.ini file in that directory and only that directory. For now its contents are:

; DrJ, inspired by http://stackoverflow.com/questions/2184513/php-change-the-maximum-upload-file-size - 12/31/14
; Maximum allowed size for uploaded files.
upload_max_filesize = 10M
 
; Must be greater than or equal to upload_max_filesize
post_max_size = 10M
 
; You'll need this to avoid errors with the Date function
; http://stackoverflow.com/questions/16765158/date-it-is-not-safe-to-rely-on-the-systems-timezone-settings
[Date]
; Defines the default timezone used by the date functions
; http://php.net/date.timezone
date.timezone = America/New_York

Appendix A
mod_ssl error after patching

I have an apache 2.2.21 server on a SLES server. After a system patch (I guess) I realized the apache web server wouldn’t start. It shows this error:

> sudo service apache201 start

Starting httpd (/usr/local/apache2/bin/httpd) httpd: Syntax error on line 54 of /usr/local/apache201/conf/httpd.conf: Cannot load /usr/local/apache201/modules/mod_ssl.so into server: /usr/local/apache201/modules/mod_ssl.so: undefined symbol: ap_map_http_request_error

I had been playing fast and loose and I borrowed the mod_ssl.so from some other system, I guess. I forget which. In other words, I dropped in by hand a mod_ssl.so into the directory /usr/lib64/apache2-prefork. I was using those system-supplied modules paired with my compiled apache. All fine until that patch. So I found another mod_ssl.so frm a different system and tried that one. It worked. Whew. These were both SLES 11 SP 4 systems. The older one (with the mod_ssl.so that still works) is dated April 18th, 2017. The one with the broken mod_ssl.so Dec 29thth 2017. That’s from a uname -a.

References and related articles
A proper guide to installing apache 2.4 on CentOS is https://jasonpowell42.wordpress.com/2013/04/05/install-apache-2-4-4-on-centos-6-4/

Some upgrade issues are covered by apache’s own guide: http://httpd.apache.org/docs/2.4/upgrading.html

Scaling up apache to handle more than a couple hundred simultaneous requests is described in this blog post.

The DigiCERT certificate inspector tool, which is what I was referring to in this post when it comes to scanning for BREACH vulnerabilities, is here.

Categories
Admin Apache Hosting Service

running a second, third, …, instance of WordPress on your server

Intro
Since I can host drjohnstechtalk.com myself on my AWS server, why not a second blog, totally unrelated, for a friend? This has not been documented as well as I would have liked though it is very straightforward. So I’ll mention a few things here.

WordPress prep activities
You follow the WordPress regular installation instructions: http://codex.wordpress.org/Installing_WordPress. But I’ll repeat the important steps for the DIY admin with their own server like me:

$ cd /tmp; wget ‐‐no‐check‐certificate https://wordpress.org/latest.tar.gz
$ tar ‐xzvf latest.tar.gz
$ sudo cp ‐r wordpress <YOUR_HTDOC_ROOT>/blog

Set up a dedicated virtual server (apache virtual server) to handle this additional domain (that’s a whole post to explain).

The main thing is to realize you can set up a separate database in your single mysql instance for your second blog:

$ mysql -u adminusername -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 5340 to server version: 3.23.54
 
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
 
mysql&gt; CREATE DATABASE 2nddatabasename;
Query OK, 1 row affected (0.00 sec)
 
mysql&gt; GRANT ALL PRIVILEGES ON 2nddatabasename.* TO "2ndwordpressusername"@"localhost"
    -&gt; IDENTIFIED BY "passwordfor2nddatabase";
Query OK, 0 rows affected (0.00 sec)
 
mysql&gt; FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.01 sec)
 
mysql&gt; EXIT
Bye
$

Then access this web site’s WordPress setup page from a browser:

URL: http://example.com/blog/wp-admin/install.php

WP-Setup-capture

Error connecting to the database?

I usually goof up something or other. I have even literally created a username that was ‘username’@’hostname’ because I read that off my example before I corrected it. I needed to specify localhost instead of hostname. But anyway, I didn’t panic. I tried to connect to the MySQL DB with the user I thought I had just created (and it didn’t work), i.e., mysql -u newuser -p – would not accept the password I knew I had just created.

I even had problems droppnig that user, again, because unless I specified the user as ‘username’@’hostname’ it could not find the user!

In the days of MariaDB these types of DB commands still work the same way, for the record.

Categories
Admin Apache

How I compile apache2

Intro
This is just for my own documentation.

The details
This worked out on apache v 2.27 where I wanted to have ldap authentication and webDAV support:

$ ./configure –enable-ldap –enable-auth-ldap –with-ldap –enable-headers –enable-rewrite –enable-proxy –enable-auth
nz –enable-auth-basic –enable-authnz-ldap –enable-dav –enable-dav-fs

Note that this is recorded in config.log.

I also compiled this on Solaris 10, where I didn’t need DAV support but needed LDAP. This worked out for me there:

$ ./configure –enable-ldap –enable-auth-ldap –with-ldap –enable-headers –enable-rewrite –enable-proxy –enable-authnz –en
able-auth-basic –enable-authnz-ldap

To prove we got the right version:

$ /usr/local/apache2/bin/httpd -v

and to show all the modules we compiled in:

$ /usr/local/apache2/bin/httpd -l

Conclusion
A reminder on how apache 2.27 was compiled.

Categories
Admin Apache CentOS Security

drjohnstechtalk.com is now an encrypted web site

Intro
I don’t overtly chase search engine rankings. I’m comfortable being the 2,000,000th most visited site on the Internet, or something like that according to alexa. But I still take pride in what I’m producing here. So when I read a couple weeks ago that Google would be boosting the search rank of sites which use encryption, I felt I had to act. For me it is equally a matter of showing that I know how to do it and a chance to write another blog posting which may help others.

Very, very few people have my situation, which is a self-hosted web site, but still there may be snippets of advice which may apply to other situations.

I pulled off the switch to using https instead of http last night. The detail of how I did it are below.

The details
Actually there was nothing earth-shattering. It was a simple matter of applying stuff i already know how to do and putting it all together. of course I made some glitches along the way, but I also resolved them.

First the CERT
I was already running an SSL web server virtual server at https://drjohnstechtalk.com/ , but it was using a self-signed certificate. Being knowledgeable about certificates, I knew the first and easiest thing to do was to get a certificate (cert) issued by a recognized certificate authority (CA). Since my domain was bought from GoDaddy I decided to get my SSL certificate form them as well. It’s $69.99 for a one-year cert. Strangely, there is no economy of scale so a two-year cert costs exactly twice as much. i normally am a strong believer in two-year certs simply to avoid the hassle of renewing, etc, but since I am out-of-practice and feared I could throw my money away if I messed up the cert, I went with a one-year cert this time. It turns out I had nothing to fear…

Paying for a certificate at GoDaddy is easy. Actually figuring out how to get your certificate issued by them? Not so much. But I figured out where to go on their web site and managed to do it.

Before the CERT, the CSR
Let’s back up. Remember I’m self-hosted? I love being the boss and having that Linux prompt on my CentOS VM. So before I could buy a legit cert I needed to generate a private key and certificate signing request (CSR), which I did using openssl, having no other fancy tools available and being a command line lover.

To generate the private key and CSR with one openssl command do this:

$ openssl req -new -nodes -out myreq.csr

It prompts you for field values, e.g.:

Here’s how that dialog went:

Generating a 2048 bit RSA private key
.............+++
..............................+++
writing new private key to 'privkey.pem'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:US
State or Province Name (full name) []:New Jersey
Locality Name (eg, city) [Default City]:
Organization Name (eg, company) [Default Company Ltd]:drjohnstechtalk.com
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:drjohnstechtalk.com
Email Address []:
 
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

and the files it created:

$ ls -ltr|tail -2

-rw-rw-r-- 1 john john     1704 Aug 23 09:52 privkey.pem
-rw-rw-r-- 1 john john     1021 Aug 23 09:52 myreq.csr

Before shipping it off to a CA you really ought to examine the CSR for accuracy. Here’s how:

$ openssl req -text -in myreq.csr

Certificate Request:
    Data:
        Version: 0 (0x0)
        Subject: C=US, ST=New Jersey, L=Default City, O=drjohnstechtalk.com, CN=drjohnstechtalk.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:e9:04:ab:7e:e1:c1:87:44:fb:fe:09:e1:8d:e5:
                    29:1c:cb:b5:e8:d0:cc:f4:89:67:23:ab:e5:e7:a6:
                    ...

What are we looking for, anyways? Well, the modulus should be the same as it is for the private key. To list the modulus of your private key:

$ openssl rsa -text -in privkey.pem|more

The other things I am looking for is the common name (CN) which has to exactly match the DNS name that is used to access the secure site.

I’m not pleased about the Default City, but I didn’t want to provide my actual city. We’ll see it doesn’t matter in the end.

For some CAs the Organization field also matters a great deal. Since I am a private individual I decided to use the CN as my organization and that was accepted by GoDaddy. So Probably its value also doesn’t matter.

the other critical thing is the length of the public key, 2048 bits. These days all keys should be 2048 bits. some years ago 1024 bits was perfectly fine. I’m not sure but maybe older openssl releases would have created a 1024 bit key length so that’s why you’ll want to watch out for that.

Examine the CERT
GoDaddy issued the certificate with some random alpha-numeric filename. i renamed it to something more suitable, drjohnstechtalk.crt. Let’s examine it:

$ openssl x509 -text -in drjohnstechtalk.crt|more

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            27:ab:99:79:cb:55:9f
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, ST=Arizona, L=Scottsdale, O=GoDaddy.com, Inc., OU=http://certs.godaddy.com/repository/, CN=Go Daddy Secure Certificate Authority - G2
        Validity
            Not Before: Aug 21 00:34:01 2014 GMT
            Not After : Aug 21 00:34:01 2015 GMT
        Subject: OU=Domain Control Validated, CN=drjohnstechtalk.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:e9:04:ab:7e:e1:c1:87:44:fb:fe:09:e1:8d:e5:
                    29:1c:cb:b5:e8:d0:cc:f4:89:67:23:ab:e5:e7:a6:
                     ...

So we’re checking that common name, key length, and what if any organization they used (in the Subject field). Also the modulus should match up. Note that they “cheaped out” and did not provide www.drjohnstechtalk.com as an explicit alternate name! In my opinion this should mean that if someone enters the URL https://www.drjohnstechtalk.com/ they should get a certificate name mismatch error. In practice this does not seem to happen – I’m not sure why. Probably the browsers are somewhat forgiving.

The apache side of the house
I don’t know if this is going to make any sense, but here goes. To begin I have a bare-bones secure virtual server that did essentially nothing. So I modified it to be an apache redirect factory and to use my brand shiny new legit certificate. Once I had that working I planned to swap roles and filenames with my regular configuration file, drjohns.conf.

Objective: don’t break existing links
Why the need for a redirect factory? This I felt as a matter of pride is important: it will permit all the current links to my site, which are all http, not https, to continue to work! That’s a good thing, right? Now most of those links are in search engines, which constantly comb my pages, so I’m sure over time they would automatically be updated if I didn’t bother, but I just felt better about knowing that no links would be broken by switching to https. And it shows I know what I’m doing!

The secure server configuration file on my server is in /etc/apache2/sites-enabled/drjohns.secure.conf. It’s an apache v 2.2 server. I put all the relevant key/cert/intermediate cert files in /etc/apache2/certs. the private key’s permissions were set to 600. The relevant apache configuration directives are here to use this CERT along with the GoDaddy intermediate certificates:

 
   SSLEngine on
    SSLCertificateFile /etc/apache2/certs/drjohnstechtalk.crt
    SSLCertificateKeyFile /etc/apache2/certs/drjohnstechtalk.key
    SSLCertificateChainFile /etc/apache2/certs/gd_bundle-g2-g1.crt

I initially didn’t include the intermediate certs (chain file), which again in my experience should have caused issues. Once again I didn’t observe any issues from omitting it, but my experience says that it should be present.

The redirect factory setup
For the redirect testing I referred to my own blog posting (which I think is underappreciated for whatever reason!) and have these lines:

# I really don't think this does anything other than chase away a scary warning in the error log...
        RewriteLock ${APACHE_LOCK_DIR}/rewrite_lock
<VirtualHost *:80>
 
        ServerAdmin webmaster@localhost
        ServerName      www.drjohnstechtalk.com
        ServerAlias     drjohnstechtalk.com
        ServerAlias     johnstechtalk.com
        ServerAlias     www.johnstechtalk.com
        ServerAlias     vmanswer.com
        ServerAlias     www.vmanswer.com
# Inspired by the dreadful documentation on http://httpd.apache.org/docs/2.0/mod/mod_rewrite.html
        RewriteEngine on
        RewriteMap  redirectMap prg:redirect.pl
        RewriteCond ${redirectMap:%{HTTP_HOST}%{REQUEST_URI}} ^(.+)$
# %N are backreferences to RewriteCond matches, and $N are backreferences to RewriteRule matches
        RewriteRule ^/.* %1 [R=301,L]

Pages look funny after the switch to SSL
One of the first casualties after the switch to SSL was that my pages looked funny. I know from general experience that this can happen if there is hard-wired links to http URLs, and that is what I observed in the page source. In particular my WP-Syntax plugin was now bleeding verbatim entries into the columns to the right if the PRE text contained long lines. Not pretty. The page source mostly had https includes, but in one place it did not. It had:

<link rel="stylesheet" href="http://drjohnstechtalk.com/blog/wp-content/plugins/wp-syntax/wp-syntax.css"

I puzzled over where that originated and I had a few ideas which didn’t turn out so well. For instance you’d think inserting this into wp-config.php would have worked:

define( 'WP_CONTENT_URL','https://drjohnstechtalk.com/blog/wp-content/');

But it had absolutely no effect. Finally I did an RTFM – the M being http://codex.wordpress.org/Editing_wp-config.php which is mentioned in wp-config.hp – and learned that the siteurl is set in the administration settings in the GUI, Settings|General WordPress Address URL and Site Address URL. I changed these to https://drjohnstechtalk.com/blog and bingo, my plugins began to work properly again!

What might go wrong when turning on SSL
In another context I have seen both those errors, which I feel are poorly documented on the Internet, so I wish to mention them here since they are closely related to the topic of this blog post.

SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:601

and

curl: (35) error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol

I generated the first error in the process of trying to look at the SSL web site using openssl. What I do to test that certificates are being properly presented is:

$ openssl s_client -showcerts -connect localhost:443

And the second error mentioned above I generated trying to use curl to do something similar:

$ curl -i -k https://localhost/

The solution? Well, I began to suspect that I wasn’t running SSL at all so I tested curl assuming I was running on tcp port 443 but regular http:

$ curl -i http://localhost:443/

Yup. That worked just fine, producing all the usual HTTP response headers plus the content of my home page. So that means I wasn’t running SSL at all.

This virtual host being from a template I inherited and one I didn’t fully understand, I decided to just junk the most suspicious parts of the vhost configuration, which in my case were:

<IfDefine SSL>
<IfDefine !NOSSL>
...
</IfDefine>
</IfDefine>

and comment those guys out, giving,

#<IfDefine SSL>
#<IfDefine !NOSSL>
...
#</IfDefine>
#</IfDefine>

That worked! After a restart I really was running SSL.

Making it stronger
I did not do this immediately, but after the POODLE vulnerability came out and I ran some tests I realized I should have explicitly chosen a cipher suite in my apache server to make the encryption sufficiently unbreakable. This section of my working with ciphers post shows some good settings.

Mixed content
I forgot about something in my delight at running my own SSL web server – not all content was coming to the browser as https and Firefox and Internet Explorer began to complain as they grew more security-conscious over the months. After some investigation I found that what it was is that I had a redirect for favicon.ico to the WordPress favicon.ico. But it was a redirect to their HTTP favicon.ico. I changed it to their secure version, https://s2.wp.com/i/favicon.ico, and all was good!

I never use the Firefox debugging tools so I got lucky. I took a guess to find out more about this mixed content and clicked on Tools|Web developer|Web console. My lucky break was that it immediately told me the element that was still HTTP amidst my HTTPS web page. Knowing that it was a cinch to fix it as mentioned above.

Conclusion
Good-ole search engine optimization (SEO) has prodded us to make the leap to run SSL. In this posting we showed how we did that while preserving all the links that may be floating ou there on the Internet by using our redirect factory.

References
Having an apache instance dedicated to redirects is described in this article.
Some common sense steps to protect your apache server are described here.
Some other openssl commands besides the ones used here are described here.
Choosing an appropriate cipher suite and preventing use of the vulnerable SSLv2/3 is described in this post.
I read about Google’s plans to encrypt the web in this Naked Security article.

Categories
Admin Apache Security SLES Web Site Technologies

RSA Web Agent Installation: what might go wrong

Intro
As usual I ran into a few problems installing the RSA Web agent for a client. With this documentation I hope to jog my memory for my next installation or help someone else out who is experiencing the same problems.

The details
I was installing it on on SLES 11 system, Web Agent version 7.1.

So I ran the CD/install program as root and went through the prompts for the initial setup. I tried to laucnh firefox at the end, which it couldn’t, but I don’t think that is significant. I start up the web server. The error.log file begins to fill up! It looks like this:

acestatus: error while loading shared libraries: libaceclnt.so: cannot open shared object file: No such file or directory
rpc_server 2389 started by 2379
RSALogoffCookieService: error while loading shared libraries: libaceclnt.so: cannot open shared object file: No such file o
r directory
AceShutdown try to kill process 2389
signal 15 received
acestatus: error while loading shared libraries: libaceclnt.so: cannot open shared object file: No such file or directory
RSALogoffCookieService: error while loading shared libraries: libaceclnt.so: cannot open shared object file: No such file o
r directory
start child 2403
[Mon Aug 18 16:17:55 2014] [notice] Apache/2.2.27 (Unix) mod_rsawebagent/7.1.0[639] DAV/2 PHP/5.2.14 with Suhosin-Patch con
figured -- resuming normal operations
Cannot register service: RPC: Authentication error; why = Client credential too weak
unable to register (300760, 1).child 2403 end
start child 2409
Cannot register service: RPC: Authentication error; why = Client credential too weak
unable to register (300760, 1).child 2409 end
start child 2410
Cannot register service: RPC: Authentication error; why = Client credential too weak
unable to register (300760, 1).child 2410 end
start child 2411
Cannot register service: RPC: Authentication error; why = Client credential too weak
unable to register (300760, 1).child 2411 end
start child 2412
Cannot register service: RPC: Authentication error; why = Client credential too weak
unable to register (300760, 1).child 2412 end
start child 2413
...

Not good.

So I eventually realize that my web server is running as user wwwrun and the RSA web agent stuff I installed as root and its directory, rsawebagent, is owned by userid 40959 – there was no attempt by the installer to match that up to the user the web server runs as. So I try a fix by hand like this:

$ chown -R wwwrun rsawebagent

Success! That succeeds in getting rid of the repeating RPC error. Now the error.log file has only a modest level of errors:

acestatus: error while loading shared libraries: libaceclnt.so: cannot open shared object file: No such file or directory
rpc_server 27766 started by 27756
RSALogoffCookieService: error while loading shared libraries: libaceclnt.so: cannot open shared object file: No such file or directory
AceShutdown try to kill process 27766
signal 15 received
acestatus: error while loading shared libraries: libaceclnt.so: cannot open shared object file: No such file or directory
RSALogoffCookieService: error while loading shared libraries: libaceclnt.so: cannot open shared object file: No such file or directory
start child 27780
[Mon Aug 18 16:25:00 2014] [notice] Apache/2.2.27 (Unix) mod_rsawebagent/7.1.0[639] DAV/2 PHP/5.2.14 with Suhosin-Patch configured -- resuming normal operations

But the thing is, it actually, mostly kind of, seems to work. You see a promising Authentication Succeeded screen in your browser after logging in to the home page. But then it directs you back to the RSA login screen. I was actually stuck on this point for a long time.

The error.log file also looks encouraging at this point:

[Mon Aug 18 16:27:28 2014] [notice] Authentication succeeded User: drj.

My insight today is to tackle the libaceclnt.so problem. I actually ran a trace of the startup to see where it was looking for that file so I could put it there. It was looking in system directories like these:

[pid 31974] open("/usr/lib64/tls/x86_64/libaceclnt.so", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 31974] stat("/usr/lib64/tls/x86_64", 0x7fff93b721b0) = -1 ENOENT (No such file or directory)
[pid 31974] open("/usr/lib64/tls/libaceclnt.so", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 31974] stat("/usr/lib64/tls", 0x7fff93b721b0) = -1 ENOENT (No such file or directory)
[pid 31974] open("/usr/lib64/x86_64/libaceclnt.so", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 31974] stat("/usr/lib64/x86_64", 0x7fff93b721b0) = -1 ENOENT (No such file or directory)
[pid 31974] open("/usr/lib64/libaceclnt.so", O_RDONLY) = -1 ENOENT (No such file or directory)
...

So I decided to make a soft link to it from /usr/lib64 such that:

 libaceclnt.so -> /usr/local/apache202/rsawebagent/libaceclnt.so

Note that my ServerRoot was /usr/local/apache202.

Now when I start up my apache202 instance I have this in error.log:

rpc_server 28874 started by 28860
grep RSALogoffCookieService /proc/*/cmdline | sed 's/\/cmdline.*\/proc\// /g' | sed 's/\/cmdline.*/ /'  | sed 's/.*\/proc\// /' | sort -u
start child 28877
grep RSALogoffCookieService /proc/*/cmdline | sed 's/\/cmdline.*\/proc\// /g' | sed 's/\/cmdline.*/ /'  | sed 's/.*\/proc\// /' | sort -u
AceShutdown try to kill process 28874
signal 15 received
grep RSALogoffCookieService /proc/*/cmdline | sed 's/\/cmdline.*\/proc\// /g' | sed 's/\/cmdline.*/ /'  | sed 's/.*\/proc\// /' | sort -u
start child 28913
[Mon Aug 18 16:36:23 2014] [notice] Apache/2.2.27 (Unix) mod_rsawebagent/7.1.0[639] DAV/2 PHP/5.2.14 with Suhosin-Patch configured -- resuming normal operations

And best of all – it actually works!

I get the RSA authentication page initially. I log on and get redirected to the actual server home page. The access.log file records my username in the access line.

Additional error observed months later
You know that symptom I described above? You see a promising Authentication Succeeded screen in your browser after logging in to the home page. But then it directs you back to the RSA login screen. My web server had been running fine for over a month when all of a sudden it behaved that way again. Confounding. So I put on my big boy pants and did an strace. Nothing popped out at me, but I was struck by frequent access to an htdocs filepath. What’s so unusual about that? I don’t use htdocs in my configurations! So where was that coming from? I re-checked my configuration. OK, this is embarrassing. I have a sweeping include statement in my top-level httpd.conf file:

# pick up all vhosts
Include conf/vhosts/*.conf

It seemed like a good idea at the time. In my conf/vhosts directory I actually had two conf files, my rsaauth.conf but also a dflt.conf!! And the dflt.conf had the references to htdocs, but no references to the RSA authentication. So it was being used to establish the location of the home directory and the other conf file to fix the authentication type, I guess.

I removed the dflt.conf file, restarted and everything began to work once again. Whew!

RPC errors returned after a few months
After a year or so of running the RPC errors mentioned above returned and I never could figure out why and I no longer needed this service so I didn’t pursue it.

Conclusion
A few errors were observed installing RSA Web Agent v 7.1 on SLES Linux. I had had similar problems on Redhat as well. I finally found some solutions and now they’re ready to use it!

References
This write-up is partially related to my blog post of installing multiple apache instances.

Categories
Admin Apache Linux

Recording Host Header in the apache access log

Intro
Guess I’ve made it pretty clear in previous posts that Apache documentation is horrible in my opinion. So the only practical way to learn something is to configure by example. In this post I show how to record the Host header contained in an HTTP request in your Apache log.

The details
Why might you want to do this? Simple, if you have multiple hosts using one access log in common. For instance I have johnstechtalk.com and drjohnstechtalk.com using the same log, which I view as a convenience for myself. But now I want to know if I’m getting my money’s worth out of johnstechtalk.com, which I don’t see as the main URL, but I I use it to to type it into the browser location bar and get directed onto my site – fewer letters.

So I assume you know where to find the log definitions. You start with that as a base and create a custom-defined access log name. These two lines, taken from my actual config file, apache2.conf, show this:

LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\" \"%{Host}i\"" DrJformat

Then I have my virtual server in a separate file containing a reference to that custom format:

#CustomLog ${APACHE_LOG_DIR}/../drjohns/access.log combined
CustomLog ${APACHE_LOG_DIR}/../drjohns/access.log DrJformat

The ${APACHE_LOG_DIR} is an environment variable defined in envvars in my implementation, which may be unconventional. you can replace it with a hard-wired directory name if that suits you better.

There is some confusion out there on the Internet. Host as used in this post refers as I have said to the value contained in the HTTP Host Request header. It is not the hostname of the client.

Here are some recorded access resulting from this format early this morning:

108.163.185.34 - - [08/Jan/2014:02:21:32 -0500] "GET /blog/2012/02/tuning-apache-as-a-redirect-engine/ HTTP/1.1" 200 11659 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.71 Safari/537.36" "drjohnstechtalk.com"
5.10.83.22 - - [08/Jan/2014:02:21:56 -0500] "GET /blog/2013/03/generate-pronounceable-passwords/ HTTP/1.1" 200 8253 "-" "Mozilla/5.0 (compatible; AhrefsBot/5.0; +http://ahrefs.com/robot/)" "drjohnstechtalk.com"
220.181.108.91 - - [08/Jan/2014:02:23:41 -0500] "GET / HTTP/1.1" 301 246 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" "vmanswer.com"
192.187.98.164 - - [08/Jan/2014:02:25:00 -0500] "GET /blog/2012/02/running-cgi-scripts-from-any-directory-with-apache/ HTTP/1.0" 200 32338 "http://drjohnstechtalk.com/blog/2012/02/running-cgi-scripts-from-any-directory-with-apache/" "Opera/9.80 (Windows NT 5.1; MRA 6.0 (build 5831)) Presto/2.12.388 Version/12.10" "drjohnstechtalk.com"

While most lines contain drjohnstechtalk.com, note that the next-to-last line has the host vmanswer.com, which is another domain one I bought and associated with my site to try it out.

Conclusion
We have shown how to record the contents of the Host header in an Apache access log.

Related rants against apache
Creating a maintenance page with Apache web server
Turning Apache into a Redirect Factory
Running CGI Scripts from any Directory with Apache

Categories
Admin Apache

Creating a maintenance page with Apache web server

Intro
Sometimes you want to run a web server that spits back the same page – a maintenance screen – no matter what URI it was accessed by. This is a simple few lines change to accomplish that.

The details
Why might you want to do this? Suppose you had a load balancer for a particular service. And suppose you have to move all the pool members at the same time to a different data center. You’re left with no service at all.

So you can use priority groups and add a lower-priority “maintenance server” to the pool which is not getting moved, and it will answer all queries destined for the service with your desired maintenance page.

I read through the dreadful documentation on Apache (how about this page for a little guilty pleasure reading) and found this way to do it. OK, disclosure time. When you hold a hammer, everything looks like a nail. I had used Mod Rewrite previously so I had some familiarity with it, and I guessed it could be used for this purpose as well. In reality there are probably lots of ways to accomplish this same end goal.

Inside your virtual server:

<VirtualHost *:80>
RewriteEngine on
RewriteRule  ^.*                 /maintenance.htm  [L]
...other usual stuff establishing home dir and permissions ...
</VirtualHost>

My maintenance page
For the record my page looks like this:

<html><head><title>Site Maintenance</title></head>
<body>
<font face="arial">
<h1>Site Maintenance</h1>
<strong>
This site is temporarily down.<br>
Service will be restored by 2 PM Saturday.
</font>
</strong>
</body>
</html>

Gotchas
The Rewrite rule would need refinement if you wanted to maintain a corporate identity and offer a maintenance page that has images, stylesheets, etc.

And after posting this I ran into another trouble. The actual pages we wanted the message for weren’t getting that message, which was quite mysterious. Instead the message was like this:

Service Temporarily Unavailable
 
The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

I guess you could do worse, but that’s not my message so where did it comes from and how come I didn’t see it before?

This one is simple enough. I had only tested with random characters, as in

curl -i http://localhost/asdasdd

I hadn’t tested with one of the actual URIs, many of which end in .jsp. Long story short, I had re-purposed a web server instance that was front-ending a jBoss application server, so it had statements that made it handle JSP pages differently! In particular, there was this:

# JBoss include stuff
Include conf/mod-jk.conf

and this:

JkMountCopy On

With those lines both commented out it began to throw my maintenance page for all requests, as originally intended. Crisis averted.

Appendix – How to redirect just a specific page
If you want to implement a redirect for just a specific page you can follow this example:

# redirect for test of PAC file - DrJ 4/10/17
RewriteEngine on
RewriteRule  /proxy.pac                 http://50.17.188.196/proxy.pac [L]

Here I am only redirecting requests for the URI proxy.pac and sending it to another server. All other pages remain unaffected.

Conclusion
We have shown how to create a web server that whatever you’ve asked of it, always returns a maintenance page along with HTTP status code of 200. This can be helpful for maintenance or moving situations.

References
My most creative use of URI rewriting is in creating an Apache “redirect factory,” which is described here.

Categories
Apache Linux Network Technologies Perl Raspberry Pi

Getting started on my Raspberry Pi

Intro
The Raspberry Pi computer is an awesome idea. Its performance is surprisingly good as well, as I will show below. Available packages? Not so impressive. I share some old X-windows tricks which will allow you to bring up the GUI without ever using the HDMI port.

The details
My Methodology
I was too lazy to set up an HDMI console plus keyboard and mouse. I’m more a server guy anyways so I’m more interested in what I can accomplish from a command prompt. And this also makes getting started that much easier. I had burned the Raspbian Wheezy image to a super-fast SD card (more on that below) the day that my Pi came in the mail. I attached power and ethernet, booted it up, guessed the IP it acquired by running some PINGs, did an ssh using the pi/raspberry user and Bingo! I was in. It couldn’t be easier. How I tested GUI applications without a console is explained further down below.

First Impressions
It feels fast.

Packages
Not much seems to be there by default – no apache, not many X utilities. There is a lame X browser called x-www-browser. I thought this is Debian, right? So we can just start downloading Debian packages, like Firefox. Wrong! It doesn’t work that way. There’s no Firefox, Safari, Chrome or Opera! It does come pre-loaded with curl, however, ha, ha.

No, the Raspbian FAQ explains why this is. It’s rather complicated. I guess the compiler works though I haven’t tested it yet. So I suppose you could compile packages from their source code.

The x-terminal-emulator is pretty decent, however.

If it comes with a web server, I didn’t notice. So I quickly checked for the availability of apache. It’s available. Then installed it:

> sudo apt-get install apache2

That worked out well. It installed it and the packages it depended on and even launched it, and it all felt fairly peppy. See the suggested fix further down if this gives you errors. The default HTML DOCROOT is /var/www. I accessed it locally:

> curl localhost

And a welcome message displayed. A good start.

Where’s the rest of my 16 GB SD card gone to?

Original disk layout:

pi@raspberrypi:~$ df -k
Filesystem     1K-blocks    Used Available Use% Mounted on
rootfs           1804128 1492908    219572  88% /
/dev/root        1804128 1492908    219572  88% /
devtmpfs          224436       0    224436   0% /dev
tmpfs              44900     204     44696   1% /run
tmpfs               5120       0      5120   0% /run/lock
tmpfs              89780       0     89780   0% /run/shm
/dev/mmcblk0p1     57288   16872     40416  30% /boot

Layout after raspi-config:

pi@raspberrypi:~$ df -k
Filesystem     1K-blocks    Used Available Use% Mounted on
rootfs          15251960 1494852  12982544  11% /
/dev/root       15251960 1494852  12982544  11% /
devtmpfs          224436       0    224436   0% /dev
tmpfs              44900     196     44704   1% /run
tmpfs               5120       0      5120   0% /run/lock
tmpfs              89780       0     89780   0% /run/shm
/dev/mmcblk0p1     57288   16872     40416  30% /boot

Whew! That was easy. All 16 GB accounted for and actively used.

Was it worth it to buy that UHS SD card?
I didn’t want a sluggish server, so I paid a couple bucks more and bought a 16 GB SD UHS (ultra high speed) card for my “disk,” not knowing whether or not the Pi had the muscle to put it to work.

A quick aside about SD cards
I did a quickie self-education on this topic. Most SD cards are rated by class, so a class 4 SD card can do 4 MB/sec I/O, and a class 10 card can do at least 10 MB/sec. Faster still are the UHS SD cards. My Sandisk, which only cost about $19, is rated for 45 MB/sec I/O. A great write-up on this topic specifically for Raspberry Pi is: Raspberry Pi SD Card Speed Test – Raspberry Pi

diskSpec.pl benchmark (higher numbers are better)
1333 file creation/destruction operations per second – Raspberry Pi with UHS SD card
6666 file creation/destruction operations per second – EBS volume on small image running CentOS in Amazon cloud
26000 file creation/destruction operations per second – high-end HP server (G7 DL380) running SLES 11

I think I provided the source for this simple Perl program I wrote, diskSpec.pl. It creates a file, writes a random number into it, then deletes it – that all counts as one operation. Here it is:

#!/usr/bin/perl
# DrJ, 1/2000
# Test disk I/O
$DIR = $ARGV[0];
chdir($DIR);
$t0 = time();
while(1) {
  $ran = rand();
  open(FILE,"&gt;$ran") || die "Cannot open file $ran in directory $DIR!!\n";
  print FILE $ran;
  close(FILE);
  unlink($ran);
  $cnt++;
  if ($cnt % 20000 == 0) {
    $rate = $cnt / (time() - $t0) ;
    print "File creation/desctruction rate: $rate\n";
  }
}

DrJ 2017 Note: The notes below are historical and does not seem to work at all for the Raspberry Pi 3 loaded with NOOBS. In NOOBS you select your OS to install. You can’t ssh to it. I know. I just tried! Even after you install Raspbian Wheezy, you still can’t access it via ssh until you enable the ssh daemon with raspi-condfig.

How to get the GUI working without a console
I have this feeling that many people trying out the Pi won’t have the faintest idea how X windows works, unlike us Unix old-timers. It’s fun to put 20-year-old lessons to work on something new. Like I said I’m lazy and didn’t feel the need to set up an actual console to the thing. I used some old X features to allow me to launch specific X-windows applications that are pre-loaded on the device, and display them on my PC. How?

On a Windows PC you install Cygwin. Then launch the XWin Server. You ssh to your pi. How do you know its IP the first time? Guess! It picks it up via DHCP, so start PINGing around the range where your other devices are numbered. My PC is 192.168.5.12/24, my pi was 192.168.5.16. Maybe you have a bunch of devices responding to PING and are unsure which is which? Your MAC table is your friend. Here’s mine:

C:\Documents and Settings&gt;arp -a
 
Interface: 192.168.5.12 --- 0x2
  Internet Address      Physical Address      Type
  192.168.5.1           00-14-f6-e0-c0-4c     dynamic
  192.168.5.16          b8-27-eb-dd-21-02     dynamic
  192.168.5.99          00-90-a9-bb-3d-76     dynamic

arp displays the MAC table with the IP-to-physical (MAC) address correspondence. So most Pi’s will have a MAC address whose beginning is similar to b8-27-eb. A quick aside. Does the MAC address follow the board (SOC) or the SD Card? The board – I tested this with a friend’s SD Card.

You login with the pi/raspberry.

Then set your DISPLAY environment variable:

> export DISPLAY=192.168.5.12:0

Most of your X applications begin with the letter “x,” so enter

> x<tab><tab>

to see a display of available programs like this:

xapian-config        xdg-screensaver      xkbevd               xpdf.real            xxd
xarchiver            xdg-settings         xkbprint             xprop                xz
xargs                xdpyinfo             xkbvleds             x-session-manager    xzcat
xauth                xdriinfo             xkbwatch             xsubpp               xzcmp
xdg-desktop-icon     xev                  xkill                xtables-multi        xzdiff
xdg-desktop-menu     xfd                  xlsatoms             x-terminal-emulator  xzegrep
xdg-email            xfontsel             xlsclients           xvinfo               xzfgrep
xdg-icon-resource    xinit                xlsfonts             x-window-manager     xzgrep
xdg-mime             xkbbell              xmessage             xwininfo             xzless
xdg-open             xkbcomp              xpdf                 x-www-browser        xzmore

Actually I don’t know how many of these are X. But at least a few are.

Start an xterm in Cygwin. In the xterm window, give permission to the Pi to use it as its Xserver:

> xhost +

Now in the Pi shell (ha, ha), type:

> x-terminal-emulator

and you should see the colorful terminal emulator on your PC in a few seconds. this is a true GUI application. You similarly launch the x-www-browser. Don’t forget to background your X-windows in the Pi shell:

<Ctrl-Z>
> bg

so you can use the one window to launch multiple X windows.

Another example the book Programming the Raspberry Pi has is the Python interactive development environment. I reasoned from the screen shots that idles3 would also be an X application – hey, they don’t have to start with the letter x – and indeed it is!

Want the whole ball of wax, a complete console? I just figured this one out by taking an educated guess:

> x-session-manager

and you will see the complete GUI on your PC! Cool, huh?

Want to get rid of the last thing you backgrounded, like, say, that x-session-mnager which has taken over your PC?! Type

> fg
<Ctrl-C>

and it will be killed.

How to get the GUI working without a console, Method 2
The above steps look a little daunting? Even I don’t want to install cygwin on my new PC. There is an alternative which can suffice for light usage.

On the Pi install a vnc server:

$ sudo apt-get install tightvncserver

Launch it:

$ vncserver

The first time only it will ask you to set up a password. Might as well make it raspberry like everything else we do on the Pi.

Then install a VNC client on your PC (Or Macbook). I use RealVNC.

Launch your VNC client and connect to your Pi’s IP address (which you need to know) + the display number, like this:

192.168.0.100:1

For a Pi at IP 192.168.0.100 in which the vncserver started display 1. Normally it will be display 1, but I guess it might be display 0.

Don’t launch vncserver more than once! You don’t want a bunch of those running and dragging on performance.

Anyways, that’s it! You should see the Pi’s GUI on your PC, but it might seem a wee bit small.

Setting a static IP
If you’re going to use the Pi more as a server like I am, I think it’s a good idea to give it a static IP. What I did is to edit /etc/network/interfaces. Mine now looks like this:

Nagios can be installed! That's pretty cool - it's a sophisticated network monitoring utility.

Get a decent browser
The web browsers that come with the Pi are horrible. Midori? Seriously? I found you can get Firefox, but the downside is that it’s sloooww. But at least it works. The secret is that it’s not called Firefox. Instead:

$ sudo apt-get install iceweasel

Yes, it’s iceweasel, not Firefox, in Debian Linux. Go figure.

My cool transparent case
I recommend to get a case. I got the one with the best reviews. It’s kind of expensive, about $20, but worth it. It’s practically a work-of-art. Clear, the PC board fits snugly. I put it in my pocket and showed it around to my friends, feeling it was well protected, and yet also a sight to behold the first time. I even has a thoughtful light guide so the LEDs look beautiful as their light follows the rectangular opening to open air. I never had this much fun in show-and-tell! I just pulled the Pi wrapped in its case from my shirt pocket and amazed those around me. So go ahead and splurge. Anyways some of the cheaper cases look just that. Here is what I bought:

Helping a friend out with his Pi
So I dutifully take my friend’s Pi home and offer to install a web server. What did I do wrong? Well, duh, I could have just taken his SD Card home and plugged it into my Pi case! That concept takes some getting used to! We all have the same hardware. Our SD cards – our disk – are what make one Pi different from another.

So I followed my own blog post to recall some things. This Pi also had a MAC address beginning with the same six characters.

The apache2 installation did not work out, however. What to do? Well, I eventually read the darn output from running it. It suggests to try this:

> sudo apt-get update

So I ran that, figuring it could do no harm. Then I re-ran

> sudo apt-get install apache2

and this time the install actually worked!

Reading a flash drive
I was curious to see if you could stick a flash drive in the thing and just read it. I didn’t think so since I thought it would be formatted for NTFS. But if you have the GUI running and bring up a file manager, I’ll be darned if it doesn’t just work. I noticed the drive is mounted as /media/Cruzer (my flash drive has the brand name Cruzer).

If you don’t launch the file manager, I think you can still work with it as follows:

$ sudo mkdir /media/Cruzer; sudo mount /dev/sda1 /media/Cruzer

Then when you’re all done and before you remove it:

$ sudo umount /media/Cruzer

So that’s pretty cool. You can create tar archives on the flash drive, plug it into someone else’s Pi and untar it, etc, just like on Windows.

Conclusion
Raspberry Pi is respectable as a computer. It will be a lot of fun to explore for the hobbyist.

References

Raspberry Pi SD Card Speed Test – Raspberry Pi – a great discussion of the various speeds of Micro SD cards and how to measure yours
Go here for my next project – using your Raspberry Pi to monitor your home’s power or Internet connection.
Interested in networking? A lot of useful tips can be found in this posting describing how to turn your Pi into a router.
Realvnc.com distributes realVNC viewers for various platforms.
How about a Raspberry Pi-driven digital photo frame? I describe an approach in this article.
Brief Nagios for Raspberry Pi writeup.

Categories
Apache JavaScript Perl

A Simple Javascript Folder Browser

Part 1

Intro
I haven’t posted much lately. I’ve been tied up creating this folder browser using client-side JavaScript. I probably made every mistake in the book, but I worked through them all and the outcome is pretty cool, if I say so myself! It works in IE, FireFox and even Blackberry!

The details
I broke down the development of this browse app into 12 stages. Given time, I might show the progression of my thinking as each version becomes closer and closer to fulfilling all initial objectives. But who has time? I’ll show the source for browse3.html, warts and all, and then skip many iterations and jump to showing the final source, browse12.html.

Browse3.html
It ain’t pretty. It isn’t even correct. But it “does stuff.” It assumes Apache web server is running and “borrows” the closed and open folder icons from apache’s /icons directory. In my case I have a top-level directory called /homepic with folders under that and sub-folders under those folders that I want the ability to browse and , ultimately, take some action such as displaying all the folder’s images in the image viewer I wrote earlier.

<!DOCTYPE html>
<html>
<head>
<script type="text/javascript">
// global object
var folder = new Object;
function displayDate()
{
document.getElementById("demo").innerHTML=Date();
}
function init() {
// big initialization - generated code from perl perusal of directories
// see http://www.quirksmode.org/js/associative.html
//  var folder = new Object;
// we need this empty assignment to extend object with subproperties later on
  folder['homepic'] = '/homepic';
  //folder['/homepic'].path = '/homepic';
  folder.homepic.state = 'closed';
  folder[0] = '/homepic/kodak_pictures';
  folder[1] = '/homepic/pictures_chronological';
  folder['/homepic/kodak_pictures'] = '/homepic/kodak_pictures';
  folder['/homepic/kodak_pictures'].state = 'closed';
  //folder['/homepic/kodak_pictures'].path = '/homepic/kodak_pictures';
  folder['/homepic/pictures_chronological'] = '/homepic/pictures_chronological';
  folder['/homepic/pictures_chronological'].state = 'closed';
  //folder['/homepic/pictures_chronological'].path = '/homepic/pictures_chronological';
  var child = 'homepic';
  var cstate = folder['homepic'].state;
  var cpath = folder[0];
}
function browse(f) {
document.getElementById("demo").innerHTML="folder: "+f+" ";
var icon;
var imgid;
var fname;
if (f == "browse") {
// see http://www.quirksmode.org/dom/intro.html#create
// one-time initialization
  init();
  document.getElementById("browse").innerHTML='';
  icon = document.createElement('IMG');
  icon.src = "/icons/folder.gif";
  icon.id = "icon-/homepic";
  icon.onclick = Function('browse("/homepic")');
  document.getElementById("browse").appendChild(icon);
  var divfolder = document.createElement('DIV');
  divfolder.id = "/homepic";
  document.getElementById("browse").appendChild(divfolder);
  var x = document.createTextNode('homepic');
  document.getElementById('/homepic').appendChild(x);
  //document.getElementById("browse").innerHTML='<img id="homepic" onclick="browse(\'homepic\')" src="/icons/folder.gif">ho
mepic<br>';
} else {
  imgid = 'icon-' + f;
// change img to one of open folder
  document.getElementById(imgid).src = "/icons/folder.open.gif";
  // nope? imgid.src  = "/icons/folder.open.gif";
  for (i=0;i<3;i++) {
    icon = document.createElement('IMG');
    icon.src = "/icons/folder.gif";
    fname = folder[f].children[i];
    //fname = folder['homepic'].children[i];
    icon.id = "icon-" + fname;
    icon.onclick = Function('browse('+fname+')');
    document.getElementById(f).appendChild(icon);
    var divfolder = document.createElement('DIV');
    divfolder.id = fname;
    document.getElementById(f).appendChild(divfolder);
    var x = document.createTextNode(fname);
    document.getElementById(f).appendChild(x);
  } // end loop over children
}
}
// note variable # or arguments are being passed
function  openfolder()
{
var f = arguments[0];
var dir = "/icons/";
var fopen = "folder.open.gif";
var fclosed = "folder.gif";
var folder = document.getElementById(f);
var ftype = folder.src;
// src includes http... Get rid of stuff in front
var patt = /.*(folder.+)/;
var ftypebare = ftype.replace(patt,"$1");
// for debugging
document.getElementById("demo").innerHTML="folder: "+f+" "+ftypebare;
if (ftypebare == fopen) {
// close folder and remove sub-folders
  folder.src = dir+fclosed;
  document.getElementById("homepic/pictures_chronological").innerHTML='';
} else {
// open up folder and reveal sub-folders
  folder.src = dir+fopen;
  for (var i = 0; i < arguments.length; i++) {
    var placeid = arguments[i];
    var srcid = '/' + placeid;
    document.getElementById(placeid).innerHTML='&nbsp;&nbsp;&nbsp;<img id="' + srcid + '" onclick="openfolder(\'pictures_ch
ronological\')" src="/icons/folder.gif"/> pictures_chronological<br>';
  }
}
 
}
</script>
</head>
<body>
 
<h1>Folder Browser</h1>
<p id="demo">Debug Aid.</p>
<p id="browse"><a href="#" onclick='browse("browse")'>Folder Browser</a></p>
 
<img id="homepic" onclick="openfolder('homepic','homepic/pictures_chronological','homepic/canon_pictures')" src="/icons/fol
der.gif"/> homepic<br>
<div id="homepic/pictures_chronological"></div>
<div id="homepic/canon_pictures"></div>
<img id="cfolder2" onclick="openfolder(2)" src="/icons/folder.gif"/><br>
<img id="cfolder3" onclick="openfolder(3)" src="/icons/folder.gif"/><br>
<img id="cfolder4" onclick="openfolder(4)" src="/icons/folder.gif"/><br>
 
</body>
</html>

So you see in browse3 I’m wrestling with how to work with JavaScript Objects (which I didn’t really know existed at that time). I badly wanted to give an associative array properties, as in the initialization line

folder['/homepic/kodak_pictures'].state = 'closed';

but I couldn’t find any examples on the Internet. I eventually learned that wasn’t a correct assignment.

So one of my biggest and most worthwhile lessons was to gain a decent understanding of JavaScript objects, which hold multiple values, and object properties.

I tried to use Firebug for Firefox, with very limited success, but at least I could step through the Javascript code and see which branch in a conditional was being executed compared to what I thought should be executed, which tipped me off to one vexing problem concerning opening and closing folders multiple times. Also just looking up the error in Internet Explorer by double-=clicking the warning sign in the corner was tremendously helpful.

So…skipping for now versions 4 – 11, we arrive at browse12.html:

Browse12.html

<!DOCTYPE html>
<html>
<head>
<script type="text/javascript">
// global object
var folder = new Object;
function displayDate()
{
document.getElementById("demo").innerHTML=Date();
}
function init() {
// big initialization - generated code from perl perusal of directories
// I think my big breakthrough was to learn about Javascript objects and their properties
// from the book Javascript The Definitive Guide, 5th edition by D.Flanagan
// Some additional inspiration came from http://www.quirksmode.org/dom/intro.html#create
// folder is an associative array with properties path, state,depth and kids (which is itself an array)
// first entry is initial top-level to get things started
  folder['browse'] = {path:'',depth:0,kids:['homepic']};
// regular entries:
  folder['/homepic'] = {path:'/homepic',depth:1,kids:['kodak_pictures','pictures_chronological']};
// values for sub-folders
// /homepic/kodak_pictures
  folder['/homepic/kodak_pictures'] = {path:'/homepic/kodak_pictures',depth:2,kids:['2002_05','2002_06']};
// /homepic/pictures_chronological
  folder['/homepic/pictures_chronological'] = {path:'/homepic/pictures_chronological',depth:2,kids:['2011_11','2011_12']};
}
function browse(f) {
document.getElementById("demo").innerHTML="folder: "+f+" ";
var icon;
var imgid;
var fname;
var table;
if (f == "browse") {
// one-time initialization
  init();
}
 
imgid = 'icon-' + f;
if (!folder[f]) {
// folder not defined: must be a terminal folder. Do something here eventually
// but for now do nothing whatsoever
} else {
  if (folder[f].state === undefined || folder[f].state == 'closed') {
  // change img to one of open folder
    if (f == 'browse') {
      document.getElementById(f).innerHTML='';
    } else {
      document.getElementById(imgid).src = "/icons/folder.open.gif";
    }
    var kids = folder[f].kids;
    for(var i=0;  i < kids.length; i++) {
      table = document.createElement("table");
      table.border = 0;
      var row = document.createElement("tr");
      var td1 = document.createElement("td");
      var td2 = document.createElement("td");
      var td3 = document.createElement("td");
      icon = document.createElement("img");
      icon.src = "/icons/folder.gif";
      if (f == "browse") {
        fname = "/" + kids[i];
      } else {
        fname = f + "/" + kids[i];
      }
      icon.id = "icon-" + fname;
      if (folder[fname])
        folder[fname].state = 'closed';
      icon.onclick = Function('browse("'+fname+'")');
      td1.width = 27 + folder[folder].depth*27;
      td1.align = "right";
      td1.appendChild(icon);
      var node = document.createTextNode(kids[i]);
      td3.appendChild(node);
      row.appendChild(td1); row.appendChild(td2); row.appendChild(td3);
      table.appendChild(row);
      document.getElementById(f).appendChild(table);
      var divfolder = document.createElement("div");
      divfolder.id = fname;
      document.getElementById(f).appendChild(divfolder);
      folder[f].state = 'open';
    } // end loop over children
  } else {
  // set folder to closed state
  // this innerHTML nullification is kind of a kludge, but it works
    document.getElementById(f).innerHTML='';
    document.getElementById(imgid).src = "/icons/folder.gif";
    folder[f].state = 'closed';
  } // end conditional over folder state
} // end condition over whether folder is defined or not
} // end function browse
 
</script>
</head>
<body>
 
<h1>Folder Browser</h1>
<p id="demo">Debug Aid.</p>
<p id="browse"><a href="#" onclick='browse("browse")'>Folder Browser</a></p>
</body>
</html>

That’s it! Not bad, huh? I pan to generate the init() function periodically and automatically from a Perl script which peruses my directories.

I could have used Ajax and generated the subfolder information on the fly as it is needed – not that I know how, I just know enough to know it is possible and therefore I could do it – but I thought this method of pre-loading all the information might be a little more efficient. If this were a folder and file browser it would be different, but for now it is just a folder browser.

So the main revelation is that I had to set my associative array members to be objects during initialization, as in

folder['/homepic'] = {path:'/homepic',depth:1,kids:['kodak_pictures','pictures_chronological']};

and one of the object values is itself an anonymous array that holds the sub-folders.

Buying an actual book was probably a good move. I went with JavaScript, The Definitive Guide, 5th edition. Note that this is not the latest edition – the 6th – but this way I could buy the book used for a lot less and not get myself further confused by HTML5, which I am not ready to tackle and which many browsers do not yet fully support. The book is pretty heavy going and the discussion of DOM was particularly difficult and the examples too few and too removed from the real world. But the Core Javascript discussion made a lot of sense to me so was by itself worth the purchase price.

In my next post I’ve posted the Perl script which can generate the folder object initialization.

Part 2, A simple Perl script to build JavaScript folder objects

Conclusion
In this part one of a two-part post I’ve provided the JavaScript that implements a very compact folder browser. It has been tested on both IE and Firefox. The 2nd part of this series will provide the Perl Jvascript code generator for automation of the object creation.

Categories
Admin Apache CentOS Linux Web Site Technologies

Major Headaches Migrating Apache from Ubuntu to CentOS

Intro
I’m changing servers from Ubuntu server to CentOS. On Ubuntu I just checked off LAMP and got my environment. In CentOS I’m doing it piece-by-piece. I don’t think my Ubuntu install is quite regular, either, as I bastardized it by adding environment variables in the Apache config file, a concept I borrowed from SLES! Turns out it is quite an ordeal to make a smooth transition. I will share all my pitfalls. I still don’t have it working, but I think I’m over the hump. [Update: now it is working, or 99% of it is working. It is a bit sluggish, however.]

The Details
I installed httpd on CentOS using yum. I also installed some php5 packages which I saw were recommended as well. First thing I noticed is that the directory structure for “httpd” as it seems to be known on CentOS, is dramatically different from “apache2” as it is known in Ubuntu. This example illustrates the point. In CentOS the main config file is

/etc/httpd/conf/httpd.conf

while in Ubuntu I had

/etc/apache2/apache2.conf

so I tarred up my /etc/apache2 files and had the thought “Let’s make this work on CentOS.” Ha. Easier said than done.

To remind, the content of /etc/apache2 is:

apache2.conf, conf.d, sites-enabled sites-available mods-enabled mods-available plus some stuff I probably added, including envvars, httpd.conf and ports.conf.

envvars contains environment variables which are subsequently referenced in the config files, like these:

export APACHE_RUN_USER=www-data
export APACHE_RUN_GROUP=www-data
export APACHE_PID_FILE=/var/run/apache2$SUFFIX.pid
export APACHE_RUN_DIR=/var/run/apache2$SUFFIX
export APACHE_LOCK_DIR=/var/lock/apache2$SUFFIX
# Only /var/log/apache2 is handled by /etc/logrotate.d/apache2.
export APACHE_LOG_DIR=/var/log/apache2$SUFFIX

First step? Well we have to hook httpd startup to our new directory somehow. I don’t recall this part so well. I think I tried this from the command line:

$ apachectl -d /etc/apache2 -f apache2.conf -k start

and it may be at that point that I got the MPM workers error. But I forget. I switched to using the service command and that particular error seemed to go away at some point. I don’t believe I needed to do anything special.

So I tried this edit to /etc/sysconfig/httpd (sparing you the failed attempts):

OPTIONS=”-d /etc/apache2 -f apache2.conf”

Now we try to launch and see what happens.

$ service httpd start

Starting httpd: httpd: Syntax error on line 203 of /etc/apache2/apache2.conf: Syntax error on line 1 of /etc/apache2/mods-enabled/alias.load: Cannot load /usr/lib/apache2/modules/mod_alias.so into server: /usr/lib/apache2/modules/mod_alias.so: cannot open shared object file: No such file or directory
[FAILED]

Fasten your seatbelts, put on your big-boy pants or whatever. We’re just getting warmed up.

Let’s look at mods-available/alias.load:

$ more alias.load

LoadModule alias_module /usr/lib/apache2/modules/mod_alias.so

Sure enough, there is not only no such file, there is not even such a directory as /usr/lib/apache2. And all the load files have references like that. Where did the httpd install put its modules anyways? Why in /etc/httpd/modules. So I made a command decision:

$ mkdir /usr/lib/apache2
$ cd !$
$ ln -s /etc/httpd/modules

So where does that put us? Here:

$ service httpd start

Starting httpd: httpd: Syntax error on line 203 of /etc/apache2/apache2.conf: Syntax error on line 1 of /etc/apache2/mods-enabled/ssl.load: Cannot load /usr/lib/apache2/modules/mod_ssl.so into server: /usr/lib/apache2/modules/mod_ssl.so: cannot open shared object file: No such file or directory
     [FAILED]

Not everyone will see this issue. I had used SSL for some testing in Ubuntu so I had that module enabled. my CentOS is a core image and did not come with an SSL module. So let’s get it.

$ yum search mod_ssl

shows the full module name to be mod_ssl.x86_64, so we install it with yum install.

How far did that get us? To here:

$ service httpd start

Starting httpd: httpd: bad user name ${APACHE_RUN_USER}
  [FAILED]

Ah, remember my environment variables from above? As I said I actually use them with lines such as:

User ${APACHE_RUN_USER}

in apache2.conf. But clearly the definitions of those environment variables is not getting passed along. I decide to see if this step might work. I append these two lines to /etc/sysconfig/httpd:

$ Read in our environment variables. Inspired by apache on SLES.
. /etc/apache2/envvars

Could any more go wrong? Sure. Lots! Then there’s this:

$ service httpd start

Starting httpd: httpd: bad user name www-data
      [FAILED]

Amongst all the other stark differences, ubuntu and CentOS use different users to run apache. Great! So I create a www-data user as userid 33, gid 33 because that’s how it was under ubuntu. but GID 33 is already taken in CentOS. It is backup. I decide I will never use it that way, and change the group name to www-data.

That brings us here. you see I have a lot of patience…

$ service httpd start

Starting httpd: Syntax error on line 218 of /etc/apache2/apache2.conf:
Invalid command 'LogFormat', perhaps misspelled or defined by a module not included in the server configuration
   [FAILED]

Now my line 218 looks pretty regular. It’s simply:

LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined

I then realized something interesting. The modules built in to httpd on centOS and apache2 are different. apache2 seems to have some modules built in for logging:

$ apache2 -l

Compiled in modules:
  core.c
  mod_log_config.c
  mod_logio.c
  prefork.c
  http_core.c
  mod_so.c

whereas httpd does not:

$ httpd -l

Compiled in modules:
  core.c
  prefork.c
  http_core.c
  mod_so.c

So I made an empty log_config.conf and a log_config.load in mods-available that reads:

LoadModule log_config_module /usr/lib/apache2/modules/mod_log_config.so

I got the correct names by looking at the apache web site documenttion on that module. And i linked those two files up in the mods-available diretory:

$ cd mods-enabled
$ ln -s ../mods-available/log_config.conf
$ ln -s ../mods-available/log_config.load

Next error, please! Certainly. It is:

$ service httpd start

Starting httpd: Syntax error on line 218 of /etc/apache2/apache2.conf:
Unrecognized LogFormat directive %O
  [FAILED]

where line 218 is as recorded above. Well, some searches showed that you need the logio module. Note that it is also compiled into to apache2, but missing from httpd. So I did a similar thing with defining the necessary mods-{available,enabled} files. logio.load reads:

LoadModule logio_module /usr/lib/apache2/modules/mod_logio.so

The next?

$ service httpd start

Starting httpd: (2)No such file or directory: httpd: could not open error log file /var/log/apache2/error.log.
Unable to open logs
   [FAILED]

Oops. Didn’t make that directory. Naturally httpd and apache2 use different directories for logging. What else could you expect?

Now we’re down to this minimalist error:

$ service httpd start

Starting httpd:     [FAILED]

The error log contained this line:

[Mon Mar 19 14:11:14 2012] [error] (2)No such file or directory: Cannot create SSLMutex with file `/var/run/apache2/ssl_mutex'

After puzzling some over this what I eventually noticed is that my environment has references to directories which I haven’t defined yet:

export APACHE_RUN_DIR=/var/run/apache2$SUFFIX
export APACHE_LOCK_DIR=/var/lock/apache2$SUFFIX

So I created them.

And now I get:

$ service httpd start

Starting httpd:           [  OK  ]

But all is still not well. I cannot stop it the proper way. Trying to read its status goes like this:

$ service httpd status

httpd dead but subsys locked

I looked this one up. Killed off processes and semaphores as recommended with ipcs -s (see this link), etc. But since my case is different, I also did something different. I modified my /etc/init.d/httpd file:

#pidfile=${PIDFILE-/var/run/httpd/httpd.pid}
pidfile=${PIDFILE-/var/run/apache2.pid}
#lockfile=${LOCKFILE-/var/lock/subsys/httpd}
lockfile=${LOCKFILE-/var/lock/subsys/apache2}

Believe it or not, this worked. I can now run service httpd status and service httpd stop. To prove it:

$ service httpd status

httpd (pid  30366) is running...

Another Error Crops Up
I eventually noticed another problem with the web site. My trajectory page was not working. Upon investigation I found this comment in my main apache error log (not my virtual server error log, which I still don’t understand):

sh: /home/drj/traj/traj4.pl: Permission denied

This had to be a result of my call-out to a perl program from a php program:

...
$data = exec('/home/drj/traj/traj4.pl'.' '.$escargs);
...

But what’s so special about that call? Worked fine on Ubuntu, and I did a directory listing to show the file was really there. Well, here’s the thing, that file is under my home directory and guess what? When you crate your users in Ubuntu the home directory permissions are set to group and others read. Not in CentOS! A listing of /home looks kind of like this:

/home$ ll

total 12
drwx------ 2 drj   drj     4096 Mar 19 15:26 drj/
...

I set the permissions for all to read:

$ sudo chmod g+rx,o+rx drj

and I was good to go. The program began to work.

May 2013 Update
I was asked how all this survived after a yum update. Answer: pretty well, but not perfectly. The daemon was fine. And what miseld me is that it started fine. But then a couple days later I was looking at my access log and realized…it wasn’t there! Nor the errors log. Well, actually, the default access and error logs were there, but not for my virtual servers.

I soon realized that

$ service httpd status

produced

httpd dead but subsys locked

Well, who reads or remembers their own posts from a year ago? I totally forgot I had already dealt with this once, and my own post didn’t show up in my DDG search. Anywho, I stepped on the same rake twice. Being less patient this time around, probably because I am one year older, I simply altered the /etc/init.d/httpd file (looks like it had been changed by the update) thusly:

#pidfile=${PIDFILE-/var/run/httpd/httpd.pid}
#lockfile=${LOCKFILE-/var/lock/subsys/httpd}
# try as an experiment - DrJ 5/3/13
pidfile=/var/run/apache2.pid
lockfile=/var/lock/apache2/accept.lock

and I made sure I had a /var/lock/apache2 directory. This worked.

I chose a lock file with that particular name because I noticed this in my /etc/apache2/apache2.conf:

LockFile ${APACHE_LOCK_DIR}/accept.lock

To clean things out as I was working and re-working this problem since I couldn’t run

$ service httpd stop

I ran instead:

$ pkill -9 -f sbin/httpd

and I removed /var/run/apache2.pid.

Now, once again, I can get a status on my httpd service and restart works as well and my access and error logs are being written.

Conclusion
This conversion exercise turned out to be quite a teaching lesson and even after all this more remains. After the mysql migration I find the performance to be sub-par – about twice as slow as it was on Ubuntu.

Four months later, CentOS has not crashed once on me. Contrast that with Ubuntu freezing every two weeks or so. I optimized MySQL to cache some data and performance is adequate. I also have since learned about bitnami, which is kind of a stack for all the stuff I was using. Check out bitnami.org.