Raspberry Pi automates cable modem power cycling task

Intro
I lose my Internet far too often – sometimes once a day. Of course I have lots of network gear in a rat’s nest of cables. I narrowed the problem down to the cable modem, which simply needs to be power cycled and all is good. Most people would call their cable company at this point. I decided to make a little project of it to see if I could get my Raspberry Pi to

– monitor the Internet connection and
– automatically power-cycle the cable modem

Cool, right?

Needless to say, if I can power cycle a modem, I can control power to all kinds of devices with the Raspberry Pi.

Is there a product already on the market?
Why yes, there is. Normally that would shut me down in my tracks because what’s the point? But the product is relatively expensive – $100, so my DIY solution is considerably less since I already own the Pi. See references for a link to the commercial solution to this problem.


Getting a control cable

This is pathetic, but, I cut out a cable from an old computer that no longer works. The jumper has more pins than I need, but I could make it work.

Setting up my GPIO
I am plugged into the end so I need to manipulate GPIO pin 21.
Become root
$ sudo su –
Get to the right directory
$ cd /sys/class/gpio
Create the pin for user manipulation
$ echo 21 > export
Move to that pin’s directory
$ cd gpio21
Set up pin for sending signal OUT
$ echo out > direction
Test what we have so far
$ cat direction

out

$ cat value
0

connTest.sh script
I put this in /usr/local/etc and called it connTest.sh. I’m still tinkering with it a bit. But it shows what we’re basically trying to do.

#!/bin/sh
# Drj 10/2017
# Test if Internet connection is still good and send signal to relay if it is not
# see https://drjohnstechtalk.com/blog/2017/10/raspberry-pi-automates-cable-modem-power-cycling-task/?preview_id=3121&preview_nonce=9b896f248d&post_format=standard&_thumbnail_id=-1&preview=true
Break=300
Sleep=15
log=/var/log/connTest
#
# one-time setup of our GPIO pin so we can control it
pin=21
cd /sys/class/gpio
echo 21 > export
cd gpio$pin
echo out > direction
 
# divert STDOUT and STDERR to log file
exec 1>$log
exec 2>&1
echo "$0 starting monitoring at "`date`
while /bin/true; do
# curl returns with status 28 if it could not reach the nameserver or if it times out
curl -s --connect-timeout 5 www.google.com > /dev/null
if [ "$?" -eq "28" ]; then
  echo "We have a connection problem at "`date`
  echo "Power cycling router and waiting for $Break seconds"
# this will shut power off
  echo 1 > value
  sleep 2
# and this will turn it back on
  echo 0 > value
# this prevents us from too aggressively power-cycling
  sleep $Break
fi
sleep $Sleep
done

conn-test in /etc/init.d

#! /bin/sh
 
### BEGIN INIT INFO
# Provides:        conn-test
# Required-Start:  $network $remote_fs $syslog
# Required-Stop:   $network $remote_fs $syslog
# Default-Start:   2 3 4 5
# Default-Stop:
# Short-Description: Start conn-test daemon
### END INIT INFO
 
# /etc/init.d/conn-test
# 10/2017, DrJ
 
# The following part always gets executed.
echo "This part always gets executed"
 
# The following part carries out specific functions depending on arguments.
case "$1" in
  start)
    echo "Starting conn-test"
    start-stop-daemon -b -S -x /usr/local/etc/connTest.sh
    echo "conn-test is running"
    ;;
  stop)
    echo "Stopping conn-test"
    start-stop-daemon -K -x /usr/local/etc/connTest.sh
    pkill -f /usr/local/etc/connTest.sh
    echo "conn-test is dead"
    ;;
  status)
    start-stop-daemon -T connTest.sh
    ;;
  *)
    echo "Usage: /etc/init.d/conn-test {start|stop|status}"
    exit 1
    ;;
esac
 
exit 0

Command for loading init script
sudo systemctl daemon-reload

In all this I had the most trouble getting the startup script to bend to my will! But I think it’s functioning now. It may not be the most efficient, but it’s workable, meaning, it starts up connTest.sh after a reboot, and sends the log ot /var/log/connTest.

Substitute below for one thousand words

Raspberry Pi GPIO pins 21 plus ground connected to the power relay


So you can almost make out the different outlets from the power relay: always on; normally on; normally off. Makes perfect sense, right?
See that green plug on the side of the relay? I was such a newbie I was shoving the wires into it, unsure how to make a good connection. Well, with a little effort it simply pulls out, revealing a screws that can be used to secure the wires in the holes.

Conclusion
It’s fun to actually turn off and on 110V AC power using your Raspberry Pi! Especially when there is a useful purpose behind it such as a cable modem which starts to perform better after being power cycled. At only $30 this is a pretty affordable DIY project. I provide some scripts which shows how to work with GPIO pins using the command line. That turns out to be not so mysterious after all…
If the switching can work fast enough, I’m thinking of a next project with lights set to musical beats…!

References and related
Raspberry Pi model 2 and 3 GPIO pins are documented here: https://www.raspberrypi.org/documentation/usage/gpio-plus-and-raspi2/.
Generic GPIO documentation – how to use it from the operating system – is here: https://www.kernel.org/doc/Documentation/gpio/sysfs.txt
A PERL example of controlling GPIO I personally find too difficult to follow is here: https://raspberrypi.stackexchange.com/questions/41014/gpio-callbacks-in-any-language
The 110 volt AC relay device, controlled by DC signal of anywhere from 3.3 to 48 volt DC, is on Amazon: https://smile.amazon.com/gp/product/B00WV7GMA2/ref=oh_aui_detailpage_o00_s00?ie=UTF8&psc=1. this is a really sweet device. Perfect for hobbyists with either Raspberry Pi or Arduino. And only $29!
Product which does all this monitoring/power cycling for you automatically: https://smile.amazon.com/dp/B015NM0LKI/ref=dp_sp_detail?psc=1. But it’s $100.
Another alternative which would also work is switching over ethernet. This device does that: https://smile.amazon.com/dp/B00KQ4R1RK/ref=dp_sp_detail?psc=1&pd_rd_i=B00KQ4R1RK&pd_rd_wg=dxYdP&pd_rd_r=8TBE6X2F39S2XWX9CGKK&pd_rd_w=bmvYC. There is a simple CGI URL you can use to turn power off/on. But, again, it’s more costly: $75.
Need gpio cables? If you don’t happen to have a desktop computer you can cannibalize, then for $10 this Ras Pi expansion kit seems like a good way to go. It’s $10. https://smile.amazon.com/Kuman-Expansion-Raspberry-Solderless-Breadboard/dp/B074DSMPYD/ref=sr_1_1_sspa?ie=UTF8&qid=1509737721&sr=8-1-spons&keywords=GPIO+CABLES&psc=1

Posted in Linux, Raspberry Pi | Tagged | Leave a comment

Server Name Indication and what it means for those with only a single IP address

Intro
Sometimes everything is there in place, ready to be used, but you just have to either mistakenly try it, or learn it works by reading about it, because it may be counter-intuitive. Such is the case with Server Name Idndication. I thought I knew enough about https to “know” that you can only have one key/certificate for a single IP address. That CERT can be a SAN (subject alternative name) CERT covering multiple names, but you only get one shot at getting your certificate right. Or so I thought. Turns out I was dead wrong.

Some details
Well, SNI guess is a protocol extension to https. You know I always wondered why in proxy server logs it was able to log the domain name? How would it know that if the http protocol conversation is all encrypted? Maybe it’s SNI at work.

Since this is an extension it has to ne supported by both server and browser. It is. Apache24 supports it. IE, Firefox and Chrome support it. Even my venerable curl supports it!

This is mainly used by big hosting companies so they can easily and flexibly cram lots of web sites onto a single IP, but us small-time self-hosted sites benefit as well. I host a few sites for friends after all.

Testing methodology
This is pretty simple. I have a couple different virtual servers. I set each up with a completely different certificate in my apache virtual server setups. Then I accessed them by name like usual. Each showed me their own, proper, certificate. That’s it! So this is more than theoretical for me. I’ve already begun to use it.

Enterprise usage
F5 BigIP supports this protocol as well, of course. This article describes how to set it up. But it looks limited to only one server name per certificate, which will be inadequate if there are SAN certificates.

Conclusion
https using Server Name Indication allows to run multiple virtual servers, each with its own unique certificate, on a single IP address.

References and related
I get my certificates for free using the acme.sh interface to Let’s Encrypt
I’ve written some about apache 2.4 in this post
I don’t think Server Name Indication is explained very well anywhere that I’ve seen. The best dewscription I’ve found is that F5 Devcentral article: https://devcentral.f5.com/articles/ssl-profiles-part-7-server-name-indication
RFC 4366 is the spec describing Server Name Indication.

Posted in Admin, Apache, Hosting Service, Web Site Technologies | Tagged , | Leave a comment

Relative time with the linux date command

The situation
A server in Europe needs to transfer a log file which is written every hour from a server in the US. The filename format is
20171013-1039.log.gz
And we want the transfer to be done every hour.

How we did it
I learned something about the date command. I wanted to do date arithmetic, like calculate the previous hour. I’ve only ever done this in Perl. Then I saw how someone did it within a bash script.

First the timezone

export TZ=America/New_York

sets the timezone to that of the server which is writing the log files. This is important.

Then get the previous hour
$ onehourago=`date ‐‐date='1 hours ago' '+%Y%m%d‐%H'`

That’s it!

Then the ftp command looks like
$ get $onehourago

If we needed the log from two hours ago we would have had

$ twohourago=`date ‐‐date='2 hours ago' '+%Y%m%d‐%H'`

If one day ago

$ onedayago=`date ‐‐date='1 days ago' '+%Y%m%d‐%H'`

One hour from now

$ onedayago=`date ‐‐date='+1 hours' '+%Y%m%d‐%H'`

etc.

Why the timezone setting?
Initially I skipped the timezone setting and I simply put 7 hours ago, given that Europe and New York are six hours apart, and that’ll work 95% of the time. But because Daylight Savings time starts and ends at different times in the two continents, that will produce bad results for a few weeks a year. So it’s cleaner to simply switch the timezone before doing the date arithmetic.

Conclusion
The linux date command has more features than I thought. We’ve shown how to create some relative dates.

References and related
On a linux system
$ info date
will give you more information and lots of examples to work from.

Posted in Admin, Linux | Tagged | Leave a comment

Fail2ban fails to work, I built my own

Intro
I’ve sung the praises of fail2ban as a modern way to shutdown those annoying probes of your cloud server. I recently got to work with a Redhat v 7.4 system, so much newer than my old CentOS 6 server. And fail2ban failed even to work! Instead of the usual extensive debugging I just wrote my own. I’m sharing it here.

The details
I have a bare-bones RHEL 7.4 system. A yum search fail2ban does not find that package. Supposedly you simply need to add the EPEL repository to make that package available but the recipe on how to do that is not obvious. So I got the source for fail2ban and built it. Although it runs, you gotta build a local jail to block ssh attempts and that’s where it fails. So instead of going down that rabbit hole – I was already too deep, I decided to heck with it and I’m building my own.

All I really wanted was to ban IPs which are hitting my sshd server endlessly, often once per second or more. I take it personally.

RHEL 7 has a new firewall concept, firewalld. It’s all new to me and I don’t want to go down that rabbit hole either, at least not right down. So I rely on that old standard of mine: cut off an attacker by making an invalid route to his IP address, along the lines of

$ route add ‐host gw 127.0.0.1

And voila, they can no longer establish a TCP connection. It’s not quite as good as a firewall rule because their source UDP packets could still get through, but come on, we don’t need to be purists. And furthermore, in practice it produces the desired behaviour: stops the ssh dictionary attacks cold.

I knocked tghis out in one night, avoiding the rabbit hole of “fixing” fail2ban. So I had to use the old stuff I know so well, perl and stupid little tricks. I call drjfail2ban.

#!/bin/perl
# suppress IPs with failed logins
# DrJ - 2017/10/07
$DEBUG = 0;
$sleep = 30;
$cutoff = 3;
$headlines = 60;
@goodusers =("drjohn1","user57");
%blockedips = ();
while(1) {
#  $time = `date +%Y%m%d%H%M%S`;
  main();
  sleep($sleep);
}
 
sub main() {
if ($DEBUG) {
  for $ips (keys %blockedips) {
    print "blocked ip: $ips "
  }
}
# man last shows what this means: -i forces IP to be displayed, etc.
open(LINES,"last -$headlines -i -f /var/log/btmp|") || die "Problem with running last -f btmp!!\n";
# output:
#ubnt     ssh:notty    185.165.29.197   Sat Oct  7 19:30    gone - no logout
while(<LINES>) {
  ($user,$ip) = /^(\S+)\s+\S+\s+(\S+)/;
  print "user,ip: $user,$ip\n" if $DEBUG;
  next if $blockedips{$ip};
#we can't handle hostnames right now
  next if $ip =~ /[a-z]/i;
  $candidateips{$ip} += 1;
  $bannedusers{$ip} = $user;
}
for (keys %candidateips) {
  $ip = $_;
# allow my usual source IPs without blocking...
  next if $ip =~ /^(50\.17\.188\.196|51\.29\.208\.176)/;
  next if $blockedips{$ip};
  $usr = $bannedusers{$ip};
  $ipct = $candidateips{$ip};
  print "ip, usr, ipct: $ip, $usr, $ipct\n" if $DEBUG;
# block
  $block = 1;
  for $gu (@goodusers) {
    print "gu: $gu\n" if $DEBUG;
    $block = 0 if $usr eq $gu;
  }
  if ($block) {
# more tests: persistence of attempt
    $hitcnt = $candidateips{$ip};
    if ($hitcnt < $cutoff) {
# do not block and reset counter for next go-around
      print "Not blocking ip $ip and resetting counter\n" if $DEBUG;
      $candidateips{$ip} = 0;
    } else {
      $blockedips{$ip} = 1;
      print "Blocking ip $ip with hit count $hitcnt at " . `date`;
# prevent further communication...
      system("route add -host $ip gw 127.0.0.1");
   }
  }
  #print "route add -host $ip gw 127.0.0.1\n";
}
close(LINES);
} # end main function

Highlights from the program
The comments are pretty self-explanatory. Just a note about the philosophy. I fear making a goof and locking myself out! So I was conservative and try to not do any blocking if the source IP matches one of my favored source IPs, or if the user matches one of my usual usernames like drjohn1. I use obscure userids and the hackers try the stupid stuff like root, admin, etc. So they may be dictionary attacking the password, but they certainly aren’t dictionary attacking the username!

I don’t mind wiping the slate clean of all created routes after sever reboot so I only plan to run this from the command line. To make it persistent until the next reboot you just run it from the root account like so (let’s say we put it in /usr/local/sbin):

$ nohup /usr/local/sbin/drjfail2ban > /var/log/drjfail2ban &

And it just sits there and runs, even after you log out.

Results
Since it hasn’t been running for long I can provide a partial log file as of this publication.

Blocking ip 103.80.117.74 with hit count 6 at Sun Oct  8 17:34:43 CEST 2017
SIOCADDRT: File exists
Blocking ip 89.176.96.45 with hit count 5 at Sun Oct  8 17:34:43 CEST 2017
SIOCADDRT: File exists
Blocking ip 31.162.51.206 with hit count 3 at Sun Oct  8 17:34:43 CEST 2017
SIOCADDRT: File exists
Blocking ip 218.95.142.218 with hit count 6 at Sun Oct  8 17:34:43 CEST 2017
SIOCADDRT: File exists
Blocking ip 202.168.8.54 with hit count 5 at Sun Oct  8 17:34:43 CEST 2017
SIOCADDRT: File exists
Blocking ip 13.94.29.182 with hit count 4 at Sun Oct  8 17:34:43 CEST 2017
SIOCADDRT: File exists
Blocking ip 40.71.185.73 with hit count 4 at Sun Oct  8 17:34:43 CEST 2017
SIOCADDRT: File exists
Blocking ip 77.72.85.100 with hit count 13 at Sun Oct  8 17:34:43 CEST 2017
SIOCADDRT: File exists
Blocking ip 201.180.104.63 with hit count 7 at Sun Oct  8 17:34:43 CEST 2017
SIOCADDRT: File exists
Blocking ip 121.14.27.58 with hit count 4 at Sun Oct  8 17:40:43 CEST 2017
Blocking ip 36.108.234.99 with hit count 6 at Sun Oct  8 17:47:13 CEST 2017
Blocking ip 185.165.29.69 with hit count 6 at Sun Oct  8 18:02:43 CEST 2017
Blocking ip 190.175.40.195 with hit count 6 at Sun Oct  8 19:05:43 CEST 2017
Blocking ip 139.199.167.21 with hit count 4 at Sun Oct  8 19:29:13 CEST 2017
Blocking ip 186.60.67.51 with hit count 5 at Sun Oct  8 20:49:14 CEST 2017

And what my route table looks like currently:

$ netstat ‐rn|grep 127.0.0.1

Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
2.177.217.155   127.0.0.1       255.255.255.255 UGH       0 0          0 lo
13.94.29.182    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
31.162.51.206   127.0.0.1       255.255.255.255 UGH       0 0          0 lo
36.108.234.99   127.0.0.1       255.255.255.255 UGH       0 0          0 lo
37.204.23.84    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
40.71.185.73    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
42.7.26.15      127.0.0.1       255.255.255.255 UGH       0 0          0 lo
46.6.60.240     127.0.0.1       255.255.255.255 UGH       0 0          0 lo
59.16.74.234    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
77.72.85.100    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
89.176.96.45    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
103.80.117.74   127.0.0.1       255.255.255.255 UGH       0 0          0 lo
109.205.136.10  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
113.195.145.13  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
118.32.27.85    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
121.14.27.58    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
139.199.167.21  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
162.213.39.235  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
176.50.95.41    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
176.209.89.99   127.0.0.1       255.255.255.255 UGH       0 0          0 lo
181.113.82.213  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
185.165.29.69   127.0.0.1       255.255.255.255 UGH       0 0          0 lo
185.165.29.197  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
185.165.29.198  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
185.190.58.181  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
186.57.12.131   127.0.0.1       255.255.255.255 UGH       0 0          0 lo
186.60.67.51    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
190.42.185.25   127.0.0.1       255.255.255.255 UGH       0 0          0 lo
190.175.40.195  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
193.201.224.232 127.0.0.1       255.255.255.255 UGH       0 0          0 lo
201.180.104.63  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
201.255.71.14   127.0.0.1       255.255.255.255 UGH       0 0          0 lo
202.100.182.250 127.0.0.1       255.255.255.255 UGH       0 0          0 lo
202.168.8.54    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
203.190.163.125 127.0.0.1       255.255.255.255 UGH       0 0          0 lo
213.186.50.82   127.0.0.1       255.255.255.255 UGH       0 0          0 lo
218.95.142.218  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
221.192.142.24  127.0.0.1       255.255.255.255 UGH       0 0          0 lo

Here’s a partial listing of the many failed logins, just to keep it real:

...
root     ssh:notty    190.175.40.195   Sun Oct  8 19:05 - 19:28  (00:23)
root     ssh:notty    190.175.40.195   Sun Oct  8 19:05 - 19:05  (00:00)
root     ssh:notty    190.175.40.195   Sun Oct  8 19:05 - 19:05  (00:00)
root     ssh:notty    190.175.40.195   Sun Oct  8 19:05 - 19:05  (00:00)
root     ssh:notty    190.175.40.195   Sun Oct  8 19:05 - 19:05  (00:00)
root     ssh:notty    190.175.40.195   Sun Oct  8 19:05 - 19:05  (00:00)
admin    ssh:notty    185.165.29.69    Sun Oct  8 18:02 - 19:05  (01:02)
admin    ssh:notty    185.165.29.69    Sun Oct  8 18:02 - 18:02  (00:00)
admin    ssh:notty    185.165.29.69    Sun Oct  8 18:02 - 18:02  (00:00)
admin    ssh:notty    185.165.29.69    Sun Oct  8 18:02 - 18:02  (00:00)
root     ssh:notty    185.165.29.69    Sun Oct  8 18:02 - 18:02  (00:00)
root     ssh:notty    185.165.29.69    Sun Oct  8 18:02 - 18:02  (00:00)
root     ssh:notty    185.165.29.69    Sun Oct  8 18:02 - 18:02  (00:00)
root     ssh:notty    36.108.234.99    Sun Oct  8 17:47 - 18:02  (00:15)
root     ssh:notty    36.108.234.99    Sun Oct  8 17:47 - 17:47  (00:00)
root     ssh:notty    36.108.234.99    Sun Oct  8 17:47 - 17:47  (00:00)
root     ssh:notty    36.108.234.99    Sun Oct  8 17:47 - 17:47  (00:00)
root     ssh:notty    36.108.234.99    Sun Oct  8 17:47 - 17:47  (00:00)
root     ssh:notty    36.108.234.99    Sun Oct  8 17:46 - 17:47  (00:00)
ubuntu   ssh:notty    121.14.27.58     Sun Oct  8 17:40 - 17:46  (00:06)
ubuntu   ssh:notty    121.14.27.58     Sun Oct  8 17:40 - 17:40  (00:00)
aaaaaaaa ssh:notty    121.14.27.58     Sun Oct  8 17:40 - 17:40  (00:00)
aaaaaaaa ssh:notty    121.14.27.58     Sun Oct  8 17:40 - 17:40  (00:00)
root     ssh:notty    206.71.63.4      Sun Oct  8 17:34 - 17:40  (00:06)
root     ssh:notty    206.71.63.4      Sun Oct  8 17:34 - 17:34  (00:00)
root     ssh:notty    89.176.96.45     Sun Oct  8 16:15 - 17:34  (01:19)
root     ssh:notty    89.176.96.45     Sun Oct  8 16:15 - 16:15  (00:00)
root     ssh:notty    89.176.96.45     Sun Oct  8 16:15 - 16:15  (00:00)
root     ssh:notty    89.176.96.45     Sun Oct  8 16:15 - 16:15  (00:00)
...

Before running drjfail2ban it was much more obnoxious, with the same IP hitting my server every second or so.

Conclusion
I found it easier to roll my own than battle someone else’s errors. It’s kind of fun for me to create these little scripts. I don’t care if anyone else uses them. I will refer to this post myself and probably re-use it elsewhere!

References and related
In an earlier time, I was singing the praises of fail2ban on CentOS.

Posted in Admin, Linux, Security | Tagged | Leave a comment

Eclipse pics: just me and my cheese grater

Intro
I was at home during the great eclipse of August 2017. I didn’t buy the special viewing glasses. I remembered the general advice that anything with holes in it would show off the shape of the moon covering the sun by its shadow.

Initially I tried paper with three holes from a hole punch. Result were OK but not that impressive.

I was idly pulling disk=hes out of the dishwasher when I came across our cheese grater. Lots and lots of holes all over. This was the thing!

Everyone I’ve shown these two loved the novelty.

The setup
I’m standing just inside the garage – nice smooth surface for shadows – by the doorway. I find a way to simultaneously hold my phone and the cheese grater, and (mostly) get sufficiently out of the way that the cheese grater’s shadow is cast unobstructed onto the garage floor. It was a little tricky.

I’ll present the full picture and then the shadow of the cheese grater blown up at two different times – 2:27 PM and 2:41 PM EST.

And, yes, the holes in the grater are actualklky round and the normal shadow would be full of round circles of light where the sun passes through the grater’s holes!

It was a hot day. I could feel the air cool down during the eclipse! And cicadas started their shrill cry.

2:41 PM pics

Click on any of these pictures to see it blown up.

2:41 PM eclipse shadow pic


And focussing on the cheese grater’s shadow:

Cheese grater shadow during the eclipse


2:27 PM pics
This time is probably closer to the greatest coverage of the moon over the sun.

Eclipse full picture


Eclipse cheese grater pic

Conclusion
A common cheese grater gives us a good idea of what the sun looked like during the eclipse, and creates something marvelous at the same time.

Posted in Popular Science | Leave a comment

Verifying a pkcs12 file with openssl

Intro
The easy way

How to examine a pkcs12 (pfx) file

$ openssl pkcs12 ‐info ‐in file_name.pfx
It will prompt you for the password a total of three times!

The hard way
I went through this whole exercise because I originally could not find the easy way!!!

Get the source for openssl.

Look for pkread.c. Mine is in /usr/local/src/openssl/openssl-1.1.0f/demos/pkcs12.

Compile it.

My first pass:

$ gcc ‐o pkread pkread.c

/tmp/cclhy4wr.o: In function `sk_X509_num':
pkread.c:(.text+0x14): undefined reference to `OPENSSL_sk_num'
/tmp/cclhy4wr.o: In function `sk_X509_value':
pkread.c:(.text+0x36): undefined reference to `OPENSSL_sk_value'
/tmp/cclhy4wr.o: In function `main':
pkread.c:(.text+0x93): undefined reference to `OPENSSL_init_crypto'
pkread.c:(.text+0xa2): undefined reference to `OPENSSL_init_crypto'
pkread.c:(.text+0x10a): undefined reference to `d2i_PKCS12_fp'
pkread.c:(.text+0x154): undefined reference to `ERR_print_errors_fp'
pkread.c:(.text+0x187): undefined reference to `PKCS12_parse'
pkread.c:(.text+0x1be): undefined reference to `ERR_print_errors_fp'
pkread.c:(.text+0x1d4): undefined reference to `PKCS12_free'
pkread.c:(.text+0x283): undefined reference to `PEM_write_PrivateKey'
pkread.c:(.text+0x2bd): undefined reference to `PEM_write_X509_AUX'
pkread.c:(.text+0x320): undefined reference to `PEM_write_X509_AUX'
collect2: ld returned 1 exit status

Corrected compile command
$ gcc ‐o pkread pkread.c ‐I/usr/local/include ‐L/usr/local/lib64 ‐lssl ‐lcrypto

Note that this works for me because I put my ssl and crypto libraries in that directory. Your installation may have put them elsewhere:

$ ll /usr/local/lib64/*.a

-rw-r--r--. 1 root root 4793162 Aug 16 15:30 /usr/local/lib64/libcrypto.a
-rw-r--r--. 1 root root  765862 Aug 16 15:30 /usr/local/lib64/libssl.a

References and related
My favorite openssl commands.

Posted in Linux, Security | Tagged , | Leave a comment

Your AWS Instance was scheduled for retirement? don’t panic

Intro
After nearly four years of continuously running my AWS instance I got this scary email:

What to do?

The details
Since I never dveloped much AWS expertise (never needed to since it just worked) I was afraid to do anything. That’s sort of why I had kept it running for three and a half years – the last time I had to stop it didn’t work out so well.

Some terms
It helps to review the terms.
image – that’s like the OS. It has a unique identifier. Mine is ami-03559b6a.
instance – that’s a particular image running on a particular virtual server, identified by unique number. Mine is i-1737a673.
retired image – the owner of the image decided to no longer make it available for new instances

What it all means
I run a retired image, so for instance I can’t right-click my instance and:

– launch another like this

What I did to keep my instance running
I didn’t! Before the retirement deadline I stopped my instance. That is a painful process because it takes hours in my case. The server becomes unavailable quickly enough, but the status is stuck in state shutting down for at least a couple hours. But, eventually, it does shut down.

Then, I start it again. That’s it!

When it starts, AWS puts it on different hardware, etc, so I guess literally it is a different instance now, running the same image. I re-associate my elastic IP, and all is good.

So when the “retirement” date came along, there was no outage of my instance as I had already stopped/started it and that was all that was needed.
Amazon’s documentation – as good as it is – isn’t that clear on this point, hence this blog posting…

Side preparations
In case I couldn’t restart my image I had taken snapshots of my EBS volumes, and prepared to run Amazon Linux, which looks pretty similar to CentOS which is what I run. But, boy, learning about VPC and routing was a pain. I had to set all that up and gain at least a rudimentary understanding of all that. None of that existed six years ago when I started out! It was much simpler back then.

What it looks like

To make things concrete, here is my view on the AWS admin portal of my instances.

Conclusion
Having your Amazon AWS instance retired is not as scary as it initially sounds. Basically, just stop and start it yourself and you’ll be fine.

Posted in CentOS, Hosting Service | Tagged | 2 Comments

Getting your micro SD card ready to load Raspbian: easier than you thought

Intro
Configuring your own micro SD card in order to install Raspbian on a Raspberry Pi is not so hard. Some of the instructions out there are a bit dated and make it out to be harder than it really is.

The details
For instance this site has some extra steps you don’t need: http://elinux.org/RPi_Easy_SD_Card_Setup.

I’d stick with the simplest possible approach, which turns out to be this set of instructions: https://diyhacking.com/install-raspbian-raspberry-pis-sd-card/

But all these instructions seem to refer to an IMG file which I don’t even see. The main thing is to download NOOBS (new out-of-box software) from https://www.raspberrypi.org/downloads/ .

Then, get the SD card formatter. But the latest version is 5, not 4, and it looks different from before – there are essentially no options!

SD Card Formatter

So go with Quick Format and it works out OK. Unless yuor SD card is used. Then choose Overwrite format. That also works but takes a lot longer.

Then when it comes to copying the image file, which makes no sense with NOOBS because the image file is hidden, I think. Just extract all the files form the NOOBs zip file and copy them over to your E: drive, of whatever drive your SD card appears as.

Then follow the instructions on your Ras Pi display.

That’s it! I know because I just did it.

(non-)Reliability of SD Card
For the record, I’m in this situation because my old micro SD card just died. This is after running it continuously for a little over two years. Not very impressive in my book. Also for the record the card came as part of a Cana kit.

Symptoms of SD card failure in my case:

– boot paused, then after 120 seconds spits out some warnings about MMC something or other.
– LED status light solid green

References and related
This is probably the best guide, but I did not find a guide with 100% correct info; https://diyhacking.com/install-raspbian-raspberry-pis-sd-card/

Posted in Raspberry Pi | Tagged | 2 Comments

Compiling curl and openssl on Redhat Linux

Intro
I have an ancient Redhat system which I’m not in a position to upgrade. I like to use curl to test web sites, but it’s getting to the point that my ancient version has no SSL versions in common with some secure web sites. I desperately wanted to upgrade curl while leaving the rest of the system as is. Is it even possible? How would you do it? All these things and more are explained in today’s riveting blog post.

The details
Redhat version
I don’t know the proper command so I do this:
$ cat /etc/system-release

ed Hat Enterprise Linux Server release 6.6 (Santiago)

Current curl version
$ ./curl ‐‐version

curl 7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.16.2.3 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2

Limited set of SSL/TLS protocols
$ curl ‐help

...
 -2/--sslv2         Use SSLv2 (SSL)
 -3/--sslv3         Use SSLv3 (SSL)
...
 -z/--time-cond <time> Transfer based on a time condition
 -1/--tlsv1         Use TLSv1 (SSL)
...

New version of curl

curl 7.55.1 (x86_64-unknown-linux-gnu) libcurl/7.55.1 OpenSSL/1.1.0f zlib/1.2.3

New SSL options

     --ssl           Try SSL/TLS
     --ssl-allow-beast Allow security flaw to improve interop
     --ssl-no-revoke Disable cert revocation checks (WinSSL)
     --ssl-reqd      Require SSL/TLS
 -2, --sslv2         Use SSLv2
 -3, --sslv3         Use SSLv3
...
     --tls-max <VERSION> Use TLSv1.0 or greater
     --tlsauthtype <type> TLS authentication type
     --tlspassword   TLS password
     --tlsuser <name> TLS user name
 -1, --tlsv1         Use TLSv1.0 or greater
     --tlsv1.0       Use TLSv1.0
     --tlsv1.1       Use TLSv1.1
     --tlsv1.2       Use TLSv1.2
     --tlsv1.3       Use TLSv1.3

Now that’s an upgrade! How did we get to this point?

Well, I tried to get a curl RPM – seems like the appropriate path for a lazy system administrator, right? Well, not so fast. It’s not hard to find an RPM, but trying to install one showed a lot of missing dependencies, as in this example:
$ sudo rpm ‐i curl‐minimal‐7.55.1‐2.0.cf.fc27.x86_64.rpm

warning: curl-minimal-7.55.1-2.0.cf.fc27.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID b56a8bac: NOKEY
error: Failed dependencies:
        libc.so.6(GLIBC_2.14)(64bit) is needed by curl-minimal-7.55.1-2.0.cf.fc27.x86_64
        libc.so.6(GLIBC_2.17)(64bit) is needed by curl-minimal-7.55.1-2.0.cf.fc27.x86_64
        libcrypto.so.1.1()(64bit) is needed by curl-minimal-7.55.1-2.0.cf.fc27.x86_64
        libcurl(x86-64) >= 7.55.1-2.0.cf.fc27 is needed by curl-minimal-7.55.1-2.0.cf.fc27.x86_64
        libssl.so.1.1()(64bit) is needed by curl-minimal-7.55.1-2.0.cf.fc27.x86_64
        curl conflicts with curl-minimal-7.55.1-2.0.cf.fc27.x86_64

So I looked at the libcurl RPM, but it had its own set of dependencies. Pretty soon it looks like a full-time job to get this thing compiled!

I found the instructions mentioned in the reference, but they didn’t work for me exactly like that. Besides, i don’t have a working git program. So here’s what I did.

Compiling openssl

I downloaded the latest openssl, 1.1.0f, from https://www.openssl.org/source/ , untar it, go into the openssl-1.1.0f directory, and then:

$ ./config ‐Wl,‐‐enable‐new‐dtags ‐‐prefix=/usr/local/ssl ‐‐openssldir=/usr/local/ssl
$ make depend
$ make
$ sudo make install

So far so good.

Compiling zlib
For zlib I was lazy and mostly followed the other guy’s commands. Went something like this:
$ lib=zlib-1.2.11
$ wget http://zlib.net/$lib.tar.gz
$ tar xzvf $lib.tar.gz
$ mv $lib zlib
$ cd zlib
$ ./configure
$ make
$ cd ..
$ CD=$(pwd)

No problems there…

Compiling curl
curl was tricky and when I followed the guy’s instructions I got the very problem he sought to avoid.

vtls/openssl.c: In function ‘Curl_ossl_seed’:
vtls/openssl.c:276: error: implicit declaration of function ‘RAND_egd’
make[2]: *** [libcurl_la-openssl.lo] Error 1
make[2]: Leaving directory `/usr/local/src/curl/curl-7.55.1/lib'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/usr/local/src/curl/curl-7.55.1/lib'
make: *** [all-recursive] Error 1

I looked at the source and decided that what might help is to add a hint where the openssl stuff could be found.

Backing up a bit, I got the source from https://curl.haxx.se/download.html. I chose the file curl-7.55.1.tar.gz. Untar it, go into the curl-7.55.1 directory,
$ ./buildconf
$ PKG_CONFIG_PATH=/usr/local/ssl/lib/pkgconfig LIBS=”‐ldl”

and then – here is the single most important point in the whole blog – configure it thusly:

$ ./configure ‐‐with‐zlib=$CD/zlib ‐‐disable‐shared ‐‐with‐ssl=/usr/local/ssl

So my insight was to add the ‐‐with‐ssl=/usr/local/ssl to the configure command.

Then of course you make it:

$ make

and maybe even install it:

$ make install

This put curl into /usr/local/bin. I actually made a sym link and made this the default version with this kludge (the following commands were run as root):

$ cd /usr/bin; mv curl{,.orig}; ln ‐s /usr/local/bin/curl

That’s it! That worked and produced a working, modern curl.

By the way it mentions TLS1.3, but when you try to use it:

$ curl ‐i ‐k ‐‐tlsv1.3 https://drjohnstechtalk.com/

curl: (4) OpenSSL was built without TLS 1.3 support

It’s a no go. But at least TLS1.2 works just fine in this version.

One other thing – put shared libraries in a common area
I copied my compiled curl from Redhat to a SLES 11 SP 3 system. It didn’t quite run. Only thing is, it was missing the openssl libraries. So I guess it’s also important to copy over

libssl.so.1.1
libcrypto.so.1.1

to /usr/lib64 from /usr/local/lib64.

Once I did that, it worked like a charm!

Conclusion
We show how to compile the latest version of openssl and curl on an older Redhat 6.x OS. The motivation for doing so was to remain compatible with web sites which are already or soon dropping their support for TLS 1.0. With the compiled version curl and openssl supports TLS 1.2 which should keep it useful for a long while.

References and related
I closely followed the instructions in this stackoverflow post: https://stackoverflow.com/questions/44270707/cant-build-latest-libcurl-on-rhel-7-3#44297265
openssl source: https://www.openssl.org/source/
curl sources: https://curl.haxx.se/download.html
Here’s a web site that only supports TLS 1.2 which shows the problem: https://www.askapache.com/. You can see for yourself on ssllabs.com

Posted in Linux, SLES, Web Site Technologies | Tagged , , , | 1 Comment

Reverse upper and lower case letters

Intro
I am a look-at-the-keyboard typist! I can’t count the number of times I’ve begun an email with CAPs lock on, and found a nice correspondence looking like this:

hI aNDRES,
 
i RE-CREATED THE SCRIPT.
...

Tired of the re-typing I researched how to quickly repair this, at least for those with a linux command prompt at their disposal.

The details
I added this to my .aliases file:

# reverse upper/lower case
reverse () { tr 'A-Za-z' 'a-zA-Z'; }

Now when I make this mistake I put the text into my clipboard, even if it is multi-line, and fix it like this:

$ echo 'hI aNDRES,

i RE-CREATED THE SCRIPT.'|reverse

Hi Andres,
 
I re-created the script.

After the single quote I pasted in my clipboard text and closed that out with another single quote.

Alternative for those uncertain about Linux aliases
Alternatively, right on the command line you would just run

$ echo 'hI aNDRES,

i RE-CREATED THE SCRIPT.'|tr 'A-Za-z' 'a-zA-Z'

Conclusion
We developed an alias expression to make exchanging upper and lower case in a character string fast and easy, if you just remember a few things and have access to a Linux command prompt.

Posted in Admin, Linux | Tagged | Leave a comment