Raspberry Pi USB webcam turned into IP camera

Intro
Why would you even want to do this when you can buy a native IP webcam for less? I’m not sure, but i found myself in this situation so it could happen to others, and I found some things that worked and some that required quite some effort.

In my previous post I spoke about using opencv on Raspberry Pi.

This post is more about getting at an image with a minimum of lag time and relatively low bandwidth.

The setup
The specific camera I am working with is an ELP mini USB camera for $20.

What I did not do
I considered bolting on an add-on to opencv to convert the video stream into mjpeg. But the process looked relatively obscure so I did not feel that was a good way to go.

I skimmed through the mjpeg (motion jpeg) standard. Looks pretty straightforward. i even considered writing my own streamer. It’s probably not too hard to write a bad one! But I feared it would be unreliable so I didn’t go that route. It’s just jpeg, separator, jpeg, separator, jpeg, etc. Here’s the Wikipedia link: https://en.wikipedia.org/wiki/Motion_JPEG.

I think the best software for is mjpg_streamer. It is not available as a simple package. So you have to compile it and patch it.

Follow his recipe
This guy’s recipe worked for me:
https://jacobsalmela.com/2014/05/31/raspberry-pi-webcam-using-mjpg-streamer-over-internet/

Mostly! I needed the patch as well (which he also mentions). his instructions for the patch aren’t accurate.

He provides a link. You need to save the contents by launching the downloaded file and saving it as input_uvc_patch.txt after opening it in Windows Notepad (if you’re doing this download through Windows).

On the Pi, you would do these steps:

cd ~/mjpg-streamer
patch ‐p0 < input_uvc_patch.txt make USE_LIBV4L2=true clean all sudo make DESTDIR=/usr/local install That is, assuming you had copied the patch file into that ~/mjpg-streamer directory. Before we get too far, I wished to mention that the command fswebcam proved somewhat useful for debugging. Here's a weird thing about that camera We had one, then I got another one. The two cameras do not behave the same way! Device files
I guess Raspberry Pi has its own version of plug-and-play. So what it means is that when you plug in the camera a device file is dynamically created called /dev/video0. Now if you happen to plug in a second USB camera, that one becomes device /dev/video1. Some utilities are designed to work with /dev/video0 and require extra arguments to deal with a camera with a different device number, e.g., fswebcam -d /dev/video1 image.jpg.

But actually running two cameras did not work out too well for me. It seemed to crash and I don’t have time to investigate that.

The working command is…
My livestream.sh file looks like this right now. It will change but this is a good document point.

#!/bin/bash
/usr/local/bin/mjpg_streamer -i "/usr/local/lib/input_uvc.so -yuv -f 10 -r 640x480" -o "/usr/local/lib/output_http.so -w /usr/local/www"

The main point is that I found this additional -yuv argument seemed to get the one webcam to work, whereas the other USB camera didn’t need that! If you don’t include it launcher.sh may appear to work, but all you see when you connect to the direct video stream looks like this image:

One time when I ran it it crashed and suggested that -yuv argument be added, so I tried it and it actually worked! That’s how i discovered that oddity.

Bandwidth with those settings
About 2 mbps. How do I measure that? simple. I bring up the web page and tool around the networking stuff until i find Change Adapter Settings (always difficult to find). Then i double-click on my active adapter and stare at the received bytes to get a feel for how much it’s incrementing by each second. Multiply by 10, and voila, you have a crude measure, perhaps +/- 30%, of your bandwidth consumed!

Latency
This is so important it needs its own section.

Latency is pretty good. Maybe anywhere from 0.2 to 0.5 seconds. We’re still working on our measurement of it.

fswebcam errors
What happens if you run fswebcam while livestream is running?
$ fswebcam /tmp/image.jpg

--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
Error selecting input 0
VIDIOC_S_INPUT: Device or resource busy

Makes sense. Only one program on the Pi can capture the output form the camera.

Does the simple command fswebcam image.jpg work all the time? No it does not! Sometimes it simply fails, which is scary.

Here is an example of two consecutive calls to fswebcam about a second apart which illustrates the problem:

$ fswebcam /tmp/image.jpg

--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
Adjusting resolution from 384x288 to 352x288.
--- Capturing frame...
Timed out waiting for frame!
No frames captured.

$ fswebcam /tmp/image.jpg

--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
Adjusting resolution from 384x288 to 352x288.
--- Capturing frame...
Captured frame in 0.00 seconds.
--- Processing captured image...
Writing JPEG image to '/tmp/image.jpg'.

To be continued…

Posted in Linux, Raspberry Pi, Web Site Technologies | Tagged , | Leave a comment

A first taste of OpenCV on a Raspberry Pi 3

Intro
I’ve done a few things to do some vision processing with OpenCV on a Raspberry Pi 3. I am a rank amateur so my meager efforts will not be of much help to anyone else. My idea is that maybe this could be used on an FRC First Robotics team’s robot. Hence I will be getting into some tangential areas where I am more comfortable.

Even though this is a work in progress I wanted to get some of it down before I forget what I’ve done so far!

Tangential Stuff

Disable WiFi
You shouldn’t have peripheral devices with WiFi enabled. Raspeberry Pi 3 comes with built-in WiFi. Here’s how to turn it off.

Add the following line to your /boot/config.txt file:

dtoverlay=pi3‐disable‐wifi

Reboot.

If it worked you should only see the loopback and eth0 interefaces in response to the ip link command, something like this:

$ ip link
1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
link/ether b8:27:eb:3f:92:f3 brd ff:ff:ff:ff:ff:ff

Hardcode an IP address the simple-minded way
On a lark I decided to try the old-fashioned method I first used on Sun Solaris, or was it even Dec Ultrix? That is, ifconfig. I thought it was to be deprecated but it works well enough for my purpose.

So something like

$ sudo ifconfig eth0 192.168.1.160

does the job, as long as the network interface is up and connected.

Autolaunch a VNC Server so we can haul the camera image back to the driver station
$ vncserver &hypher;geometry 640×480 ‐Authentication=VncAuth :1

Launch our python-based opencv program and send output to VNC virtual display

$ export DISPLAY=:1
$ /home/pi/.virtualenvs/cv/bin/python green.py > /tmp/green.log 2>&1 &

The above was just illustrative. What I actually have is a single script, launcher.sh which puts it all together. Here it is.

#!/bin/sh
# DrJ
sleep 2
# set a hard-wired IP - this will have to change!!!
sudo ifconfig eth0 192.168.1.160
# launch small virtual vncserver on DISPLAY 1
vncserver -Authentication=VncAuth :1
# launch UDP server
$HOME/server.py > /tmp/server.log 2>&1 &
# run virtual env
cd $HOME
# don't need virtualenv if we use this version of python...
#. /home/pi/.profile
#workon cv
#
# now launch our python video capture program
#
export DISPLAY=:1
/home/pi/.virtualenvs/cv/bin/python green.py > /tmp/green.log 2>&1 &

OpenCV (open computer Vision)
opencv is a bear and you have to really work to get it onto a Pi 3. There is no apt-get install opencv. You have to download and compile the thing. There are many steps and few accurate documentation sources on the Internet as of this writing (January 2018).

I think this guide by Adrian is the best guide:

Install guide: Raspberry Pi 3 + Raspbian Jessie + OpenCV 3

However I believe I still ran into trouble and I needed this cmake command in stead of the one he provides:

cmake -D CMAKE_BUILD_TYPE=RELEASE \
        -D CMAKE_INSTALL_PREFIX=/usr/local \
        -D INSTALL_C_EXAMPLES=OFF \
        -D ENABLE_PRECOMPILED_HEADERS=OFF \
        -D INSTALL_PYTHON_EXAMPLES=ON \
        -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib-3.1.0/modules \
        -D BUILD_EXAMPLES=ON ..

I also replaced opencv references to version 3.0.0 with 3.1.0.

I also don’t think I got make -j4 to work. Just plain make.

An interesting getting started tutorial on images, opencv, and python:

http://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_gui/py_image_display/py_image_display.html#display-image

Simplifying launch of VNC Viewer
I wrote a simple-minded DOS script which launches UltraVNC with a password. So with a double-click it should work).

Here’s a Dos .bat file to launch ultravnc viewer by double-clicking on it.

if not "%minimized%"=="" goto :minimized
set minimized=true
start /min cmd /C "%~dpnx0"
goto :EOF
:minimized
c:\apps\ultravnc\vncviewer -password raspberry 192.168.1.160:1

I’m sure there’s a better way but I don’t know it.

The setup
We have a USB camera plugged into the Pi.
A green disc LED light.
A green filter over the camera lens.
A target with two parallel strips of retro-reflective tape we are trying to suss out from everything else.
Some sliders to control the sensitivity of our color matching.
The request to analyze the video in opencv as well as display it on the driver station.
Have opencv calculate the pixel distance (“correction”) from image center of the “target” (the two parallel strips).
Send this correction via a UDP server to any client who wants to know the correction.

Here is our current python program green.py which does these things.

import Tkinter as tk
from threading import Thread,Event
from multiprocessing import Array
from ctypes import c_int32
import cv2
import numpy as np
import sys
#from Tkinter import *
#cap = cv2.VideoCapture(0)
global x
global f
x = 1
y = 1
f = "green.txt"
 
class CaptureController(tk.Frame):
    NSLIDERS = 7
    def __init__(self,parent):
        tk.Frame.__init__(self)
        self.parent = parent
 
        # create a synchronised array that other threads will read from
        self.ar = Array(c_int32,self.NSLIDERS)
 
        # create NSLIDERS Scale widgets
        self.sliders = []
        for ii in range(self.NSLIDERS):
            # through the command parameter we ensure that the widget updates the sync'd array
            s = tk.Scale(self, from_=0, to=255, length=650, orient=tk.HORIZONTAL,
                         command=lambda pos,ii=ii:self.update_slider(ii,pos))
            if ii == 0:
                s.set(0)  #green min
            elif ii == 1:
                s.set(0)
            elif ii == 2:
                s.set(250)
            elif ii == 3:
                s.set(3)  #green max
            elif ii == 4:
                s.set(255)
            elif ii == 5:
                s.set(255)
            elif ii == 6:
                s.set(249)  #way down below
            s.pack()
            self.sliders.append(s)
 
        # Define a quit button and quit event to help gracefully shut down threads
        tk.Button(self,text="Quit",command=self.quit).pack()
        self._quit = Event()
        self.capture_thread = None
 
    # This function is called when each Scale widget is moved
    def update_slider(self,idx,pos):
        self.ar[idx] = c_int32(int(pos))
 
    # This function launches a thread to do video capture
    def start_capture(self):
        self._quit.clear()
        # Create and launch a thread that will run the video_capture function
#        self.capture_thread = Thread(cap = cv2.VideoCapture(0), args=(self.ar,self._quit))
        self.capture_thread = Thread(target=video_capture, args=(self.ar,self._quit))
        self.capture_thread.daemon = True
        self.capture_thread.start()
 
    def quit(self):
        self._quit.set()
        try:
            self.capture_thread.join()
        except TypeError:
            pass
        self.parent.destroy()
 
# This function simply loops over and over, printing the contents of the array to screen
def video_capture(ar,quit):
    print ar[:]
    cap = cv2.VideoCapture(0)
    Xerror = 0
    Yerror = 0
    XerrorStr = '0'
    YerrorStr = '0'
    while not quit.is_set():
        # the slider values are all readily available through the indexes of ar
        # i.e. w1 = ar[0]
        # w2 = ar[1]
        # etc.
        # Take each frame
        _, frame = cap.read()
        # Convert BGR to HSV
        hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
        # define range of blue color in HSV
        lower_green = np.array([ar[0],ar[1],ar[2]])
        upper_green = np.array([ar[3],ar[4],ar[5]])
        # Threshold the HSV image to get only green colors
        mask = cv2.inRange(hsv, lower_green, upper_green)
        # Bitwise-AND mask and original image
        res = cv2.bitwise_and(frame,frame, mask= mask)
        cv2.imshow('frame', frame)
#        cv2.imshow('mask',mask)
#        cv2.imshow('res',res)
        #------------------------------------------------------------------
        img = cv2.blur(mask,(5,5))   #filter (blur) image to reduce errors
        cv2.imshow('img',img)
        ret,thresh = cv2.threshold(img,127,255,0)
        im2,contours,hierarchy = cv2.findContours(thresh, cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
        print 'number of contours==640x480====================  ', len(contours)
        target=0
        if len(contours) > 0:
            numbercontours = len(contours)
            while numbercontours > 0:
                numbercontours = numbercontours -1  # contours start at 0
                cnt = contours[numbercontours]   #this is  getting the first contour found, could look at 1,2,3 etc
                x,y,w,h = cv2.boundingRect(cnt)
#
#---line below has the limits of the area of the target-----------------------
#
                #if w * h > 4200 and w * h < 100000:  #area of capture must exceed  to exit loop
                if h > 30 and w < h/3:  #area of capture must exceed  to exit loop
                    print ' X   Y  W  H  AREA      Xc  Yc      xEr yEr'
                    Xerror = (-1) * (320 - (x+(w/2)))
                    XerrorStr = str(Xerror)
                    Yerror = 240 - (y+(h/2))
                    YerrorStr = str(Yerror)
                    print  x,y,w,h,(w*h),'___',(x+(w/2)),(y+(h/2)),'____',Xerror,Yerror
                    break
 
#-------        draw horizontal and vertical center lines below
                cv2.line(img,(320,0),(320,480),(135,0,0),5)
                cv2.line(img,(0,240),(640,240),(135,0,0),5)
                displaySTR = XerrorStr + '  ' + YerrorStr
                font = cv2.FONT_HERSHEY_SIMPLEX
                cv2.putText(img,displaySTR,(10,30), font, .75,(255,255,255),2,cv2.LINE_AA)
                cv2.imshow('img',img)
# wrtie to file for our server'
                sys.stdout = open(f,"w")
                print 'H,V:',Xerror,Yerror
                sys.stdout = sys.__stdout__
                target=1
                #
                #--------------------------------------------------------------------
        if target==0:
                # no target found. print non-physical values out to a file
                sys.stdout = open(f,"w")
                print 'H,V:',1000,1000
                sys.stdout = sys.__stdout__
        k = cv2.waitKey(1) & 0xFF    #parameter is wait in millseconds
        if k == 27:   # esc key on keboard
            cap.release()
            cv2.destroyAllWindows()
            break
 
if __name__ == "__main__":
    root = tk.Tk()
    selectors = CaptureController(root)
    selectors.pack()
#    q = tk.Label(root, text=str(x))
#    q.pack()
    selectors.start_capture()
    root.mainloop()

Well, that was a big program by my standards.

Here’s the UDP server that goes with it. I call it server.py.

#!/usr/bin/env python
# inspired by https://gist.github.com/Manouchehri/67b53ecdc767919dddf3ec4ea8098b20
# first we get client connection, then we read data frmo file. This order is important so we get the latest, freshest data!
 
 
import socket
import re
 
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
 
server_address = '0.0.0.0'
server_port = 5005
 
server = (server_address, server_port)
sock.bind(server)
print("Listening on " + server_address + ":" + str(server_port))
 
while True:
# read up to 32 bytes from client
        payload, client_address = sock.recvfrom(32)
        print("Request from client: " + payload)
# get correction from file
        while True:
                with open('green.txt','r') as myfile:
                        data=myfile.read()
#H,V:  9 -14
                data = data.split(":")
                if len(data) == 2:
                        break
        sent = sock.sendto(data[1], client_address)

For development testing I wrote a UDP client to go along with that server. I called it recvudp.py.

#!/usr/bin/env python
import socket
UDP_IP = "127.0.0.1"
UDP_PORT = 5005
 
print "UDP target IP:", UDP_IP
print "UDP target port:", UDP_PORT
 
sock = socket.socket(socket.AF_INET, # Internet
                 socket.SOCK_DGRAM) # UDP
# need to send one newline minimum to receive server's message...
MESSAGE = "correction";
sock.sendto(MESSAGE, (UDP_IP, UDP_PORT))
# get data
data, addr = sock.recvfrom(1024) # buffer size is 1024 bytes
print "received message:", data

Problems
Lag is bad. Probably 1.5 seconds or so.
Video is green, but then we designed it that way.
Bandwidth consumption of VNC is way too high. We’re supposed to be under 7 mbps and it is closer to 12 mbps right now.
Probably won’t work under the bright lights or an arena or gym.
Sliders should be labelled.
Have to turn a pixel correction into an angle.
Have to suppress initial warning about ssh default password.

To be improved, hopefully…

Posted in Linux, Python, Raspberry Pi | Tagged | Leave a comment

Linux Tip: too lazy to write a startup script? how to fake it

Intro
systemd can be pretty formidable to master. Say you have your own little script you like to run but you don’t want to bother with inserting it into the systemd facility. What can you do?

The details
A simple trick is to insert the startup of the script into a crontab like this:

@reboot <path-to-your-script>

For more details on how and why this works and some other crontab oddities try

$ man ‐s5 crcontab

An old Unix hand pointed this out to me recently. I am going to make a lot more use of it…

Of course niceties such as run levels, order of startup, etc are not really controllable (I guess). Or maybe it is the case that all your @reboot scripts are processed in order, top to bottom.

Posted in Admin, Linux | Leave a comment

Raspberry Pi automates cable modem power cycling task

Intro
I lose my Internet far too often – sometimes once a day. Of course I have lots of network gear in a rat’s nest of cables. I narrowed the problem down to the cable modem, which simply needs to be power cycled and all is good. Most people would call their cable company at this point. I decided to make a little project of it to see if I could get my Raspberry Pi to

– monitor the Internet connection and
– automatically power-cycle the cable modem

Cool, right?

Needless to say, if I can power cycle a modem, I can control power to all kinds of devices with the Raspberry Pi.

Is there a product already on the market?
Why yes, there is. Normally that would shut me down in my tracks because what’s the point? But the product is relatively expensive – $100, so my DIY solution is considerably less since I already own the Pi. See references for a link to the commercial solution to this problem.


Getting a control cable

This is pathetic, but, I cut out a cable from an old computer that no longer works. The jumper has more pins than I need, but I could make it work.

Setting up my GPIO
I am plugged into the end so I need to manipulate GPIO pin 21.
Become root
$ sudo su –
Get to the right directory
$ cd /sys/class/gpio
Create the pin for user manipulation
$ echo 21 > export
Move to that pin’s directory
$ cd gpio21
Set up pin for sending signal OUT
$ echo out > direction
Test what we have so far
$ cat direction

out

$ cat value
0

connTest.sh script
I put this in /usr/local/etc and called it connTest.sh. I’m still tinkering with it a bit. But it shows what we’re basically trying to do.

#!/bin/sh
# Drj 10/2017
# Test if Internet connection is still good and send signal to relay if it is not
# see https://drjohnstechtalk.com/blog/2017/10/raspberry-pi-automates-cable-modem-power-cycling-task/?preview_id=3121&preview_nonce=9b896f248d&post_format=standard&_thumbnail_id=-1&preview=true
Break=300
Sleep=15
log=/var/log/connTest
#
# one-time setup of our GPIO pin so we can control it
pin=21
cd /sys/class/gpio
echo 21 > export
cd gpio$pin
echo out > direction
 
# divert STDOUT and STDERR to log file
exec 1>$log
exec 2>&1
echo "$0 starting monitoring at "`date`
while /bin/true; do
# curl returns with status 28 if it could not reach the nameserver or if it times out
curl -s --connect-timeout 5 www.google.com > /dev/null
if [ "$?" -eq "28" ]; then
  echo "We have a connection problem at "`date`
  echo "Power cycling router and waiting for $Break seconds"
# this will shut power off
  echo 1 > value
  sleep 2
# and this will turn it back on
  echo 0 > value
# this prevents us from too aggressively power-cycling
  sleep $Break
fi
sleep $Sleep
done

conn-test in /etc/init.d

#! /bin/sh
 
### BEGIN INIT INFO
# Provides:        conn-test
# Required-Start:  $network $remote_fs $syslog
# Required-Stop:   $network $remote_fs $syslog
# Default-Start:   2 3 4 5
# Default-Stop:
# Short-Description: Start conn-test daemon
### END INIT INFO
 
# /etc/init.d/conn-test
# 10/2017, DrJ
 
# The following part always gets executed.
echo "This part always gets executed"
 
# The following part carries out specific functions depending on arguments.
case "$1" in
  start)
    echo "Starting conn-test"
    start-stop-daemon -b -S -x /usr/local/etc/connTest.sh
    echo "conn-test is running"
    ;;
  stop)
    echo "Stopping conn-test"
    start-stop-daemon -K -x /usr/local/etc/connTest.sh
    pkill -f /usr/local/etc/connTest.sh
    echo "conn-test is dead"
    ;;
  status)
    start-stop-daemon -T connTest.sh
    ;;
  *)
    echo "Usage: /etc/init.d/conn-test {start|stop|status}"
    exit 1
    ;;
esac
 
exit 0

Command for loading init script
sudo systemctl daemon-reload

In all this I had the most trouble getting the startup script to bend to my will! But I think it’s functioning now. It may not be the most efficient, but it’s workable, meaning, it starts up connTest.sh after a reboot, and sends the log to /var/log/connTest.

after about a monht my conntest file looks like this:

< pre lang="text">
./connTest.sh starting monitoring at Wed 1 Nov 06:29:50 EDT 2017
/usr/local/etc/connTest.sh starting monitoring at Wed 1 Nov 18:44:18 EDT 2017
/usr/local/etc/connTest.sh starting monitoring at Wed 1 Nov 18:48:20 EDT 2017
/usr/local/etc/connTest.sh starting monitoring at Wed 1 Nov 18:54:19 EDT 2017
We have a connection problem at Sat 25 Nov 20:41:21 EST 2017


Substitute below for one thousand words

Raspberry Pi GPIO pins 21 plus ground connected to the power relay


So you can almost make out the different outlets from the power relay: always on; normally on; normally off. Makes perfect sense, right?
See that green plug on the side of the relay? I was such a newbie I was shoving the wires into it, unsure how to make a good connection. Well, with a little effort it simply pulls out, revealing a screws that can be used to secure the wires in the holes.

Conclusion
It’s fun to actually turn off and on 110V AC power using your Raspberry Pi! Especially when there is a useful purpose behind it such as a cable modem which starts to perform better after being power cycled. At only $30 this is a pretty affordable DIY project. I provide some scripts which shows how to work with GPIO pins using the command line. That turns out to be not so mysterious after all…
If the switching can work fast enough, I’m thinking of a next project with lights set to musical beats…!

References and related
Raspberry Pi model 2 and 3 GPIO pins are documented here: https://www.raspberrypi.org/documentation/usage/gpio-plus-and-raspi2/.
Generic GPIO documentation – how to use it from the operating system – is here: https://www.kernel.org/doc/Documentation/gpio/sysfs.txt
A PERL example of controlling GPIO I personally find too difficult to follow is here: https://raspberrypi.stackexchange.com/questions/41014/gpio-callbacks-in-any-language
The 110 volt AC relay device, controlled by DC signal of anywhere from 3.3 to 48 volt DC, is on Amazon: https://smile.amazon.com/gp/product/B00WV7GMA2/ref=oh_aui_detailpage_o00_s00?ie=UTF8&psc=1. this is a really sweet device. Perfect for hobbyists with either Raspberry Pi or Arduino. And only $29!
Product which does all this monitoring/power cycling for you automatically: https://smile.amazon.com/dp/B015NM0LKI/ref=dp_sp_detail?psc=1. But it’s $100.
Another alternative which would also work is switching over ethernet. This device does that: https://smile.amazon.com/dp/B00KQ4R1RK/ref=dp_sp_detail?psc=1&pd_rd_i=B00KQ4R1RK&pd_rd_wg=dxYdP&pd_rd_r=8TBE6X2F39S2XWX9CGKK&pd_rd_w=bmvYC. There is a simple CGI URL you can use to turn power off/on. But, again, it’s more costly: $75.
Need gpio cables? If you don’t happen to have a desktop computer you can cannibalize, then for $10 this Ras Pi expansion kit seems like a good way to go. It’s $10. https://smile.amazon.com/Kuman-Expansion-Raspberry-Solderless-Breadboard/dp/B074DSMPYD/ref=sr_1_1_sspa?ie=UTF8&qid=1509737721&sr=8-1-spons&keywords=GPIO+CABLES&psc=1

Posted in Linux, Raspberry Pi | Tagged | Leave a comment

Server Name Indication and what it means for those with only a single IP address

Intro
Sometimes everything is there in place, ready to be used, but you just have to either mistakenly try it, or learn it works by reading about it, because it may be counter-intuitive. Such is the case with Server Name Idndication. I thought I knew enough about https to “know” that you can only have one key/certificate for a single IP address. That CERT can be a SAN (subject alternative name) CERT covering multiple names, but you only get one shot at getting your certificate right. Or so I thought. Turns out I was dead wrong.

Some details
Well, SNI guess is a protocol extension to https. You know I always wondered why in proxy server logs it was able to log the domain name? How would it know that if the http protocol conversation is all encrypted? Maybe it’s SNI at work.

Since this is an extension it has to ne supported by both server and browser. It is. Apache24 supports it. IE, Firefox and Chrome support it. Even my venerable curl supports it!

This is mainly used by big hosting companies so they can easily and flexibly cram lots of web sites onto a single IP, but us small-time self-hosted sites benefit as well. I host a few sites for friends after all.

Testing methodology
This is pretty simple. I have a couple different virtual servers. I set each up with a completely different certificate in my apache virtual server setups. Then I accessed them by name like usual. Each showed me their own, proper, certificate. That’s it! So this is more than theoretical for me. I’ve already begun to use it.

Enterprise usage
F5 BigIP supports this protocol as well, of course. This article describes how to set it up. But it looks limited to only one server name per certificate, which will be inadequate if there are SAN certificates.

Conclusion
https using Server Name Indication allows to run multiple virtual servers, each with its own unique certificate, on a single IP address.

References and related
I get my certificates for free using the acme.sh interface to Let’s Encrypt
I’ve written some about apache 2.4 in this post
I don’t think Server Name Indication is explained very well anywhere that I’ve seen. The best dewscription I’ve found is that F5 Devcentral article: https://devcentral.f5.com/articles/ssl-profiles-part-7-server-name-indication
RFC 4366 is the spec describing Server Name Indication.

Posted in Admin, Apache, Hosting Service, Web Site Technologies | Tagged , | Leave a comment

Relative time with the linux date command

The situation
A server in Europe needs to transfer a log file which is written every hour from a server in the US. The filename format is
20171013-1039.log.gz
And we want the transfer to be done every hour.

How we did it
I learned something about the date command. I wanted to do date arithmetic, like calculate the previous hour. I’ve only ever done this in Perl. Then I saw how someone did it within a bash script.

First the timezone

export TZ=America/New_York

sets the timezone to that of the server which is writing the log files. This is important.

Then get the previous hour
$ onehourago=`date ‐‐date='1 hours ago' '+%Y%m%d‐%H'`

That’s it!

Then the ftp command looks like
$ get $onehourago

If we needed the log from two hours ago we would have had

$ twohourago=`date ‐‐date='2 hours ago' '+%Y%m%d‐%H'`

If one day ago

$ onedayago=`date ‐‐date='1 days ago' '+%Y%m%d‐%H'`

One hour from now

$ onedayago=`date ‐‐date='+1 hours' '+%Y%m%d‐%H'`

etc.

Why the timezone setting?
Initially I skipped the timezone setting and I simply put 7 hours ago, given that Europe and New York are six hours apart, and that’ll work 95% of the time. But because Daylight Savings time starts and ends at different times in the two continents, that will produce bad results for a few weeks a year. So it’s cleaner to simply switch the timezone before doing the date arithmetic.

Conclusion
The linux date command has more features than I thought. We’ve shown how to create some relative dates.

References and related
On a linux system
$ info date
will give you more information and lots of examples to work from.

Posted in Admin, Linux | Tagged | Leave a comment

Fail2ban fails to work, I built my own

Intro
I’ve sung the praises of fail2ban as a modern way to shutdown those annoying probes of your cloud server. I recently got to work with a Redhat v 7.4 system, so much newer than my old CentOS 6 server. And fail2ban failed even to work! Instead of the usual extensive debugging I just wrote my own. I’m sharing it here.

The details
I have a bare-bones RHEL 7.4 system. A yum search fail2ban does not find that package. Supposedly you simply need to add the EPEL repository to make that package available but the recipe on how to do that is not obvious. So I got the source for fail2ban and built it. Although it runs, you gotta build a local jail to block ssh attempts and that’s where it fails. So instead of going down that rabbit hole – I was already too deep, I decided to heck with it and I’m building my own.

All I really wanted was to ban IPs which are hitting my sshd server endlessly, often once per second or more. I take it personally.

RHEL 7 has a new firewall concept, firewalld. It’s all new to me and I don’t want to go down that rabbit hole either, at least not right down. So I rely on that old standard of mine: cut off an attacker by making an invalid route to his IP address, along the lines of

$ route add ‐host gw 127.0.0.1

And voila, they can no longer establish a TCP connection. It’s not quite as good as a firewall rule because their source UDP packets could still get through, but come on, we don’t need to be purists. And furthermore, in practice it produces the desired behaviour: stops the ssh dictionary attacks cold.

I knocked tghis out in one night, avoiding the rabbit hole of “fixing” fail2ban. So I had to use the old stuff I know so well, perl and stupid little tricks. I call drjfail2ban.

#!/bin/perl
# suppress IPs with failed logins
# DrJ - 2017/10/07
$DEBUG = 0;
$sleep = 30;
$cutoff = 3;
$headlines = 60;
@goodusers =("drjohn1","user57");
%blockedips = ();
while(1) {
#  $time = `date +%Y%m%d%H%M%S`;
  main();
  sleep($sleep);
}
 
sub main() {
if ($DEBUG) {
  for $ips (keys %blockedips) {
    print "blocked ip: $ips "
  }
}
# man last shows what this means: -i forces IP to be displayed, etc.
open(LINES,"last -$headlines -i -f /var/log/btmp|") || die "Problem with running last -f btmp!!\n";
# output:
#ubnt     ssh:notty    185.165.29.197   Sat Oct  7 19:30    gone - no logout
while(<LINES>) {
  ($user,$ip) = /^(\S+)\s+\S+\s+(\S+)/;
  print "user,ip: $user,$ip\n" if $DEBUG;
  next if $blockedips{$ip};
#we can't handle hostnames right now
  next if $ip =~ /[a-z]/i;
  $candidateips{$ip} += 1;
  $bannedusers{$ip} = $user;
}
for (keys %candidateips) {
  $ip = $_;
# allow my usual source IPs without blocking...
  next if $ip =~ /^(50\.17\.188\.196|51\.29\.208\.176)/;
  next if $blockedips{$ip};
  $usr = $bannedusers{$ip};
  $ipct = $candidateips{$ip};
  print "ip, usr, ipct: $ip, $usr, $ipct\n" if $DEBUG;
# block
  $block = 1;
  for $gu (@goodusers) {
    print "gu: $gu\n" if $DEBUG;
    $block = 0 if $usr eq $gu;
  }
  if ($block) {
# more tests: persistence of attempt
    $hitcnt = $candidateips{$ip};
    if ($hitcnt < $cutoff) {
# do not block and reset counter for next go-around
      print "Not blocking ip $ip and resetting counter\n" if $DEBUG;
      $candidateips{$ip} = 0;
    } else {
      $blockedips{$ip} = 1;
      print "Blocking ip $ip with hit count $hitcnt at " . `date`;
# prevent further communication...
      system("route add -host $ip gw 127.0.0.1");
   }
  }
  #print "route add -host $ip gw 127.0.0.1\n";
}
close(LINES);
} # end main function

Highlights from the program
The comments are pretty self-explanatory. Just a note about the philosophy. I fear making a goof and locking myself out! So I was conservative and try to not do any blocking if the source IP matches one of my favored source IPs, or if the user matches one of my usual usernames like drjohn1. I use obscure userids and the hackers try the stupid stuff like root, admin, etc. So they may be dictionary attacking the password, but they certainly aren’t dictionary attacking the username!

I don’t mind wiping the slate clean of all created routes after sever reboot so I only plan to run this from the command line. To make it persistent until the next reboot you just run it from the root account like so (let’s say we put it in /usr/local/sbin):

$ nohup /usr/local/sbin/drjfail2ban > /var/log/drjfail2ban &

And it just sits there and runs, even after you log out.

Results
Since it hasn’t been running for long I can provide a partial log file as of this publication.

Blocking ip 103.80.117.74 with hit count 6 at Sun Oct  8 17:34:43 CEST 2017
SIOCADDRT: File exists
Blocking ip 89.176.96.45 with hit count 5 at Sun Oct  8 17:34:43 CEST 2017
SIOCADDRT: File exists
Blocking ip 31.162.51.206 with hit count 3 at Sun Oct  8 17:34:43 CEST 2017
SIOCADDRT: File exists
Blocking ip 218.95.142.218 with hit count 6 at Sun Oct  8 17:34:43 CEST 2017
SIOCADDRT: File exists
Blocking ip 202.168.8.54 with hit count 5 at Sun Oct  8 17:34:43 CEST 2017
SIOCADDRT: File exists
Blocking ip 13.94.29.182 with hit count 4 at Sun Oct  8 17:34:43 CEST 2017
SIOCADDRT: File exists
Blocking ip 40.71.185.73 with hit count 4 at Sun Oct  8 17:34:43 CEST 2017
SIOCADDRT: File exists
Blocking ip 77.72.85.100 with hit count 13 at Sun Oct  8 17:34:43 CEST 2017
SIOCADDRT: File exists
Blocking ip 201.180.104.63 with hit count 7 at Sun Oct  8 17:34:43 CEST 2017
SIOCADDRT: File exists
Blocking ip 121.14.27.58 with hit count 4 at Sun Oct  8 17:40:43 CEST 2017
Blocking ip 36.108.234.99 with hit count 6 at Sun Oct  8 17:47:13 CEST 2017
Blocking ip 185.165.29.69 with hit count 6 at Sun Oct  8 18:02:43 CEST 2017
Blocking ip 190.175.40.195 with hit count 6 at Sun Oct  8 19:05:43 CEST 2017
Blocking ip 139.199.167.21 with hit count 4 at Sun Oct  8 19:29:13 CEST 2017
Blocking ip 186.60.67.51 with hit count 5 at Sun Oct  8 20:49:14 CEST 2017

And what my route table looks like currently:

$ netstat ‐rn|grep 127.0.0.1

Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
2.177.217.155   127.0.0.1       255.255.255.255 UGH       0 0          0 lo
13.94.29.182    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
31.162.51.206   127.0.0.1       255.255.255.255 UGH       0 0          0 lo
36.108.234.99   127.0.0.1       255.255.255.255 UGH       0 0          0 lo
37.204.23.84    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
40.71.185.73    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
42.7.26.15      127.0.0.1       255.255.255.255 UGH       0 0          0 lo
46.6.60.240     127.0.0.1       255.255.255.255 UGH       0 0          0 lo
59.16.74.234    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
77.72.85.100    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
89.176.96.45    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
103.80.117.74   127.0.0.1       255.255.255.255 UGH       0 0          0 lo
109.205.136.10  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
113.195.145.13  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
118.32.27.85    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
121.14.27.58    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
139.199.167.21  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
162.213.39.235  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
176.50.95.41    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
176.209.89.99   127.0.0.1       255.255.255.255 UGH       0 0          0 lo
181.113.82.213  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
185.165.29.69   127.0.0.1       255.255.255.255 UGH       0 0          0 lo
185.165.29.197  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
185.165.29.198  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
185.190.58.181  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
186.57.12.131   127.0.0.1       255.255.255.255 UGH       0 0          0 lo
186.60.67.51    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
190.42.185.25   127.0.0.1       255.255.255.255 UGH       0 0          0 lo
190.175.40.195  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
193.201.224.232 127.0.0.1       255.255.255.255 UGH       0 0          0 lo
201.180.104.63  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
201.255.71.14   127.0.0.1       255.255.255.255 UGH       0 0          0 lo
202.100.182.250 127.0.0.1       255.255.255.255 UGH       0 0          0 lo
202.168.8.54    127.0.0.1       255.255.255.255 UGH       0 0          0 lo
203.190.163.125 127.0.0.1       255.255.255.255 UGH       0 0          0 lo
213.186.50.82   127.0.0.1       255.255.255.255 UGH       0 0          0 lo
218.95.142.218  127.0.0.1       255.255.255.255 UGH       0 0          0 lo
221.192.142.24  127.0.0.1       255.255.255.255 UGH       0 0          0 lo

Here’s a partial listing of the many failed logins, just to keep it real:

...
root     ssh:notty    190.175.40.195   Sun Oct  8 19:05 - 19:28  (00:23)
root     ssh:notty    190.175.40.195   Sun Oct  8 19:05 - 19:05  (00:00)
root     ssh:notty    190.175.40.195   Sun Oct  8 19:05 - 19:05  (00:00)
root     ssh:notty    190.175.40.195   Sun Oct  8 19:05 - 19:05  (00:00)
root     ssh:notty    190.175.40.195   Sun Oct  8 19:05 - 19:05  (00:00)
root     ssh:notty    190.175.40.195   Sun Oct  8 19:05 - 19:05  (00:00)
admin    ssh:notty    185.165.29.69    Sun Oct  8 18:02 - 19:05  (01:02)
admin    ssh:notty    185.165.29.69    Sun Oct  8 18:02 - 18:02  (00:00)
admin    ssh:notty    185.165.29.69    Sun Oct  8 18:02 - 18:02  (00:00)
admin    ssh:notty    185.165.29.69    Sun Oct  8 18:02 - 18:02  (00:00)
root     ssh:notty    185.165.29.69    Sun Oct  8 18:02 - 18:02  (00:00)
root     ssh:notty    185.165.29.69    Sun Oct  8 18:02 - 18:02  (00:00)
root     ssh:notty    185.165.29.69    Sun Oct  8 18:02 - 18:02  (00:00)
root     ssh:notty    36.108.234.99    Sun Oct  8 17:47 - 18:02  (00:15)
root     ssh:notty    36.108.234.99    Sun Oct  8 17:47 - 17:47  (00:00)
root     ssh:notty    36.108.234.99    Sun Oct  8 17:47 - 17:47  (00:00)
root     ssh:notty    36.108.234.99    Sun Oct  8 17:47 - 17:47  (00:00)
root     ssh:notty    36.108.234.99    Sun Oct  8 17:47 - 17:47  (00:00)
root     ssh:notty    36.108.234.99    Sun Oct  8 17:46 - 17:47  (00:00)
ubuntu   ssh:notty    121.14.27.58     Sun Oct  8 17:40 - 17:46  (00:06)
ubuntu   ssh:notty    121.14.27.58     Sun Oct  8 17:40 - 17:40  (00:00)
aaaaaaaa ssh:notty    121.14.27.58     Sun Oct  8 17:40 - 17:40  (00:00)
aaaaaaaa ssh:notty    121.14.27.58     Sun Oct  8 17:40 - 17:40  (00:00)
root     ssh:notty    206.71.63.4      Sun Oct  8 17:34 - 17:40  (00:06)
root     ssh:notty    206.71.63.4      Sun Oct  8 17:34 - 17:34  (00:00)
root     ssh:notty    89.176.96.45     Sun Oct  8 16:15 - 17:34  (01:19)
root     ssh:notty    89.176.96.45     Sun Oct  8 16:15 - 16:15  (00:00)
root     ssh:notty    89.176.96.45     Sun Oct  8 16:15 - 16:15  (00:00)
root     ssh:notty    89.176.96.45     Sun Oct  8 16:15 - 16:15  (00:00)
...

Before running drjfail2ban it was much more obnoxious, with the same IP hitting my server every second or so.

Conclusion
I found it easier to roll my own than battle someone else’s errors. It’s kind of fun for me to create these little scripts. I don’t care if anyone else uses them. I will refer to this post myself and probably re-use it elsewhere!

References and related
In an earlier time, I was singing the praises of fail2ban on CentOS.

Posted in Admin, Linux, Security | Tagged | Leave a comment

Eclipse pics: just me and my cheese grater

Intro
I was at home during the great eclipse of August 2017. I didn’t buy the special viewing glasses. I remembered the general advice that anything with holes in it would show off the shape of the moon covering the sun by its shadow.

Initially I tried paper with three holes from a hole punch. Result were OK but not that impressive.

I was idly pulling disk=hes out of the dishwasher when I came across our cheese grater. Lots and lots of holes all over. This was the thing!

Everyone I’ve shown these two loved the novelty.

The setup
I’m standing just inside the garage – nice smooth surface for shadows – by the doorway. I find a way to simultaneously hold my phone and the cheese grater, and (mostly) get sufficiently out of the way that the cheese grater’s shadow is cast unobstructed onto the garage floor. It was a little tricky.

I’ll present the full picture and then the shadow of the cheese grater blown up at two different times – 2:27 PM and 2:41 PM EST.

And, yes, the holes in the grater are actualklky round and the normal shadow would be full of round circles of light where the sun passes through the grater’s holes!

It was a hot day. I could feel the air cool down during the eclipse! And cicadas started their shrill cry.

2:41 PM pics

Click on any of these pictures to see it blown up.

2:41 PM eclipse shadow pic


And focussing on the cheese grater’s shadow:

Cheese grater shadow during the eclipse


2:27 PM pics
This time is probably closer to the greatest coverage of the moon over the sun.

Eclipse full picture


Eclipse cheese grater pic

Conclusion
A common cheese grater gives us a good idea of what the sun looked like during the eclipse, and creates something marvelous at the same time.

Posted in Popular Science | Leave a comment

Verifying a pkcs12 file with openssl

Intro
The easy way

How to examine a pkcs12 (pfx) file

$ openssl pkcs12 ‐info ‐in file_name.pfx
It will prompt you for the password a total of three times!

The hard way
I went through this whole exercise because I originally could not find the easy way!!!

Get the source for openssl.

Look for pkread.c. Mine is in /usr/local/src/openssl/openssl-1.1.0f/demos/pkcs12.

Compile it.

My first pass:

$ gcc ‐o pkread pkread.c

/tmp/cclhy4wr.o: In function `sk_X509_num':
pkread.c:(.text+0x14): undefined reference to `OPENSSL_sk_num'
/tmp/cclhy4wr.o: In function `sk_X509_value':
pkread.c:(.text+0x36): undefined reference to `OPENSSL_sk_value'
/tmp/cclhy4wr.o: In function `main':
pkread.c:(.text+0x93): undefined reference to `OPENSSL_init_crypto'
pkread.c:(.text+0xa2): undefined reference to `OPENSSL_init_crypto'
pkread.c:(.text+0x10a): undefined reference to `d2i_PKCS12_fp'
pkread.c:(.text+0x154): undefined reference to `ERR_print_errors_fp'
pkread.c:(.text+0x187): undefined reference to `PKCS12_parse'
pkread.c:(.text+0x1be): undefined reference to `ERR_print_errors_fp'
pkread.c:(.text+0x1d4): undefined reference to `PKCS12_free'
pkread.c:(.text+0x283): undefined reference to `PEM_write_PrivateKey'
pkread.c:(.text+0x2bd): undefined reference to `PEM_write_X509_AUX'
pkread.c:(.text+0x320): undefined reference to `PEM_write_X509_AUX'
collect2: ld returned 1 exit status

Corrected compile command
$ gcc ‐o pkread pkread.c ‐I/usr/local/include ‐L/usr/local/lib64 ‐lssl ‐lcrypto

Note that this works for me because I put my ssl and crypto libraries in that directory. Your installation may have put them elsewhere:

$ ll /usr/local/lib64/*.a

-rw-r--r--. 1 root root 4793162 Aug 16 15:30 /usr/local/lib64/libcrypto.a
-rw-r--r--. 1 root root  765862 Aug 16 15:30 /usr/local/lib64/libssl.a

References and related
My favorite openssl commands.

Posted in Linux, Security | Tagged , | Leave a comment

Your AWS Instance was scheduled for retirement? don’t panic

Intro
After nearly four years of continuously running my AWS instance I got this scary email:

What to do?

The details
Since I never dveloped much AWS expertise (never needed to since it just worked) I was afraid to do anything. That’s sort of why I had kept it running for three and a half years – the last time I had to stop it didn’t work out so well.

Some terms
It helps to review the terms.
image – that’s like the OS. It has a unique identifier. Mine is ami-03559b6a.
instance – that’s a particular image running on a particular virtual server, identified by unique number. Mine is i-1737a673.
retired image – the owner of the image decided to no longer make it available for new instances

What it all means
I run a retired image, so for instance I can’t right-click my instance and:

– launch another like this

What I did to keep my instance running
I didn’t! Before the retirement deadline I stopped my instance. That is a painful process because it takes hours in my case. The server becomes unavailable quickly enough, but the status is stuck in state shutting down for at least a couple hours. But, eventually, it does shut down.

Then, I start it again. That’s it!

When it starts, AWS puts it on different hardware, etc, so I guess literally it is a different instance now, running the same image. I re-associate my elastic IP, and all is good.

So when the “retirement” date came along, there was no outage of my instance as I had already stopped/started it and that was all that was needed.
Amazon’s documentation – as good as it is – isn’t that clear on this point, hence this blog posting…

Side preparations
In case I couldn’t restart my image I had taken snapshots of my EBS volumes, and prepared to run Amazon Linux, which looks pretty similar to CentOS which is what I run. But, boy, learning about VPC and routing was a pain. I had to set all that up and gain at least a rudimentary understanding of all that. None of that existed six years ago when I started out! It was much simpler back then.

What it looks like

To make things concrete, here is my view on the AWS admin portal of my instances.

Conclusion
Having your Amazon AWS instance retired is not as scary as it initially sounds. Basically, just stop and start it yourself and you’ll be fine.

Posted in CentOS, Hosting Service | Tagged | 2 Comments