Categories
Linux Raspberry Pi

Interpreting speech with a Raspberry Pi

Or the beginning or creating your own smart speaker

Intro

Imagine you could use a low-cost device to interpret speech without the aid of the big cloud services and their complexity and security and big-brotherly-ness. Well if you have a DIY mindset, you can!

I wanted to control the raspberry pi-based slideshow I have written about many times in the past with voice commands. The question became How could I do it and is it even possible at all? And would I need to master the complex apis provided by either Amazon or Google cloud services? Well, it turns out that it is possible to do passable speech to text without any external cloud provider; and I am very excited to share what I’ve learned so far.

Equipment

raspberry pi 4 (even my old RPI 3 seems to work)

USB microphone

Raspberry Pi OS

Skills

basic linux and python skills are required

vosk – your main tool

I’m going to cut to the chase and just tell you that the vosk api is how I got this all working, but not before I drove into several dead-ends.

Here are the vosk installation instructions, which do work on RPi:

Vosk Installation (alphacephei.com)

It will be helpful to install and test the examples:

git clone https://github.com/alphacep/vosk-api
cd vosk-api/python/example
python3 ./test_simple.py test.wav

On my RPi 4 it took 36 s the first time, and 6.6 s the second time to run this test.wav. So I got worried and fully expected it would be just too slow on these underpowed RPi systems.

But I forged ahead and looked for an example which could do real-time speech-to-text. They provide a microphone example. It requires some additional packages. But even after installing them it still produced a nasty segmentation fault. So I gave up on that. Then I noticed an ffmpeg-based example. Well, turns out I have lots of prior ffmpeg experience as I also post about live recording of audio with the raspberry pi.

It turns out their example was simply to use ffmpeg to interpret a file, but I didn’t know that to begin with. But I know my way around ffmpeg that I could use it for processing a lvie stream. So I made those changes, and voila. I’m glad to say I was dead wrong about the processing speed. On the RPi 4 it can keep up with its text-to-speech task in real time!

Basic program to examine your speech in real time

I developed the following python script based off one of the python examples from the api. I call it drjtst4.py, just to give it a name:

                    

#!/usr/bin/env python3

import subprocess
import re
from modules import aux_modules

from vosk import Model, KaldiRecognizer, SetLogLevel

SAMPLE_RATE = 16000

SetLogLevel(0)

model = Model(lang="en-us")
rec = KaldiRecognizer(model, SAMPLE_RATE)
start,start_a = 0,0
input_device = 'plughw:1,0'
phrase = ''
accumulating = False
# wake word hey photo is often confused with a photo by vosk...
wake_word_re = '^(hey|a) photo'

with subprocess.Popen(["ffmpeg","-loglevel", "quiet","-f","alsa","-i",
                            input_device,
                            "-ar", str(SAMPLE_RATE) , "-ac", "1", "-f", "s16le", "-"],
                            stdout=subprocess.PIPE) as process:

    while True:
        data = process.stdout.read(4000)
        if len(data) == 0:
            break
        if rec.AcceptWaveform(data):
            print('in first part')
            print(rec.Result())
            text = rec.PartialResult()
# text is a "string" which is basically a dict
            start,start_a,accumulating,phrase = aux_modules.process_text(wake_word_re,text,start,start_a,accumulating,phrase)
        else:
# this part always seems to be executed for whatever reason
            print('in else part')
            text = rec.PartialResult()
            start,start_a,accumulating,phrase = aux_modules.process_text(wake_word_re,text,start,start_a,accumulating,phrase)
            print(rec.PartialResult())

# we never seem to get here
    print(rec.FinalResult())
    print('In final part')
    text = rec.FinalResult()

I created a modules directory and in it a file called aux_modules.py. It look like this:

                    

import re,time,json

def process_text(wake_word_re,text_s,start,start_a,accumulating,phrase):
    max = 5.5 # seconds
    inactivity = 10 # seconds
    short_max = 1.5 # seconds
    elapsed = 0
    if time.time() - start_a < inactivity:
# Allow some time to elapse since we just took an action
        return start,start_a,accumulating,phrase
# convert text to real text. Real text is in 'partial'
    text_d = json.loads(text_s)
    text = ''
    if 'partial' in text_d:
        text = text_d['partial']
    if 'text' in text_d:
        text = text_d['text']
    if not text == '': phrase = text
    if re.search(wake_word_re,text):
        if not accumulating:
            start = time.time()
            accumulating = True
            print('Wake word detected. Now accumulating text.')
    l = len(re.split(r'\s',text))
    print('text, word ct',text,l)
    if accumulating:
        elapsed = time.time() - start
        print('Elapsed time:',elapsed)
        if l > 1:
           phrase = text
    if elapsed > max or (elapsed > short_max and l == 1):
# we're at a natural ending here...
        print('This is the total text',phrase)
# do some action
# reset everything
        accumulating = False
        phrase = ''
        start_a = time.time()
    return start,start_a,accumulating,phrase

And you just invoke it with python3 drjtst4.py.

Sample session output
in else part
text, word ct 1
{
"partial" : ""
}
in else part
text, word ct hey 1
{
"partial" : "hey"
}
in else part
text, word ct hey 1
{
"partial" : "hey"
}
in else part
text, word ct hey 1
{
"partial" : "hey"
}
in else part
Wake word detected. Now accumulating text.
text, word ct hey photo 2
Elapsed time: 0.0004639625549316406
{
"partial" : "hey photo"
}
in else part
text, word ct hey photo 2
Elapsed time: 0.003415822982788086
{
"partial" : "hey photo"
}
in else part
text, word ct hey photo 2
Elapsed time: 0.034906625747680664
{
"partial" : "hey photo"
}
in else part
text, word ct hey photo 2
Elapsed time: 0.09063172340393066
{
"partial" : "hey photo"
}
in else part
text, word ct hey photo 2
Elapsed time: 0.2488384246826172
{
"partial" : "hey photo"
}
in else part
text, word ct hey photo 2
Elapsed time: 0.33771753311157227
{
"partial" : "hey photo"
}
in else part
text, word ct hey photo place 3
Elapsed time: 0.7102789878845215
{
"partial" : "hey photo place"
}
in else part
text, word ct hey photo place 3
Elapsed time: 0.7134637832641602
{
"partial" : "hey photo place"
}
in else part
text, word ct hey photo player 3
Elapsed time: 0.8728365898132324
{
"partial" : "hey photo player"
}
in else part
text, word ct hey photo player 3
Elapsed time: 0.8759913444519043
{
"partial" : "hey photo player"
}
in else part
text, word ct hey photo play slideshow 4
Elapsed time: 1.0684640407562256
{
"partial" : "hey photo play slideshow"
}
in else part
text, word ct hey photo play slideshow 4
Elapsed time: 1.0879075527191162
{
"partial" : "hey photo play slideshow"
}
in else part
text, word ct hey photo play slideshow 4
Elapsed time: 1.3674390316009521
{
"partial" : "hey photo play slideshow"
}
in else part
text, word ct hey photo play slideshow 4
Elapsed time: 1.3706269264221191
{
"partial" : "hey photo play slideshow"
}
in else part
text, word ct hey photo play slideshow 4
Elapsed time: 1.5532972812652588
{
"partial" : "hey photo play slideshow"
}
in else part
text, word ct hey photo play slideshow 4
Elapsed time: 1.5963218212127686
{
"partial" : "hey photo play slideshow"
}
in else part
text, word ct hey photo play slideshow 4
Elapsed time: 1.74298095703125
{
"partial" : "hey photo play slideshow"
}
in else part
text, word ct hey photo play slideshow 4
Elapsed time: 1.842745065689087
{
"partial" : "hey photo play slideshow"
}
in else part
text, word ct hey photo play slideshow 4
Elapsed time: 1.9888567924499512
{
"partial" : "hey photo play slideshow"
}
in else part
text, word ct hey photo play slideshow 4
Elapsed time: 2.0897343158721924
{
"partial" : "hey photo play slideshow"
}
in first part
{
"text" : "hey photo play slideshow"
}
text, word ct 1
Elapsed time: 2.3853299617767334
This is the total text hey photo play slideshow
in else part
{
"partial" : ""
}
in else part
{
"partial" : ""
}
A word on accuracy

It isn’t Alexa or Google. No one expected it would be, right? But if you’re a native English speaker it isn’t too bad. You can see it trying to correct itself.

The desire to choose an uncommon wake word of three syllables is at direct odds with how neural networks are trained! So… although I desired my wake word to be “hey photo,” I also allow “a photo.” A photo was probably in their training set whereas Hey photo certainly was not. Hence the bias against recognizing a unique wake word. And no way will I re-train their model – way too much effort. But to lower false positives this phrase has to occur at the beginning of a spoken phrase.

Turning this into a smart speaker

You can see I’ve got all the pieces set up. At least I think I do! I’ve got my wake word. I don’t have natural language processing but I think I can forgo that. I have a place in the code where I print out the “final text.” That’s where the spoken command is perceived to have been uttered and and a potential action could be exectured at that point.

Dead ends

To be fleshed out later as time permits.

Conclusion

I have demonstrated how speech-to-text without use of complex cloud apis such as those provided by Amazon and Google can be easily achieved on an inexpensive raspberry pi.

I will be building on this facility in subsequent posts as I turn my RPi-powered slideshow into a slideshow which reacts to voice commands!

References and related

Vosk Installation (alphacephei.com)

Raspberry Pi slideshow

This conference USB mic works really well for me.

Categories
Linux

libavddvice.so.58 not found running ffprobe

Intro

I use ffmpeg regularly on my RPi 4. I’ve written several articles. For no apparent reason, today ffprobe, which I use to get the length or a recording, stopped working for no apparent reason. The error was something like

libavddvice.so.58 not found

I hadn’t done any upgrades or anything at all, I think.

The solution

$ sudo apt-get install libavdevice-dev

Even though it looked like it was already present as shown by a dpkg -l, running this command installed a bunch of related libraries and seemed to fix things up just fine.

ffmpeg version 4.3.4-0

Raspbian 11 bullseye

References and relate

https://drjohnstechtalk.com/blog/2021/10/from-audio-recording-to-youtube-with-two-button-clicks-and-a-raspberry-pi/

Categories
Admin Linux Network Technologies Web Site Technologies

The IT Detective Agency: This site can’t be reached

Intro

It’s been awhile since I’ve had the opportunity to relatean IT mystery. After awhile they are repates of what’s already happened in the past, or it’s too complex to relate, or I was only peripherally involved. But today I came across a good one. It falls into the never been seen before category.

The details

A web server behind my web application firewall became unreachable. In the browser they get a message This site can’t be reached. The app owners came to me looking for input. I checked the WAF and it was fine. The virtual server was looking healthy. So I took a packet trace, something to this effect:

$ tcpdump -nni 0.0 host 192.168.2.124

14:00:45.180349 IP 192.68.1.13.42045 > 192.68.2.124.443: Flags [S], seq 1106553901, win 23360, options [mss 1460,sackOK,TS val 3715803515 ecr 0], length 0 out slot1/tmm3 lis=/Common/was90extqa.drjohn.com.app/was90extqa.drjohn.com_vs port=0.53 trunk=
14:00:45.181081 IP 192.68.2.124 > 192.68.1.13: ICMP host 192.68.2.124 unreachable - admin prohibited filter, length 64 in slot1/tmm2 lis= port=0.47 trunk=
14:00:45.181239 IP 192.68.1.13.42045 > 192.68.2.124.443: Flags [R.], seq 1106553902, ack 0, win 0, length 0 out slot1/tmm3 lis=/Common/was90extqa.drjohn.com.app/was9
0extqa.drjohn.com port=0.53 trunk=

I’ve never seen that before, ICMP host 192.68.2.124 unreachable – admin prohibited filter. But I know ICMP can be used to relay out-of-band routing information on occasion, though I do not see it often. I suspect it is a BAD THING and forces the connection to be shut down. Question is, where was it coming from?

The communication is via a firewall so I check the firewall. I see a little more traffic so I narrow the filter down:

$ tcpdump -nni 0.0 host 192.168.2.124 host 443

And then I only see the initial SYN packet followed by the RST – from the same source IP! So since I didn’t see the bad ICMP packet on the firewall, but I do see it on the WAF, I preliminarily conclude the problem exists on the WAF.

Rookie mistake! Did you fall for it? So very, very often, in the heat of debugging, we invent some unit test which we’ve never done before, and we have to be satisified with the uncertainty in the testing method and hope to find a control test somehow, somewhere to validate our new unit test.

Although I very commonly do compound filters, in this case it makes no sense, as I realized a few minutes later. My port 443 filter would of course exclude logging the bad ICMP packets because ICMP does not use tcp port 443! So I took that out and re-run it. Yup. bad ICMP packet still present on the firewall, even on the interface of the firewall directly connected to the server.

So at this point I have proven to my satisfaction that this packet, which is ruining the communication, really comes frmo the server.

What the server guys say

Server support is outsourced. The vendor replies

As far as the patching activities go , there is nothing changed to the server except distro upgrading from 15.2 to 15.3. no other configs were changed. This is a regular procedure executed on almost all 15.2 servers in your environment. No other complains received so far…

So, the usual It’s not us, look somewhere else. So the app owner asks me for further guidance. I find it’s helpful to create a test that will convince the other party of the error with their service. And what is one test I would have liked to have seen but didn’t cnoduct? A packet trace on the server itself. So I write

I would suggest they (or you) do a packet trace on the server itself to prove to themselves that this server is not behaving ini an acceptable way, network-wise, if they see that same ICMP packet which I see.

The resolution

This kind of thing can often come to a stand-off, or many days can be wasted as an issue gets escalated to sufficiently competent technicians. In this case it wasn’t so bad. A few hours later the app owners write and mention that the home-grown local firewall seemed suspect to them. They dsabled it and this traffic began to work.

They are reaching out to the vendor to understand what may have happened.

Case: closed!

Conclusion

An IT mystery was resolved today – something we’ve never seen but were able to diagnose and overcome. We learned it’s sometimes a good thing to throw a wider net when seeing unexpected reset packets because maybe just maybe there is an ICMP host unreachable packet somewhere in the mix.

Most firewalls would just drop packets and you wait for a timeout. But this was a homegrown firewall running on SLES 15. So it abides by its own ways of working, I guess. So because of the RST, your connection closes quickly, not timing out as with a normal network firewall.

As always, one has to maintain an open mind as to the true source of an issue. What was working yesterday does not today. No one admits to changing anything. Finding clever ad hoc unit tests is the way forward, and don’t forget to validate the ad hoc test. We use curl a lot for these kinds of tests. A browser is a complex beast and too much of a black box.

Categories
Linux

Install WSL2 for Windows 10 Home Edition: not as easy as they say, but not impossible either, and definitely worth it

Intro

I installed WSL2 on my work laptop a couple weeks ago. It didn’t go terribly smoothly but now that I have it, I love it. I had been using a Cygwin environment, but I fear that is looking a little long in the tooth. WSL2 is fast to start up. But the main contrast is that while Cygwin is an emulator, WSL2 is a true hypervisor so you get a full-fledged linux VM, right on your PC. Of course this was always possible with products like VirtualBox or whatnot, but Microsoft has sort of built in this capability with newer versions of Windows 10, so there’s no mussing with external software any longer.

But at work I have Windows 10 Professional, of course. What about at home where I have Windows 10 Home Edition like most of us? My understanding is that you could not run a hypervisor with Windows 10 Home Edition. And I was probably right, until recently. But now you can. I know because I just managed it tonight.

None of the tutorials out there were exactly right, but they all contained pieces of the truth. So my contribution is to add weight to the correct steps you’ll need to take. Unfortunately I only get to do it once so my notes aren’t the best. Still, I may be able to spare you some pitfalls.

Why you should want WSL2

If you love linux command-line, then I would say this is a must-have.

What doesn’t work

You’ll see suggestions to fire up powershell and simply run

wsl –install

Chances are about 95% that that won’t work if you are reading this article – would that it would be so simple.

Instead, do this

Open a powershell window as administrator. To do that type powershell in the start menu, and look around at all the options. Pick out the one that mentions Run as administrator.

Running Powershell as Administrator

Then enter this command into the PS window.

dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart

Then this.

dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart

Then this.

wsl --set-default-version 2

You need to update your kernel. Download this WSL2 kernel update file and install it: https://wslstorestorage.blob.core.windows.net/wslblob/wsl_update_x64.msi

A reboot at this point is probably a good idea.

Now you need to get yourself a linux distro to install.

There are certain wsl commands you can issue which will helpfully give you the URL to the Debian distro: You put the URL into the browser and it redirects you to the MS Store. But I forget what that is. perhaps wsl -d Debian. But I suppose you can simply go to the MS Store directly and search for Debian and install it.

Enhance your experience with Windows Terminal

One good suggestion out of Windows Central is to use Windows Terminal. At least it looks good. I haven’t had time to try it myself. I normally just fire up a CMD window and type wsl. My Debian starts immediately and I have a satisfactory command-line environment. But working with multiple windows will be nice so I have to check it out.

Just look for Windows Terminal in the MS Store.

Windows Central suggestions

A web site called Windows Central has a pretty good stepwise guide. But their advertising is so obnoxious, I’m afraid to accidentally touch any part of the page for fear of getting sent to one of their many advertisers. Even still it probably happened about five times. So I won’t make the link to them too prominent. And, anyway, their guide is a little oversimplified.

My equipment

I have a four-year old HP Pavilion laptop running Windows 10 2021 H2 if I remember correctly. It has solid state drives so it’s not too slow, and it boots pretty quickly.

BIOS – basically impossible to get into these days

I’m sure people who do this for a living will disagree, but for ordinary people it’s basically impossible to disrupt the boot process to modify the BIOS settings. And you may need to do that. In fact that was the hardest thing of all for me. Pressing F10 or delete or Escape or F2 – and does that mean hold the FN key down first?? No one explains that, and I don’t have patience to watch a YouTube video. But after trying a bunch of combinations and booting a bunch of times, and never getting into the BIOS settings, I was really glad to learn Windows 10 offers an alternate way! And it works…

Access BIOS settings from Windows 10

Very briefly, the steps are:

Windows Settings > Update & Security > Recovery > Restart Now > Advanced Startup -> Restart Now > Reboot > select Troubleshoot > Advanced Options > UEFI Firmware Settings > (BIOS menu) enable virtualization > Save.

To see the details, go to this HP article: How to Enter BIOS Setup on Windows PCs | HP® Tech Takes

Why you may need to alter the BIOS settings

Well, on my laptop my installation of Debian kept failing with this error. Error: 0x80370102 The virtual machine could not be started because a required feature is not installed. I read on a Microsoft site that could be because the ability to run virtual servers was not enabled in the BIOS. And, yes, that turned out to be absolutely true. It was disabled. So I enabled it and bam, the Debian install started asking me for a username and password, and I was running a Debian VM!

To be fleshed out as my time permits…

But, I love my Debian linux. It’s just like Raspberry Pi OS Lite. I just install packages as I need them: python, pip, curl, bind9-dnsutils, ssh, etc.

Operating inbound TCP services

After the initial thrill wears off, you realize you may need practical things that you have on your Raspberry Pi such as an ssh server or a web server. I believe this will be possible. Still working on it. After installing ssh you can fire it up:

$ sudo service ssh start

This post describes some of those service commands which you have under a WSL linux install: [3 Fixes] System Has Not Been Booted With Systemd as Init System (partitionwizard.com)

If you ignore that article you may see this error! System has not been booted with systemd as init system (PID 1). Can’t operate.

Back to your ssh server. Now you can already connect to it from the Windows system itself, e.g., from a CMD window:

C:\Users\me> ssh user@localhost

user is the Debian user you set during initial setup. So, anyway, that works and that’s cool. But you’re still locked out from the outside.

This helpful Microsoft article discusses networking for WSL2. Apparently it is still evolving and so it’s a bit primitive right now: Accessing network applications with WSL | Microsoft Docs

From a CMD Window launched as administrator:

netsh interface portproxy add v4tov4 listenport=22 listenaddress=0.0.0.0 connectport=22 connectaddress=172.22.167.12

But this does not work in my case. Firewall thing, I’m sure. Yes! for me, where I also run Mcafee, I needed to go to their firewall settings > Ports and system services. Then I had to add a service for TCP port 22 – the ssh default port. Then it began to work and my RPi could ssh and sftp to my Debian VM! sftp kind of hanged a bit. Have to see how bad that is.

References and related

That obnoxious Windows Central article I mentioned above with a lot of the WSL2 installation information. It’s a veritable minefield of links to irrelevant stuff, so you’ve been warned: How to install Linux WSL2 on Windows 10 and Windows 11 | Windows Central

WSL2 kernel update.

Seeing Error: 0x80370102? Try Troubleshooting Windows Subsystem for Linux | Microsoft Docs A whole host of other WSL2 errors are addressed in this article as well.

This article purports to be for servers, but I think it’s applicable to PCs as well. It gets pretty technical. System requirements for Hyper-V on Windows Server | Microsoft Docs

About starting system services such as the ssh daemon: [3 Fixes] System Has Not Been Booted With Systemd as Init System (partitionwizard.com)

A good overview of WSL2 networking: Accessing network applications with WSL | Microsoft Docs

How to Enter BIOS Setup on Windows PCs | HP® Tech Takes

Categories
Admin Linux Raspberry Pi

Scripts checker

Intro

Imagine an infrastructure team empowered to create its own scripts to do such things as regularly update external dynamic lists (EDLs) or interact with APIs in an automated fashion. At some point they will want to have a meta script in place to check the output of the all the automation scripts. This is something I developed to meet that need.

I am getting tired of perl, and I still don’t know python, so I decided to enhance my bash scripting for this script. I learned some valuable things along the way.

checklogs.sh

I call the script checklogs.sh Here it is.

                    

#!/bin/bash
# DrJ 12/2021
# it is desired to run this using the logrotate mechanism
#
# logrotate invokes with /bin/sh so we have to do this trick...
if [ ! "$BASH_VERSION" ] ; then
  exec /bin/bash "$0" "$@"
  exit
fi
DIR=$(cd $(dirname $0);pwd)
INI=$DIR/log.ini
DAY=2 # Day of week to analyze full week of logs. Monday is 1, Tuesday 2, etc
DEBUG=0
maxdiff=10
maxerrors=10
minstarts=10
TMPDIR=/var/tmp
cd $TMPDIR
recipients="john@drjetc.com"
#
checklog2() {
  [[ "$DEBUG" -eq "1" ]] && echo ID, $ID, LPATH, $LPATH, START, $START, ERROR, $ERROR, END, $END
  LPATH="${LPATH}${wildcard}"
  zgrep -Ec "$START" ${LPATH}|cut -d: -f2|while read sline; do starts=$((starts + sline));echo $starts>starts; done
  zgrep -Ec "$END" ${LPATH}|cut -d: -f2|while read sline; do ends=$((ends + sline));echo $ends>ends; done
  zgrep -Ec "$ERROR" ${LPATH}|cut -d: -f2|while read sline; do errors=$((errors + sline));echo $errors>errors; done
  exampleerrors=$(zgrep -E "$ERROR" ${LPATH}|head -10)
  starts=$(cat starts)
  ends=$(cat ends)
  errors=$(cat errors)
  info="${info}===========================================
$ID SUMMARY
  Total starts: $starts
  Total finishes: $ends
  Total errors: $errors
  Most recent errors: "
  info="${info}${exampleerrors}
==========================
"
  unset NEW
# get cumulative totals
  starttot=$((starttot + starts))
  endtot=$((endtot + ends))
  errortot=$((errortot + errors))
  [[ "$DEBUG" -eq "1" ]] && echo starttot, $starttot, endtot, $endtot, errortot, $errortot
  [[ "$DEBUG" -eq "1" ]] || rm starts ends errors
} # end of checklog2 function

checklog() {
# clear out stats and some variables
starttot=0;endtot=0;errortot=0;info=""
#this IFS and following line is trick to preserve those darn backslash charactes in the input file
IFS=$'\n'
for line in $(<$INI); do
  [[ "$line" =~ ^# ]] || {
  pval=$(echo "$line"|sed s'/: */:/')
  lhs=$(echo $pval|cut -d: -f1)
  rhs=$(echo "$pval"|cut -d: -f2-)
  lhs=$(echo $lhs|tr [:upper:] [:lower:])
  [[ "$DEBUG" -eq "1" ]] && echo line is "$line", pval is $pval, lhs is $lhs, rhs is "$rhs"
  if [ "$lhs" = "identifier" ]; then
    [[ "$DEBUG" -eq "1" ]] && echo matched lhs = identifer section
    [[ -n "$NEW" ]] && checklog2
    ID="$rhs"
  fi
  [[ "$lhs" = "path" ]] && LPATH="$rhs" && NEW=false
  [[ "$lhs" = "error" ]] && ERROR="$rhs"
  [[ "$lhs" = "start" ]] && START="$rhs"
  [[ "$lhs" = "end" ]] && END="$rhs"
  }
done
# call one last time at the end
checklog2
} # end of checklog function

anomalydetection() {
# a few tests - you can always come up with more...
  diff=$((starttot - endtot))
  [[ $diff -gt $maxdiff ]] || [[ $starttot -lt $minstarts ]] || [[ $errortot -gt $maxerrors ]] && {
    ANOMALIES=1
    [[ "$DEBUG" -eq "1" ]] && echo ANOMALIES, $ANOMALIES, starttot, $starttot, endtot, $endtot, errortot, $errortot
  }
} # end function anomalydetection

sendsummary() {
  subject="Weekly summary of automation scripts - please review"
  [[ -n "$ANOMALIES" ]] && subject="${subject} - ANOMALIES DETECTED PLEASE REVIEW CAREFULLY!!"

  intro="This summarizes the results from the past week of running automation scripts on script server.
Please check that values seem reasonable. If things are out of range, look at the script server.

"

  [[ "$DEBUG" -eq "1" ]] && echo subject, $subject, intro, "$intro", info, "$info"
  [[ "$DEBUG" -eq "1" ]] && args="-v"
  echo "${intro}${info}"|mail "$args" -s "$subject" $recipients
} # end function sendsummary

# MAIN PROGRAM
# always check the latest log
checklog
anomalydetection

# only check all logs if it is certain day of the week. Monday = 1, etc
day=$(date +%u)
[[ "$DEBUG" -eq "1" ]] && echo day, $day
[[ $day -eq $DAY ]] || [[ -n "$ANOMALIES" ]] && {
  [[ "$DEBUG" -eq "1" ]] && echo calling checklog with wildcard set
  wildcard='*'
  checklog
  sendsummary
}

[[ "$DEBUG" -eq "1" ]] && echo message so far is "$info"

log.ini
                    

# The suggestion: To have a configuration file with log identifiers
#(e.g. “anydesk-edl”) and per identifier: log file path (“/var/log/anydesk-edl.log”),
# error pattern (“.+\[Error\].+”), start pattern (“.+\[Notice\] Starting$”) end pattern (“.+\[Notice\] Done$”).
#Then just count number of executions (based on start/end) and number of errors.

# the start/end/error values are interpreted as extended regular expressions - see regex(7) man page
identifier: anydesk-edl
path: /var/log/anydesk-edl.log
error: .+\[Error\].+
start: .+\[Notice\] Starting$
end: .+\[Notice\] Done$

identifier: firewall-requester-to-edl
path: /var/log/firewall-requester-to-edl.log
error: .+\[Error\].+
start: .+\[Notice\] Starting$
end: .+\[Notice\] Done$

identifier: sase-ips-to-bigip
path: /var/log/sase-ips-to-bigip.log
error: .+\[Error\].+
start: .+\[Notice\] Starting$
end: .+\[Notice\] Done$

What this script does

So when the guy writes an automation script, he is so meticulous that he follows the same convention and hooks it into the syslogger to create uniquely named log files for it. He writes out a [Notice] Starting when his script starts, and a [Notice] Done when it ends. And errors are reported with an [Error] details. Some of the scripts are called hourly. So we agreed to have a script that checks all the other scripts once a week and send a summary email of the results. I look to see that the count of starts and ends is roughly the same, and I report back the ten most recent errors from a given script. I also look for other basic things. That's the purpose of the function anomalydetection in my script. It's just basic tests. I didn't want to go wild.

But what if there was a problem with one of the scripts, wouldn't we want to know sooner than possibly six days later? So I decided to have my script run every day, but only send email on the off days if an anomaly was detected. This made the logic a tad more complex, but nothing bash and I couldn't handle. It fits the need of an overworked operational staff.

Techniques I learned and re-learned from developing this script

cron scheduling - more to it than you thought

I used to naively think that it suffices to look into the crontab files of all users to discover all the scheduled processes. What I missed is thinking about how log rotate works. How does it work? Turns out there is another section of cron for jobs run daily, weekly and monthly. logrotate is called from cron.daily.

logrotate - potential to do more

The person who wrote the automation scripts is a much better scripter than I am. I didn't want to disappoint so I put in the extra effort to discover the best way to call my script. I reasoned that logrotate would offer the opportunity to run side scripts, and I was absolutely right about that! You can run a script just before the logs rotate, or just after. I chose the just before timing - prerotate. In actual fact logrotate calls the prerotate script with all the log files to be rotate as arguments, which you notice we don't take advantage of, because at the time we were unsure how we were going to interface. But I figure let's just leave it now. man logrotate to learn more.

By the way although I developed on a generic Debian system, it should work on a Raspberry Pi as well since it is Debian based.

BASH - the potential to do more, at a price

You'll note that I use some bash-specific extensions in my script. I figure bash is near universal, so why not? The downside is that when logrotate invokes an external script, it calls is using old-fashioned shell. And my script does not work. Except I learned this useful trick:

if [ ! "$BASH_VERSION" ]; then
  exec /bin/bash "$0" "$@"
  exit
fi

Note this is legit syntax in SHELL and a legit conditional operator expression. So it means if you - and by you I mean the script talking about itself - are invoked via SHELL, then invoke yourself via BASH and exit the parent afterwards. And this actually does work (To do: have to check which occurs first, the syntax checking or the command invocation).

More

Speaking of that conditional, if you want to know all the major comparison tests, do a man test. I have around to use the double bracket expressions [[ more and more, though they are BASH specific I believe. The double bracket can be followed by a && and then an open curly brace { which can introduce a block of code delimited of course by a close curly brace }. So for me this is an attractive alternative to SHELL's if conditional then code block fi syntax, and probably just slightly more compact. Replace && with || to execute the code block when the condition does not evaluate to be true.

zgrep is grep for compressed files, but we knew that right? But it's agnostic - it works like grep on both compressed and uncompressed files. That's important because with rotated logs you usually have a combination of both.

Now the expert suggested a certain regular expression for the search string. It wasn't working in my first pass. I reasoned that zgrep may have a special mode to act more like egrep which supports extended regular expressions (EREs). EREs aren't really the same as perl-compatible regular expressions (PCREs) but for this kind of simple stuff we want, they're close enough. And sure enough zgrep has the -E option to force it to interpret the expression as an ERE. Great.

RegEx

So in the log.ini file the regular expression has a \[...\] syntax. The backslash is actually required because otherwise the [...] syntax is interpreted as a character class, where all the characters between the brackets get tried to match a single character in the string to be matched. That's a very different match!

My big thing was - will I have to further escape those lines read in from log.ini, perhaps to replace a \[ with a \\[? Stuff like that happens. I found as long as I used those double quotes around the variables (see below) I did not need to further escape them. Similarly, I found that the EREs in log.ini did not need to be placed between quotes though the guy initially proposed that. It looks cleaner without them.

Variable scope

I wasted a lot of time on a problem which I thought may be due to some weird variable scoping. I've memorized this syntax cat file|while read line; do etc, etc so I use it a lot in my tiny scripts. It's amazing I got away with it as much as I have because it has one huge flaw. if you start using variables within the loop you can't really suck them out, unless you write them to a file. So while at first I thought it was a problem of variable scoping - why do my loop variables have no values when the code comes out of the loop? - it really isn't that issue. It's that the pipe, |, created a forked process which has its own variables. So to avoid that I switched to this weird syntax for line in $(<$INI); do etc. So it does the line-by-line file reading as before but without the pipe and hence without the "variable scope" problem.

But in another place in the script - where I add up numbers - I felt I could not avoid the pipe. So there I do write the value to a file.

The conclusion is that with the caveat that if you know what you're doing, all variables have global scope, and that's just as it should be. Hey, I'm from the old Fortran 66/77 school where we were writing Monte Carlos with thousands of lines of code and dozens of variables in a COMMON block (global scope), and dozens of contributors. It worked just fine thank you very much. Because we knew what we were doing.

Adding numbers in bash

Speaking of adding, I can never remember how to add numbers (integers). In bash you can do starts=$((starts + sline)) , where starts and sline are integers. At least this worked in Debian linux Stretch. I did not really get the same to work so well in SLES Linux - at least not inside a loop where I most needed it.

When you look up how to add numbers in bash there are about a zillion different ways to do it. I'm trying to stick to the built-in way.

Sending mail in Debian linux

You probably need to configure a smarthost if you haven't used your server to send emails up until now. You have to reconfigure of the exim4 package:

dpkg-reconfigure exim4-config

This also can be done on a RPi if you ever find you need for it to send out emails.

Variables

If a variables includes linebreaks and you want to see that, put it between double-quotes, e.g., echo "$myVariableWithLineBreaks". If you don't do that it seems to remove the linebreaks. Use of the double quotes also seems to help avoid mangling variables that contain meta characters found in regular expressions such as .+ or \[.

Result of executing the commands

I grew up using the backtick metacharacter, `, to indicate that the enclosed command should be executed. E.g., old way:

DIR=`dirname $0`

But when you think about it, that metacharacter is small, and often you are unlucky and it sits right alongside a double quote or a single quote, making for a visual trainwreck. So this year I've come to love the use of $(command to be executed) syntax instead. It offers much improved readability. But then the question became, could I nest a command within a command, e.g., for my DIR assignment? I tried it. Now this kind of runs counter to my philosophy of being able to examine every single step as it executes because now I'm executing two steps at once, but since it's pretty straightforward, I went for it. And it does work. Hence the DIR variable is assigned with the compound command:

DIR=$(cd $(dirname $0);pwd)

So now I wonder if you can go more than two levels deep? Each level is an incrementally bad idea - just begging for undetectable mistakes, so I didn't experiment with that!

By the way the reason I needed to do that is that the script jumps around to another directory to create temporary files, and I wanted it to be able to reference the full path to its original directory, so a simpler DIR=$(dirname $0) wasn't going to cut it if it's called with a relative path such as ./checklogs.sh

Debugging

I make mistakes left and right. But I know what results I expect. So I generously insert statements as variables get assigned to double check them, prefacing them with a conditional [[ $DEBUG -eq 1 ]] && print out these values. As I develop DEBUG is set to 1. When it's finally working, I usually set it to 0, though in some script I never quite reach that point. It looks like a lot of typing, but it's really just cut and paste and not over-thinking it for the variable dump, so it's very quick to type.

Another thing I do when I'm stuck is to watch as the script executes in great detail by appending -xv to the first line, e.g., #!/bin/bash -xv. But the output is always confusing. Sometimes it helps though.

Techniques I'd like to use in the future

You can assign a function to a variable and then call that variable. I know that will have lots of uses but I'm not used to the construct. So maybe for my next program.

Conclusion

This fairly simple yet still powerful script has forced me to become a better BASH shell scripter. In this post I review some of the basics that make for successful scripting using the BASH shell. I feel the time invested will pay off as there are many opportunities to write such utility scripts. I actually prefer bash to perl or python for these tasks as it is conceptually simpler, less ambitious, less pretentious, yes, far less capable, but adequate for my tasks. A few rules of the road and you're off and running! bash lends itself to very quick testing cycles. Different versions of bash introduced additional features, and that gets trying. I hope I have found and utilized some of the basic stuff that will be available on just about any bash implementation you are likely to run across.

References and related

The nitty gritty details about BASH shell can be gleaned by doing a man bash. It seems daunting at first but it's really not too bad once you learn how to skim through it.

Categories
Admin Firewall Linux

Linux: how to estimate bandwidth usage to a particular subnet

Intro

Let’s say someone asks you to estimate the total bandwith used by a particular subnet, or a particular service such as https on port 443. I provide a crude way to do that using tcpdump on a not-too-busy server.

The code

I call it bandwidth.sh. By the way, I ran it on a Checkpoint Gaia appliance so it works there as well.

                    

#!/bin/bash
# DrJ 11/21
sleep=120
file=/tmp/ctpackets
sum() {
sum=0
cat $file|while read line; do
 length=$(echo $line|awk '{print $17}'|sed 's/)//')
 sum=$(expr $sum + $length)
 echo $sum
done
}
while /bin/true; do
tcpdump -c1000 -v -nni eth1 net 216.71/16 > $file
#10:29:49.471455 IP (tos 0x0, ttl 126, id 32399, offset 0, flags [none], proto: UDP (17), length: 105) 10.32.25.126.3391 > 216.71.170.32.61445: UDP, length 77
total=$(sum|tail -1)
t0=$(head -1 $file|awk '{print $1}')
t1=$(tail -1 $file|awk '{print $1}')
h0=$(echo $t0|cut -d: -f1|sed 's/^0//')
h1=$(echo $t1|cut -d: -f1|sed 's/^0//')
m0=$(echo $t0|cut -d: -f2|sed 's/^0//')
m1=$(echo $t1|cut -d: -f2|sed 's/^0//')
s0=$(echo $t0|cut -d: -f3|sed 's/^0//')
s1=$(echo $t1|cut -d: -f3|sed 's/^0//')
s0=$(echo $s0|cut -d\. -f1|sed 's/^0//')
s1=$(echo $s1|cut -d\. -f1|sed 's/^0//')
[ -z "$h0" ] && h0=0
[ -z "$h1" ] && h1=0
[ -z "$m0" ] && m0=0
[ -z "$m1" ] && m1=0
[ -z "$s0" ] && s0=0
[ -z "$s1" ] && s1=0
t0secs=$((3600*$h0+60*$m0+$s0))
t1secs=$((3600*$h1+60*$m1+$s1))
#echo total bytes: $total
elapsed=$(($t1secs-$t0secs))
#echo elapsed time: $elapsed
kbps=$(($total*8/$elapsed/1000))
echo $kbps kbps at $(date)
sleep $sleep
done

The idea

Running tcpdump with the -v switch gives us packet length. We find that length and sum it up. Here we used a filter epxression of 216.71/16 to capture only the traffic from that subnet.

The number of packets to capture has to be tuned to how busy it gets. Now it’s set to only capture 1000 packets. And you see my crude timings are truncated at the second. So 1000 packets in one second or about 1.5 MBytes/sec = 12 Mbps is the maximum sensitivy of this approach. I doub it will really work for interfaces with more thn 100 Mbps, even after you scaled up the count (and don’t forget to change the denominator in the kbps line!

Here’s a sample output:

1000 packets captured
2002 packets received by filter
0 packets dropped by kernel
5 kbps at Wed Nov 3 12:09:45 EDT 2021

I think it’s important to note the number of packets dropped by the kernel. So if it gets too busy as I underatdn it, it will at least try to tell yuo that it couldn’t capture all the data and at that point you can no longer trust this method. Perhaps with enhanced statistical methods it could be salvaged.

I don’t run it continuously to also give the kernel a breather. It probably doesn’t make much difference, but every two minutes seems plenty frequent to me…

Conclusion

We have demonstrated a crude but better-than-nothing script to calculate bandwidth for a given tcpdump filter expression. It won’t win any awards, but it contains some worthwhile ideas. And it seems to work at low bandwidth levels.

Categories
Internet Mail Linux Raspberry Pi

From Audio Recording to YouTube with two button clicks and a Raspberry Pi

Intro

This post builds on the success of previous posts and uses elements from them. I don’t honestly expect anyone to repeat all the ingredients I have assembled here. But I have created them in a fairly modular way so you can pick out those elements which will help your project.

But, it is true, I have gotten the user experience of recording audio from, e.g., a band practice, down to a click of the ENTER button to start the recording, another click to stop it, and a click of the UP ARROW button to process the audio recording – turn it into a video – and upload it to YouTube, mark it as UNLISTED, and send the link to me in an email. Pretty cool if I say so myself. I am refining things as I write this to make it more reliable.

This write-up is not terribly detailed. It presumes at least a medium skill level with linux.

Ingredients
  • RPi 3 or RPi 4
  • Raspberry OS desktop running Pixel desktop environment
  • tiger VNC, i.e., the package tigervnc-scraping-server
  • chromium-browser (but it comes with)
  • xdotool (apt-get install xdotool)
  • xsel (apt-get install xsel)
  • YouTube account
  • crontab entries – see below
  • you do not need an HDMI display, except for the OS setup
  • a vnc viewer such as Real VNC
  • exim4 and bsd-mailx packages
The scripts

recordswitch.sh

                    

#!/bin/bash
# DrJ 8/2021
# Control the livestream of audio to youtube
# works in conjunction with an attached keyboard
# I use bash interpreter to give me access to RegEx matching
HOME=/home/pi
log=$HOME/audiocontrol.log
program=ffmpegwireless9.sh
##program=tst.sh # testing
PGM=$HOME/$program
# de-press ENTER button produces this:
matchE="1, 28, 0"
# up arrow
matchU="1, 103, 0"

epochsOld=0
cutoff=3 # seconds
DEBUG=1
ledtime=10
#
echo "$0 starting monitoring at "$(date)
# Note the use of script -q -c to avoid line buffering of the evread output
script  -q -c $HOME/evread.py /dev/null|while read line; do
[[ $DEBUG -eq 1 ]] && echo line is $line
# seconds since the epoch
epochs=$(date +%s)
elapsed=$((epochs-$epochsOld))
if [[ $elapsed -gt $cutoff ]]; then
  if [[ "$line" =~ $matchE ]]; then
# ENTER button section - recording
    echo "#################"
    echo We caught this input: $line at $(date)
# see if we are already running our recording program or not
    pgrep -f $program>/dev/null
# 0 means it's been found
    if [ $? -eq 0 ]; then
# kill it
      echo KILLING $program
      pkill -9 -f $program; pkill -9 arecord; pkill -9 ffmpeg
      pkill -9 -f blinkLED
      echo Shine the PWR LED
      $HOME/shineLED.sh
    else
# start it
      echo Blinking PWR LED
      $HOME/blinkLED.sh &
      echo STARTING $PGM
      $PGM > $PGM.log.$(date +%m-%d-%y:%H:%M) 2>&1 &
    fi
    epochsOld=$epochs
  elif [[ "$line" =~ $matchU ]]; then
# UP ARROW button section - processing
    echo "###########"
    echo processing commencing at $(date)
    $HOME/blinktwiceLED.sh &
    echo start processing of the recording
    $HOME/process.sh >> process.log 2>&1
    pkill -9 -f LED
    $HOME/shineLED.sh
    epochsOld=$epochs
  fi
[[ $DEBUG -eq 1 ]] && echo No action taken. Continue to listen
fi
done

ffmpegwireless9.sh

                    

#!/bin/sh
ffmpeg \
-thread_queue_size 4096 \
-f alsa -i plughw:1,0 \
-thread_queue_size 64 \
-f lavfi -i color=color=darkgray \
-c:v libx264 -pix_fmt yuv420p -g 18  -x264opts no-scenecut -b:v 50k \
-bufsize 512k \
-acodec libmp3lame -ar 44100 \
-threads 8 \
-b:a 128k \
-r 5 \
-s 480x320 \
-flush_packets 1 \
-f mp3 file:record-$(date +%m-%d-%y-%H-%M).mp3 \
< /dev/null

mp32flv.sh

                    

#!/bin/sh
# DrJ 10/2021
#
# Note that ffmpeg runs at ~ 4 x real-time when it is producing this flv video file
#
line=$1
time=$(ffprobe -v error -show_entries format=duration   -of default=noprint_wrappers=1:nokey=1 file:${line}|tail -1)
echo recording time: $time s
echo $time > duration
video=$(echo ${line}|sed 's/mp3/flv/')
  ffmpeg \
 -i file:${line} \
 -f lavfi -i color=color=darkgray \
 -c:v libx264 -pix_fmt yuv420p -g 18  -x264opts no-scenecut -b:v 50k \
 -bufsize 512k \
 -acodec libmp3lame -ar 44100 \
 -threads 8 \
 -b:a 128k \
 -r 5 \
 -s 480x320 \
 -t $time \
 -f flv file:${video} \
 < /dev/null

auto-upload.sh

                    

#!/bin/sh
# automate upload of YouTube videos
#
# define some functions
randomsleep(){
# sleep random amount between 1.5 to 2.5 seconds
t10=$(shuf -n1 -i 15-25)
t=$(echo $t10/10|bc -l)
sleep $t
}
drjtool(){
randomsleep
xdotool $1 $2 $3
randomsleep
}

echo Start video upload
echo set display to main display
export DISPLAY=:0
# launch chromium
echo launch chromium
chromium-browser --kiosk https://studio.youtube.com/ > /dev/null 2>&1 &
sleep 25
echo move to CREATE button
drjtool mousemove 579 19
echo click on CREATE button
drjtool click 1
echo move to Upload videos
drjtool mousemove 577 34
echo click Upload videos
drjtool click 1
echo move to SELECT FILES
drjtool mousemove 305 266
echo click on SELECT FILES
drjtool click 1
echo move mouse to Open button
drjtool mousemove 600 396
echo click open and pause a bit for video upload
drjtool click 1
sleep 20
secs=$(cat duration)
moretime=$(echo $secs/60|bc -l)
sleep $moretime
echo "mouse to NEXT button (accept defaults)"
drjtool mousemove 558 386
echo click on NEXT
drjtool click 1
echo move to radio button No it is not made for kids
drjtool mousemove  117 284
echo click radio button
drjtool click 1
echo back to NEXT button
drjtool mousemove 551 384
echo click NEXT
drjtool click 1
echo 'click NEXT again (then says no copyright issues found)'
drjtool click 1
echo click NEXT again
drjtool click 1
echo move to Unlisted visibility radio button
# [note that public would be drjtool mousemove 142 235, private is 142 181]
drjtool mousemove 142 208
echo click Unlisted
drjtool click 1
echo move to copy icon
drjtool mousemove 532 249
echo echo copy URL to clipboard
drjtool click 1
echo move to Save
drjtool mousemove 551 384
echo click Save
drjtool click 1
echo move to CLOSE
drjtool mousemove 434 304
echo click close
drjtool click 1

echo video URL
xsel -b|tee clipboard
echo '
kill chromium browser'
sleep 25
echo kill chromium
kill -9 %1
sleep 2
url=$(cat clipboard|xargs -0 echo)
echo url is $url

process.sh

                    

#!/usr/bin/bash
HOME=/home/pi
sleeptime=5
cd $HOME
# loop over all mp3 files in home directory
ls -1 record*mp3|while read line;do
 echo working on $line at $(date)
 video=$(echo ${line}|sed 's/mp3/flv/')
 echo creating flv video file $video
# create the video first
 ./mp32flv.sh $line
 echo move $line to mp3 directory
 [[ -d mp3s ]] || mkdir mp3s
 mv $line mp3s
 echo mv flv to upload directory
 [[ -d 00uploads ]] || mkdir 00uploads
 mv $video 00uploads
 echo start the upload
 ./auto-upload.sh
 echo get the url to this video on YouTube
 url=$(cat clipboard|xargs -0 echo)
 echo test that it worked
 if [[ ! "$url" =~ "http" ]]; then
   echo FAIL. Try once again
   ./auto-upload.sh
 fi
 echo send mail to Drj
 ./announceit.sh
 echo move video $video to flvs directory
 mv ./00uploads/$video flvs
 echo sleep for a bit before starting the next one
 sleep $sleeptime
done
echo All done with processing at $(date)

blinkLED.sh

                    

#!/bin/sh
# DrJ 8/30/2021
# https://www.jeffgeerling.com/blogs/jeff-geerling/controlling-pwr-act-leds-raspberry-pi
# put LED into GPIO mode
echo gpio | sudo tee /sys/class/leds/led1/trigger > /dev/null
# flash the bright RED PWR (power) LED quickly to signal whatever
while /bin/true; do
  echo 0|sudo tee /sys/class/leds/led1/brightness > /dev/null
  sleep 0.5
  echo 1|sudo tee /sys/class/leds/led1/brightness > /dev/null
  sleep 0.5
done

shineLED.sh

                    

#!/bin/sh
# DrJ 8/30/2021
# https://www.jeffgeerling.com/blogs/jeff-geerling/controlling-pwr-act-leds-raspberry-pi
# put LED into GPIO mode
echo gpio | sudo tee /sys/class/leds/led1/trigger > /dev/null
# turn on the bright RED PWR (power) LED
echo 1|sudo tee /sys/class/leds/led1/brightness > /dev/null

blinktwiceLED.sh

                    

#!/bin/sh
# DrJ 8/30/2021
# https://www.jeffgeerling.com/blogs/jeff-geerling/controlling-pwr-act-leds-raspberry-pi
# put LED into GPIO mode
echo gpio | sudo tee /sys/class/leds/led1/trigger > /dev/null
# flash the bright RED PWR (power) LED quickly to signal whatever
while /bin/true; do
  echo 0|sudo tee /sys/class/leds/led1/brightness > /dev/null
  sleep 3
  echo 1|sudo tee /sys/class/leds/led1/brightness > /dev/null
  sleep 0.35
  echo 0|sudo tee /sys/class/leds/led1/brightness > /dev/null
  sleep 0.35
  echo 1|sudo tee /sys/class/leds/led1/brightness > /dev/null
  sleep 0.35
done

announceit.sh

                    

#!/bin/sh
url=$(cat clipboard|xargs -0 echo)
mailx -r yourAddress@whatever.com -s "New youtube video $url posted" yourAddress@whatever.com<<EOF
Check out our latest recording:

      $url

Regards,
Yourself
EOF

crontab entries

                    

@reboot sleep 15; /home/pi/recordswitch.sh > recordswitch.log 2>&1
# launch vnc server on display 1
@reboot sleep 65;x0vncserver -passwordfile ~/.vnc/passwd -display :0 >  x0vncserver.log 2>&1

The idea

The recordswitch.sh script waits for input from the remote controller. It is programmed to kick off ffmpegwireless9.sh if the ENTER button is pushed, or process.sh if the UPLOAD button is pushed.

For testing purposes you may want to run process.sh by hand, i.e., ./process.sh, while you are viewing the display using a VNC viewer alongside the terminal screen.

The scripts are quite verbose and give lots of helpful output in their log files.

Upgrading from Raspberry Pi Lite to Raspberry Pi Desktop

I always like to start my RPi OS install with Raspberry Pi Lite. But to follow the upload parts of this post you really need Raspberry Pi Desktop. This article is a good write-up of how to upgrade to Desktop from Lite: How To Upgrade Raspbian Lite to Desktop (PIXEL, KDE, MATE, …) – RaspberryTips

Tips

Unfortunately the plugin I use inserts a blank line at the top. Those should all be removed.

After getting all the script, make them all executable in one go with a command such as chmod +x *sh

To read the input from the remote controller you need to set up evread.py and there may be some python work to do. This post has those details.

The chromium bowser needs to be run by hand one time over your VNC viewer. Its size has to be shrunk to 50% by running CTRL SHIFT – about four times. You need to log in to your YouTube or Gmail account so it remembers your credentials. And you need to og through the motions of uploading a video so it knows to use the 00uploads directory next time.

Don’t run a recording and an upload at the same time. I think the CPU would be taxed so I did not test that out. But you can record one day – even multiple recordings, and upload them a day or days later. That should work OK. It just processes the files one at a time, hopefully (untested).

announceit.sh is pretty dodgy. You have to understand SMTP mail somewhat to have a spitting chance for that to work. Fortunately I was an SMTP admin previously. So my ISP, Optimum, has a filter in place which prevents ordinary residential customers from sending out normal email to arbitrary SMTP addresses. However, to my surprise, they do run a mail relay server which you can connect to on the standard tcp port 25. I don’t really want to give it away but you can find it with the appropriate Internet search. I assume it is only for Optimum customers. Perhaps your ISP has something similar. So after you install exim4, you can configure a “smarthost” with the command dpkg-reconfigure exim4-config. But, again, you have to know a bit what you are doing. Suffice it to say that I got mine to work.

But for everyone else who can’t figure that out, just comment out this line in process.sh ./announceit.sh. put a # character in the front of the line to do that.

I have really only tested recordings of up to 45 minutes. I think an hour should be fine. I would suggest to break it up for longer.

The files can take a lot of space so you may need to clean up older files if you are a frequent user.

I’ve had about one failure during the upload out of about seven tests. So reliability is pretty good, but probably not perfect.

Why not just livestream? True, it’s sooo much easier. And I’ve covered how to do that previously. But, maybe it’s my WiFi, but its reliability was closer to 50% in my actual experience. I needed greater reliability and turns out I didn’t need the live aspect of the whole thing, just the recording for later critiqueing.

The recording approach I’ve taken uses ffmpeg to directly produce a mp3 file – it’s more compact than a WAV file. In and of itself the mp3 file may be useful to you, to, e.g., include as an attachment in email or whatever. For instance for a single song. All the mp3s are finally stored in a folder called mp3s, and all the videos are finally stored in a folder called flvs.

About that upload

The upload itself is super awesome to watch. I captured an actual automated upload with the script running on the right and the X Window display on the left in this YouTube video.

So the upload part was covered in this previous post.

Fixing recording which sounds like chipmunks

Somehow I managed to use some of these tools the other day and my mp3s ended up sounding like Alvin and the Chipmunks! I wondered if there was a way to recover them. I found there is, though I had to develop it a bit. It uses the new-ish rubberband filter of ffmpeg. I call this tiny script dechipmunk.sh:

                    

#!/bin/sh
input=$1
#ffmpeg -thread_queue_size 2048 -i $input -y -ac 1 -filter:a rubberband='tempo=2' -loglevel warning stretched$input
#ffmpeg -thread_queue_size 2048 -i $input -y -ac 1 -filter:a rubberband='pitch=2' -loglevel warning stretched$input
ffmpeg -thread_queue_size 2048 -i $input -y -ac 1 -filter:a "rubberband=pitch=0.3333:tempo=0.3333" -loglevel warning stretched$input

In my case I had to slow things down and lower the pitch by the same factor: one third, hence the 0.3333.

How to pass multiple options to an ffmpeg filter

In doing the above I had to work out the syntax for passing more than one option to the new rubberband filter of ffmpeg. I wanted to specify both the pitch and the tempo options. So you see from above they had to be separated with a colon and the whole filter expression enclosed in quotes. Hence the funny-looking

“rubberband=pitch=0.3333:tempo=0.3333”

Future development

Well, I’m thinking of removing the chit-chat from the recording in an automated fashion. That may mean applying machine learning, or maybe something simpler if someone has covered this territory before for the RPi. But it might be a good excuse to do a shallow dive into machine learning.

Conclusion

I’m sure this method of YouTube upload is very flaky and will probably only work once or twice, if at all. But at least in my trials, it did work a few times. So perhaps it could be hardened and made more error-correcting. There are a lot of moving parts for it all to work. But it’s definitely cool to watch it go when it is working!

References and related

Rii infrared remote control – only $12: Amazon.com: Rii MX3 Multifunction 2.4G Fly Mouse Mini Wireless Keyboard & Infrared Remote Control & 3-Gyro + 3-Gsensor for Google Android TV/Box, IPTV, HTPC, Windows, MAC OS, PS3 : Electronics

Reading keyboard input.

How To Upgrade Raspbian Lite to Desktop (PIXEL, KDE, MATE, …) – RaspberryTips

YouTube Livestreaming with a click of a button on Raspberry Pi

Automated YouTube video uploading from Raspberry Pi without using the YouTube api

Categories
Linux Raspberry Pi

Automated YouTube video uploading from Raspberry Pi without using the YouTube api

2021 Intro

This post promises more than it actually delivers, ha, ha. It is squarely aimed at the more mid to advanced RPi enthusiast. Most who read this will get discouraged and look for another solution. I did the same in fact and I will review my failures with alternatives.

The essence of this aproach is screen automation with a very nice tool called xdotool. For me it works. It will definitely, 100% require some tweaks for anyone else. This is not run a few installs, copy this code and you’re good to go. But if you have the patience, you wil be rewarded with either fully or at least semi-automated video uploads to YouTube from your Raspberry Pi.

One caveat. Please obey YouTube’s terms of service. In other words, don’t abuse this! As soon as someone starts using this method in an aggressive or abusive fashion, we will all lose this capability. They have crack security experts and could squelch this approach in a heartbeat.

I actually don’t have all the peices in place for myself, but I have enough cool stuff that I wanted to begin to share my findings.

One beautiful thing about what I’m going to show is that you get to see the cursor moving about the screen in response to your automated commands – you see exactly what it;s doing, which screens it’s clicking through, etc. So if there’s an issue – say YouTube changes its layout – you’ll most likely be able to know how to adapt.

What’s wrong with using YouTube’s api?

Plenty. It used to be feasible. It certainly would make all our lives a lot easier. But YouTube is not a charity. They have squeezed out the little guy by making the barrier to entry so high that it’s really only available to highly determined IT folks. It’s just too difficult to figure out all the neeed screens, etc, and all the help guides refer to older api versions where things were different. YouTube clamped down in July 2020 on who or what can use their api. There’s a lot of old HowTos pre-dating that that will just lead you to dead-ends. So, go ahead, I dare you to stop reading this and use the api and report back. Maybe yuo manage to create a project, great, and an api key, great, and even to assign your api key the correct YouTube specific permissions – all great, and associate crednetials – super, and finally borrow someone’s code to upload a video – been there, done that. That video will be listed private. So then you try to root around to see what you have to do to make it public. Ah, a project review. Great. You were only in test mode. So no your confronted with this form. If they cared about the little guy there would be a radio button – “I only wish to upload a few videos a week for a small cadre of users, spare me the bureaucracy,” and that’d be it. But, no… Are you applying for a quota? Huh? I just want to upload a video and have it marked as unlisted. Some users remarked they filled out the form, never got their project reviewed and never heard back. Maybe they’re the exception, I don’t know. It’s just over the top for me so I give up.

OK. So, maybe YoutubeUploader?

Nope. Doesn’t work. It’s based on the old stuff.

OK. What about that guy’s api-less Node.js uploader?

Maybe. I could not get it to work on RPi. But I didn’t try super hard. I just like rolling my own, frankly. My approach is much more transparent. At least this approach inspired me to imagine the approach I am about to share. Because I believe the Node.js guy is just doing screen scraping but you can’t even see the screens.

Or simply do a Livestream?

Agreed. Livestreaming is quite straightforward by comparison with what I developed. My blog post about one click livestreaming covers it. But I have not had good results with reliability. As often as it works, it doesn’t work. With this new approach I’m going to try to create separate steps so that if anything goes wrong, an individual step can be re-run. Another advantage of separating steps is that a recording can be done “in the field” and without WiFi access. Remember an RPi 3 works great for hours with a decent portable USB battery that’s normally used for phones. Then the resulting recording can be converted to video and uploaded once the RPi is back to its usual WiFi SSID.

Preliminary upload, October 2021

In the video below the right screen is a terminal window showing what the script is doing. It needs some tweaking, and the YouTube window gets stuck so it’s not showing some of the screens. But it’s already totally awesome – and it worked!

Watch an actual video get uploaded
Code for the above
                    

#!/bin/sh
# automate upload of YouTube videos
#
# define some functions
randomsleep(){
# sleep random amount between 1.5 to 2.5 seconds
t10=$(shuf -n1 -i 15-25)
t=$(echo $t10/10|bc -l)
sleep $t
}
drjtool(){
randomsleep
xdotool $1 $2 $3
randomsleep
}

echo Start video upload
echo set display to main display
export DISPLAY=:0
# launch chromium
echo launch chromium
chromium --kiosk https://studio.youtube.com/ > /dev/null 2>&1 &
sleep 20
echo move to CREATE button
drjtool mousemove 579 19
echo click on CREATE button
drjtool click 1
echo move to Upload videos
drjtool mousemove 577 34
echo click Upload videos
drjtool click 1
echo move to SELECT FILES
drjtool mousemove 305 266
echo click on SELECT FILES
drjtool click 1
echo move mouse to Open button
drjtool mousemove 600 396
echo click open and pause a bit for video upload
drjtool click 1
sleep 20
echo "mouse to NEXT button (accept defaults)"
drjtool mousemove 558 386
echo click on NEXT
drjtool click 1
echo move to radio button No it is not made for kids
drjtool mousemove  117 284
echo click radio button
drjtool click 1
echo back to NEXT button
drjtool mousemove 551 384
echo click NEXT
drjtool click 1
echo 'click NEXT again (then says no copyright issues found)'
drjtool click 1
echo click NEXT again
drjtool click 1
echo move to Unlisted visibility radio button
# [note that public would be drjtool mousemove 142 235, private is 142 181]
drjtool mousemove 142 208
echo click Unlisted
drjtool click 1
echo move to copy icon
drjtool mousemove 532 249
echo echo copy URL to clipboard
drjtool click 1
echo move to Save
drjtool mousemove 551 384
echo click Save
drjtool click 1
echo move to CLOSE
drjtool mousemove 434 270
echo click close
drjtool click 1

echo video URL
xsel -b|tee clipboard
echo kill chromium browser
sleep 5
echo kill chromium
kill -9 %1
url=$(cat clipboard|xargs -0 echo)
echo url is $url

I call the script auto-test.sh, just to give it a name.

Ingredients
  • RPi 3 or RPi 4
  • Raspberry OS with full GUI and autologin set up
  • tiger VNC, i.e., the package tigervnc-scraping-server
  • chromium browser – but I think that comes with the GUI install
  • xdotool (apt-get install xdotool)
  • xsel (apt-get install xsel)
  • YouTube account
  • crontab entries – see below
  • you do not need an HDMI display, except for the OS setup
  • a vncviewer such as Real VNC
Idea

Don’t use the GUI for anything else!

Crontab (do a crontab -e to get into your crontab) should contain these lines:

                    

@reboot sleep 15; /home/pi/recordswitch.sh > recordswitch.log 2>&1
# launch vnc server on display 1
@reboot sleep 65;x0vncserver -localhost no -passwordfile ~/.vnc/passwd -display :0 >  x0vncserver.log 2>&1

Work with chromium the first time by hand. As I recall you should:

  • Create a directory like 00uploads – so it appears highest in the list
  • put a single video in 00uploads
  • Do an upload by hand (to help chromium remember to choose this upload directory)
  • launch chromium browser
  • log into your YouTube account at https://studio.youtube.com
  • shrink the browser until its size is 50% (Ctrl-Shift – about four times)
  • Don’t add other tabs and stuff to Chromium

Then subsequent launches of chromium should remember a bunch of these settings, specifically, your login info, the shrunken size, the upload directory, and maybe the (lack of) other tabs.

The beauty of this approach is that it is more transparent than the alternatives. You see exactly what your program is doing. You can issue the xdotool commands by hand to, e.g., change up the coordinates a little bit. Or even enter a video title.

So getting back to the idea, the automation idea is to finish a video somehow, then move it to the 00uploads dircetory, invoke this uploader program, then either move it to a uploaded directory or some such.

Imagine the versatility if I used my remote controller for RPi to map one button for audio recording, and a second button for automating video upload! Well, when I find the time that’s what I plan to do. I will make a separate post where the recording and uploading are shown – more or less the culmination of all the pieces.

Oh, and back to the idea again, I wanted to share the unlisted link with band members. So, you see how it is basically in the result of xsel -b since xsel copied the clipboard which contained the YouTube URL for this video we just uploaded? I have to fix up the parsing because some junk characters are getting included, but I plan to email that link to myself first, where I will do a brief manual check, and then forward it to the rest of the band. so, again, it’s really cool that we could even think to pull that off with this simplistic approach.

Techniques developed for this project

Lots.

I “discovered” – in the sense that Columbus discovered America – xdotool as an amazing X Windows screen automation tool. I knew of autohotkey for Windows so inquired what was like it for X Windows. I further learned that xdotool is generally broken when it comes to use with traditional VNC servers such as the native tightvncserver. It simply doesn’t work. But Tiger VNC is a scraping server so it like shares your console screen and makes it available via VNC protocol. That’s required because to develop this approach you have to see what you’re doing. All those coordinates? it comes from experimentation.

I also learned how to embed a YouTube video in my blog post. In fact this is the very first video I made for a blog post. So I did a screen recording for the first time with screen recorder for Windows.

I landed on the idea of a side-by-side video showing my terminal running the automated script in one window and the effect it is having on the chromium browser in the other window running on the RPi.

I put a wrapper around xdotool to make things cleaner. (But it’s not done yet.)

I changed to two-factor authentication to see if it made a difference. It did not. It still remembers the authentication, thankfully, at least for a few days. I wonder for how long though. Hmm.

Kiosk mode. By launching chromium with kiosk mode it not only gives us more screen real estate to work in, it in principle should also permit you to interact with chromium in a regular fashion and still have it come up in a known, fixed position, which is an absolute requirement of this approach. All the buttons have to have the same coordinates from invocation to invocation.

I also developed ffmpeg-based converters which take wav files and converts them to mp3’s (a nice compact format. wav files are space hogs), and another which takes mp3 files and adds a gray screen and converts them to flv ([Adobe] Flash Video, I guess – a compact video file format which YouTube accepts).

I also learned the ffmpeg command to tell exactly how long a recording is.

I also learned how to turn off blocking in ffmpeg so that its constantly writing packets and thus not losing audio data at the end when stopped.

I came up with the idea of randomizing the sleep time between clicks to make it seem more human-like.

And mostly for the purpose of demonstrations, though it also greatly helps in debugging, I introduced around two seconds of sleep both before and after a command is issued. That really makes things a lot clearer.

The results of the clipboard, xsel -b, contains null bytes. I had trouble parsing it to pull out just the url, but finally landed on using xargs -0 which is designed to parse null-delimited strings. And it worked! This was a late edition and did not make it into the video, but is in the provided script above towards the bottom.

ffmpeg chokes on too-complicated filenames. Who knew? I had files containing colon (:) and dash (-) characters which work perfectly fine in linux, but ffmpeg was interpreting part of the filename as a command-line argument it appeared. The way out of that mess was to introduce file: in front of the filename.

I’ll probably put my ffmpeg tricks into my next post because I want to keep this one lean and focused on this one upload automation topic.

References and related

This post was preceded by my post on how to stream live to YouTube with a click of a button on a Raspberry Pi.

Categories
Linux Raspberry Pi Web Site Technologies

Raspberry Pi Project: YouTube livestreaming with a click of a button

Intro

Here I’ve combined work I’ve done previously into one single useful application: I can initiate the live streaming of our band practice on YouTube with the click of a single button on a remote control, and stop it with another click.

Equipment

Raspberry Pi 3 or 4 with Raspberry Pi OS, e.g., Raspbian Lite is just fine

Logitech webcam or USB microphone

USB extender (my setup needed this, others may not)

Universal USB-based remote control – see references for a known good one

Method 1

In this method I rapidly blink the onboard red power (PWR) LED of the RPi while streaming is active. Outside of those times it is a solid red. This is my preferred mode – it’s a very visible sign that things are working. I am very excited about this approach.

blinkLED.sh

                    

#!/bin/sh
# DrJ 8/30/2021
# https://www.jeffgeerling.com/blogs/jeff-geerling/controlling-pwr-act-leds-raspberry-pi
# put LED into GPIO mode
echo gpio | sudo tee /sys/class/leds/led1/trigger > /dev/null
# flash the bright RED PWR (power) LED quickly to signal whatever
while /bin/true; do
  echo 0|sudo tee /sys/class/leds/led1/brightness > /dev/null
  sleep 0.5
  echo 1|sudo tee /sys/class/leds/led1/brightness > /dev/null
  sleep 0.5
done

shineLED.sh

                    

#!/bin/sh
# DrJ 8/30/2021
# https://www.jeffgeerling.com/blogs/jeff-geerling/controlling-pwr-act-leds-raspberry-pi
# put LED into GPIO mode
echo gpio | sudo tee /sys/class/leds/led1/trigger > /dev/null
# turn on the bright RED PWR (power) LED
echo 1|sudo tee /sys/class/leds/led1/brightness > /dev/null

broadcastswitch.sh

                    

#!/bin/bash
# DrJ 8/2021
# Control the livestream of audio to youtube
# works in conjunction with an attached keyboard
# I use bash interpreter to give me access to RegEx matching
HOME=/home/pi
log=$HOME/audiocontrol.log
program=continuousaudio.sh
##program=tst.sh # testing
PGM=$HOME/$program
# de-press ENTER button produces this:
match="1, 28, 0"
epochsOld=0
cutoff=3 # seconds
DEBUG=1
ledtime=10
#
echo "$0 starting monitoring at "$(date)
# Note the use of script -q -c to avoid line buffering of the evread output
script  -q -c $HOME/evread.py /dev/null|while read line; do
[[ $DEBUG -eq 1 ]] && echo line is $line
# seconds since the epoch
epochs=$(date +%s)
elapsed=$((epochs-$epochsOld))
if [[ $elapsed -gt $cutoff ]]; then
  if [[ "$line" =~ $match ]]; then
    echo "#################"
    echo We caught this inpupt: $line at $(date)
# see if we are already running continuousaudio or not
    pgrep -f $program>/dev/null
# 0 means it's been found
    if [ $? -eq 0 ]; then
# kill it
      echo KILLING $program
      pkill -9 -f $program; pkill -9 ffmpeg
      pkill -9 -f blinkLED
      echo Shine the PWR LED
      $HOME/shineLED.sh
    else
# start it
      echo Blinking PWR LED
      $HOME/blinkLED.sh &
      echo STARTING $PGM
      $PGM > $PGM.log.$(date +%m-%d-%y:%H:%M) 2>&1 &
    fi
    epochsOld=$epochs
  fi
[[ $DEBUG -eq 1 ]] && echo No action taken. Continue to listen
fi
done

The crontab entry and the referenced files are the same as in Method 2.

Method 2

In method 2 I flash the built-in LED on the webcam for a few seconds before starting the audio, and again when the streaming has terminated – as visible signal that the button press registered.

broadcastswitch.sh

                    

#!/bin/bash
# DrJ 8/2021
# Control the livestream of audio to youtube
# works in conjunction with an attached keyboard
# I use bash interpreter to give me access to RegEx matching
HOME=/home/pi
log=$HOME/audiocontrol.log
program=continuousaudio.sh
##program=tst.sh # testing
PGM=$HOME/$program
# de-press ENTER button produces this:
match="1, 28, 0"
epochsOld=0
cutoff=3 # seconds
DEBUG=1
ledtime=10
#
echo "$0 starting monitoring at "$(date)
# Note the use of script -q -c to avoid line buffering of the evread output
script  -q -c $HOME/evread.py /dev/null|while read line; do
[[ $DEBUG -eq 1 ]] && echo line is $line
# seconds since the epoch
epochs=$(date +%s)
elapsed=$((epochs-$epochsOld))
if [[ $elapsed -gt $cutoff ]]; then
  if [[ "$line" =~ $match ]]; then
    echo "#################"
    echo We caught this inpupt: $line at $(date)
# see if we are already running continuousaudio or not
    pgrep -f $program>/dev/null
# 0 means it's been found
    if [ $? -eq 0 ]; then
# kill it
      echo KILLING $program
      pkill -9 -f $program; pkill -9 ffmpeg
      sleep 1
      echo turn on led for a few seconds
      $HOME/videotst.sh &
      sleep $ledtime
      pkill -9 ffmpeg
    else
# start it
      echo turn on led for a few seconds
      $HOME/videotst.sh &
      sleep $ledtime
      pkill -9 ffmpeg
      sleep 1
      echo STARTING $PGM
      $PGM &
    fi
    epochsOld=$epochs
  fi
[[ $DEBUG -eq 1 ]] && echo No action taken. Continue to listen
fi
done

videotst.sh

                    

#!/bin/sh
# just to get the webcam to light up...
ffmpeg -i /dev/video0 -f null - < /dev/null > /dev/null 2>1 &

crontab entry

@reboot sleep 15; /home/pi/broadcastswitch.sh > broadcastswitch.log 2>&1

continuousaudio.sh and ffmpegwireless6.sh

See this post: https://drjohnstechtalk.com/blog/2019/04/live-stream-to-youtube-from-a-raspberry-pi-webcam/

evread.py

See this post: https://drjohnstechtalk.com/blog/2020/12/how-to-create-a-software-keyboard/

The idea

I press the Enter button once on the remote to begin the livestream to YouTube. I press it a second time to stop.

By extension this could also control other programs as well (like the photo frame). And other keys could be mapped to other functions. Record-only, don’t livestream, anyone?

I want to do these things because it’s a little tight in the room where I want to livestream – hard to get around. So this keeps me from having to squeeze past other people to access the RPi to for instance power cycle it. In my previous treatment, I had livestreaming start up as soon as the RPi booted up, which means it would only stop when it was similarly powered off, which I found somewhat limiting.

The purpose of videotst.sh in Method 2

videotst.sh serves almost no purpose whatsoever! It can simply be commented out. It’s somewhat specific to my webcam.

You see, I wanted to get some feedback that when I pressed the ENTER button the remote control the RPi had read that and was trying to start the livestream. I thought of flashing one of the built-in LEDs on the RPi. I still need to look into that.

With the robotics team we had soldered on an external LED onto one of the GPIO pins, but that’s way too much trouble.

So what videotst.sh does for me is to engage the webcam, specifically its video component, throwing away the actual video but with the net result that the webcam’s built-in green LED illuminates for a few seconds! That lets me know, “Yeah, your button press was registered and we’re beginning to start the livestream.” You see, because when you run ffmpegwireless6.sh with this webcam, it’s all about the audio. It only uses the audio of the webcam and thus the green “in use” LED never illuminates, unfortunately, while it is livestreaming the pure audio stream. So, similarly, when you press ENTER a second time to stop the stream I illuminate the webcam’s LED for a few seconds by using videotst.sh once again.

Techniques developed for this project

evread.py does some nasty buffering of its output, meaning, although it dos read the key presses on the remote, it holds the results “close to its chest,” and then spits them out, all at once, when the buffer is full. Well, that totally defeats the purpose needed here where I want to know if there’s been a single click. After some insightful Internet searches (note that I did not use Google as a verb, a practice I carry into my personal communication) I discovered the program script, which, when armed with the arguments -q -c, allows you to unbuffer the output of a program! And, it actually works. Cool.

And I made the command decision to “eat” the input. You see the timer of 3 seconds in broadcastswitch.sh? After you’ve done any button press it throws away any further button presses for the next three seconds. I just think that’ll reduce the misfires. In fact, I might take up the practice of double-clicking the ENTER button just to be sure I actually pressed it.

I’m using the double bracket notation more in my bash scripts. It permits use of a RegEx comparison operator. =~. I love regular expressions. More the perl style, PCRE, while this uses extended regular expressions, ERE. But I suppose those are good as well.

Getting control over the power LED was a nice coup. I’m only disappointed that you cannot control its brightness. In the dark it throws off quite a bit of light. But you cannot.

The green LED does not seem nearly as bright so I chose not to play with it. What I don’t want is to have to strain to see whether the thing is livestreaming or not.

Of course getting the whole remote control thing to work at all is another great advancement.

Techniques still to be developed

I still might investigate using voice-driven commands in place of a remote. Obviously, that’s a big nut to crack. Even if I managed to turn it on, turning it off while ffmpeg has commandeered the audio channel is even harder. I wonder if ffmpeg can split the audio stream so another process can be run alongside it to listen for voice commands? Or if an upstream process in front of ffmpeg could be used for that purpose? Or simply run with two microphones (seems wasteful of material)?? Needs research.

Suppose you want to take this on the road? Internet service can be unreliable after all. It’s well known you can power the RPi 3 for many hours with a small portable battery. So how about mapping a second button on the remote to a record-only mode (using the arecord utility, for instance)?? Then you can upload the audio at a time of you convenience. That’s something I can definitely program if I find I need it.

Lingering Problems with this approach

Despite all the care I’ve taken with the continuousaudio.sh script, still, there are times when YouTube does not show that a livestream is going on. I have no idea why at this point. If I knew the cause, I’d have fixed it!

As the livestream aspect of this is actual immaterial to me, I will probably switch to a pure recording mode where I upload in a later step – perhaps all done by the remote control for pure convenience.

Since this blog post has become popular, I may keep it preserved as is and start a new one for this recording approach as some people may genuinely be interested in the livestream aspect.

A very rough estimate of the failure rate is maybe as high as 50% but probably no lower than 25%. So, not great odds if you’re relying on success.

There’s another issue which I consider more minor. The beginning of the stream always sounds like a tape played on fast forward for a few seconds. The end also cut off a few seconds early I think.

Conclusion

We have presented a novel approach to livestreaming on a Raspberry Pi 3 using a remote control for added convenience. All the techniques were home-developed at drjohnstechtalk.com. The materials don’t cost much and it really does work.

References and related

Rii infrared remote control – only $12: Amazon.com: Rii MX3 Multifunction 2.4G Fly Mouse Mini Wireless Keyboard & Infrared Remote Control & 3-Gyro + 3-Gsensor for Google Android TV/Box, IPTV, HTPC, Windows, MAC OS, PS3 : Electronics

25′ USB 2 extender for placing a webcam or USB mic at a distance from the RPi, $15: Amazon.com: HDE USB Extension Cable (USB 2.0 Type A Male to Female) High Speed Data and Power Extension Cable with Active Repeater (25 ft) : Electronics

Reading keyboard input.

Using Remote control to interact with a RPi-based photo frame.

ffmpeg settings to send just audio to YouTube, suppressing video

Battery to make the RPi 3 portable, $17: Amazon.com: Omars Power Bank 10000mAh USB C Battery Pack Slimline Portable Charger with Dual USB Output Compatible with iPhone Xs/XR/XS Max/X, iPad, Galaxy S9 / Note 9 : Cell Phones & Accessories. I’m not exactly sure what to do for an RPi 4 however.

Exetended Regular Expressions

How to control the power to the RPi’s LEDs: https://github.com/mlagerberg/raspberry-pi-setup/blob/master/5.2-leds.md

Categories
Linux Raspberry Pi

Raspberry Pi + LED Matrix Display project

Intro

A friend bought a bunch of parts and thought we could work on it together. He wanted to reproduce this project: Raspberry Pi Audio Spectrum Display – Hackster.io

So of interest here is that it involved a 64×64 LED matrix display, and a “hat” sold by Adafruit to supposedly make things easier to connect.

Now I am not a hardware guy and never pretended to be. When we realized that it required soldering, I bought those supplies but I didn’t want to be the one to mess things up so he volunteered for that. And yes, it was a mess.

Equipment

See later on in the post for the equipment we used.

The story continues

Who knew that gold on the PCB could be ruined so easily after you’ve changed your mind about a soldered joint and decided to undo what you’ve done? The experience pretty much validated my whole approach to staying away from soldering. So we ruined that hat and had to order another one. While we waited for it I developed a certain strtategy to deal with the shortcomings of the partially destroyed hat. See the section at the bottom entitled Recipe for a broken hat.

Comments on the project

I don’t think the guy did a good job. Details left out where needed, extra stuff added. For instance you don’t need to order that Firebeetle – it’s never used! Also, I’ve been told that his python isn’t very good either. But in general we followed his skimpy instructions. So we ordered the LED matrix from DFRobot, not Adafruit, and I think it’s different. In our case we did indeed follow the project suggestion to wire the 2×8 pin as shown in their (DFRobot’s) picture, leaving out the white wire. Once we soldered the white wire to the hat’s gpio pin 24, we were really in business. What we did not need to do is to solder pin E to either pin 8 or 16. (This is something you apparently need to do for the Adafruit LED.) In our testing it didn’t seem to matter whether or not those connections were made so we left it out on our second hat.

You think he might have mentioned just how much soldering is involed. Let’s see you have 2×20 connector, a 2×8 connector plus a single gpio connection = 57 pins to solder. Yuck.

What we got to work

Deploying images on these LED displays is cool. You just kind of have to see it. It’s hard to describe why. The picture below does not do it justice. Think stadium scoreboard.

In rpi-rgb-led-matrix/utils directory we followed the steps in the README.md file to compile the LED viewer:

sudo apt-get update
sudo apt-get install libgraphicsmagick++-dev libwebp-dev -y
make led-image-viewer
                    
#!/bin/sh
# invert images because the sound stuff is otherwise upside-down
sudo led-image-viewer –led-pixel-mapper=”U-mapper;Rotate:180″ –led-gpio-mapping=adafruit-hat –led-cols=64 –led-rows=64 /home/pi/walk-in-the-woods.jpg
Walk in the woods

Do we have flicker? Just a tiny bit. You wouldn’t notice it unless yuo were staring at it for a few seconds, and even then it’s just isolated to a small section of the display. Probably shoddy soldering – we are total amateurs.

Tip for your images

Consider that you only have 64 x 64 pixles to work with. So crop your pictures beforehand to focus on the most interesting aspect – people if there are people in the picture (like we’ve done in the above image), specifically faces if there are faces. Otherwise everything will just look like blurs and blobs. You yourself do not have to resize your pictures down to 64 x 64 – the led software will do the resizing. So just focus on cropping down to a square-sized part of the picture you want to draw attention to.

Real-time audio

So my friend got a USB microphone. I developed the following script to make the python example work with real-time sounds – music playing, conversatiom, whatever. It’s really cool – just the slightest lag. But, yes, the LEDs bounce up in response to louder sounds.

So in the directory rpi-rgb-led-matrix/bindings/python/samples I created the script drjexample.sh.

                    

#!/bin/sh
# DrJ 6/21
# make the LED react to live sounds by use of a USB microphone
# I am too lazy to look up how to make the python program read from STDIN so I will just
# make the equivalent thing by creating test.wav as a nmed pipe. It's an old linux trick.

rm test.wav; mkfifo test.wav

# background the python program. It will patiently wait for input
sudo python spectrum_matrix.py &

# Now run ffmpeg
# see my own post, https://drjohnstechtalk.com/blog/2019/04/live-stream-to-youtube-from-a-raspberry-pi-webcam/
ffmpeg \
-thread_queue_size 4096 \
-f alsa -i plughw:1,0 \
-ac 2 \
-y \
test.wav

So note that by having inverted the image (180 degree rotation) we have the sounds bars and images both in the same direction so we can switch between the two modes.

I believe to get the python bindings to work we needed to install some additional python libraries, but that part is kind of a blur now. I think what should work is to follow the directions in the README.md file in the directory rpi-rgb-led-matrix/bindings/python, namely

sudo apt-get update && sudo apt-get install python2.7-dev python-pillow -y
make build-python
sudo make install-python

Hopefully that takes care of it. For sure you need numpy.

Future project ideas

How about a board that normally plays a slideshow, but when the ambient sound reaches a certain level – presumably because music is playing – it switches to real-time sound bar mode?? We think it’s doable.

Recipe for a broken hat

For the LED matrix display we got the DFRobot one since that’s what was linked to in the project guide. But the thing is, the reviewer’s write-up is incomplete so what you need to do involves a little guesswork.

At the end of the day all we could salvage while we wait for a new Adafruit hat to come in is the top fourth of the display! The band below it is either blank, or if we push on the cables a certain way, an unreliable duplicate of the top fourth.

The next band suffers from a different problem. Its blue is non-functional. So it’s no good…

And the last band rarely comes on at all.

OK? So we’re down to a 16 x 64 pixel useable area.

But despite all those problems, it’s still kind of cool, I have to admit! I know at work we have these digital sign boards and this reminds me of that. So first thing I did was to create a custom banner – scrolling text.

I call this display program drjexample2.sh. I put it in the directory rpi-rgb-led-matrix/examples-api-use

                    
#!/bin/sh
# DrJ – 6/21
# nice example
sudo ./demo -D1 –led-rows=64 –led-cols=64 –led-multiplexing=1 –led-brightness=50 -m25 /home/pi/pil_text.ppm

But that requires the existence of a ppm file containing the text I wanted to scroll, since I was working by example. So to create that custom PPM file I created this python script.

                    
#!/usr/bin/python3
from PIL import Image, ImageDraw, ImageFont

# our fonts
###fnt = ImageFont.truetype(“/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf”, 14)
fnt = ImageFont.truetype(“/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf”, 16)
fnt36 = ImageFont.truetype(“/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf”, 36)
fnt2 = ImageFont.truetype(“/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf”, 18)
fntBold = ImageFont.truetype(“/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf”, 40)

###img = Image.new(‘RGB’, (260, 30), color = (73, 109, 137))
img = Image.new(‘RGB’, (260, 30), color = ‘black’)

d = ImageDraw.Draw(img)
###d.text((0,0), “Baby, welcome to our world!!!”, font=fnt, fill=(255,255,0))
d.text((0,0), “Baby, welcome to our world!!!”, font=fnt, fill=’yellow’)

#img.save(‘pil_text.png’)
img.save(‘pil_text.ppm’, format=’PPM’)

Once I discovered the problem with the bands – by way of running all the demos and experimenting with the arguments a bit – I noticed this directory: rpi-rgb-led-matrix/utils. I perked right up because it held out the promise of displaying jpeg images. Anyone who has seen any of my posts know that I am constantly putting out raspberry pi based photo displays in one form or another. For instance see https://drjohnstechtalk.com/blog/2021/01/raspberry-pi-photo-frame-using-the-pictures-on-your-google-drive-ii/

or

https://drjohnstechtalk.com/blog/2020/12/raspberry-pi-photo-frame-using-your-pictures-on-your-google-drive/

Matrix Display

But see how cool it is? No? It’s a sleeping, recumbent baby. It’s like the further away from it you get, the clearer it becomes. Trust me in person it does look good. And it feels like creating one of those bright LED displays they use in ballparks.

But this same picture also shows the banding problem.

To get the picture displayed I first cropped it to make it wide and short. I wisely chose a picture which was amenable to that approach. I created this display script:

                    
#!/bin/sh
sudo led-image-viewer –led-no-hardware-pulse –led-gpio-mapping=adafruit-hat –led-cols=64 –led-rows=64 –led-multiplexing=1 /home/pi/baby-sleeping.jpg

But before that can all work, you need to compile the program. Just read through the README.md. It gives these instructions which I followed:

sudo apt-get update
sudo apt-get install libgraphicsmagick++-dev libwebp-dev -y
make led-image-viewer

It went kind of slowly on my RPi 3, but it worked without incident. When I initially ran the led-image-viewer nothing displayed. So the script above shows the results of my experimentation which seems appropriate for our particular matrix display.

How did we get here?

Just to mention it, we followed the general instructions in that project. So I guess no need to repeat the recipe here.

Slideshow

You know I’m not going to let an opportunity to create a slideshow go to waste. So i created a second appropriately horizontal image which I might effectively show in my narrow available band of 16 x 64 pixels. just to share the little script, it is here:

                    
#!/bin/sh
cd /home/pi
sudo led-image-viewer –led-no-hardware-pulse –led-gpio-mapping=adafruit-hat –led-cols=64 –led-rows=64 –led-multiplexing=1 -w6 -f baby-and-mom.jpg baby-sleeping.jpg

This infinitely loops over two pictures, displaying each for six seconds.

Start on boot

I have the slideshow start on boot using a simple technique I’ve developed. You edit the crontab file (crontab -e) and enter at the bottom:

                    
@reboot sleep 35; /home/pi/rpi-rgb-led-matrix/utils/jdrexample2.sh > drjexample.log 2>&1
Lessons learned

My friend ordered all the stuff listed on the DFRobot project page, including their 64×64 LED matrix. Probably a mistake. They basically don’t document it and refer you to Adafruit, where they deal with a 64×64 LED matrix – their own – which may or may not have the same characteristics, leaving you somewhat in limbo. Next time I would order from Adafruit.

As mentioned above that gold foil on printed circuit boards really does come off pretty easily, and then you’re hosed. Because of the lack of technical specs we were never really sure if we needed to solder the E contact to 8 or to 16 and destroyed all those terminals in the process of backing out.

I actually created custom ppm files of solid colors, red, blue, green, white, so that I could prove my suspicions about the third band. Red and green display fine, blue not at all. White displays as yellow.

Viewed close up, the LED matrix doesn’t look like much, and of course I was close up when I was working with it. But when I stepped back I realized how beautiful a brightly illuminated picture of a baby can be! The pixels merge and your mind fills in the spaces between I guess.

The original idea was to tackle sound but I got stuck on the ability to use it as a photo frame (you know me). But he wants to return to sound which I am dreading….

Testing audio

In our first tests. the audio example wasn’t working. But now it seems to be. The project guy’s python code is named spectrum_matrix.py if I recall correctly. It goes into rpi-rgb-led-matrix/bindings/python/samples. And as he says, you run it from that directory as

$ sudo python spectrum_matrix.py

But, his link to test.wav is dead – yet another deficiency in his write-up. At least in my testing not every possible WAV file may work. This one, moo sounds, does however. http://soundbible.com/grab.php?id=1778&type=wav So, it plays for a few seconds – I can hear it through earphones – and the LEDs kind of go up and down. We recorded a wav file and found that that does not work. The error reads like this:

Home directory not accessible: Permission denied
W: [pulseaudio] core-util.c: Failed to open configuration file '/root/.config/pulse//daemon.conf': Permission denied
W: [pulseaudio] daemon-conf.c: Failed to open configuration file: Permission denied
Traceback (most recent call last):
File "spectrum_matrix.py.orig", line 56, in
matrix = calculate_levels(data,chunk,sample_rate)
File "spectrum_matrix.py.orig", line 49, in calculate_levels
power = np.reshape(power,(64,chunk/64))
File "/usr/lib/python2.7/dist-packages/numpy/core/fromnumeric.py", line 292, in reshape
return _wrapfunc(a, 'reshape', newshape, order=order)
File "/usr/lib/python2.7/dist-packages/numpy/core/fromnumeric.py", line 56, in _wrapfunc
return getattr(obj, method)(*args, **kwds)
ValueError: cannot reshape array of size 2048 into shape (64,64)

Note that I had renamed the original spectrum_matrix.py to spectrum_matrix.py.orig because we started messing with it. Actually, I pretty much get the same error on the file that works; it’s just that I get it at the end of the LED show, not immediately.

Superficially, the two files differ somewhat in their recording format:

$ file ~/voice.wav moo.wav

/home/pi/voice.wav: RIFF (little-endian) data, WAVE audio, Microsoft PCM, 16 bit, mono 44100 Hz
moo.wav: RIFF (little-endian) data, WAVE audio, Microsoft PCM, 16 bit, stereo 32000 Hz

I played the voice.wav on a Windows PC – it played just fine. Just a little soft.

So what’s the essential difference between the two files? Well, something that jumps out is that the one is mono, the other stereo. Can we somehow test for that? Yes! I made the simplest possible conversion af a mono to a stereo file with the following ffmpeg command:

$ ffmpeg -i ~/voice.wav -ac 2 converted.wav

$ file converted.wav


converted.wav: RIFF (little-endian) data, WAVE audio, Microsoft PCM, 16 bit, stereo 44100 Hz

I copy converted.wav to test.wav and re-run spectrum_matrix.py. This time it works!

Not sure how my friend produced his wave file. But I want to make one on my own. He had plugged a USB microphone into the RPi. I have done research somewhat related to this – publishing a livestream to Youtube, audio only, video grayed out. That’s in this post: https://drjohnstechtalk.com/blog/2019/04/live-stream-to-youtube-from-a-raspberry-pi-webcam/ So I am not afraid to se ffmpeg any longer. So I created this tiny script, record.sh, with my desired arguments:

                    
#!/bin/sh
# see my own post, https://drjohnstechtalk.com/blog/2019/04/live-stream-to-youtube-from-a-raspberry-pi-webcam/
ffmpeg \
-thread_queue_size 4096 \
-f alsa -i plughw:1,0 \
-ac 2 \
/tmp/ffmpeg.wav

And I ran it while speaking loudly into the mic. It ran OK. The output file comes out as

test.wav: RIFF (little-endian) data, WAVE audio, Microsoft PCM, 16 bit, stereo 48000 Hz

And…it plays with the LEDs dancing. Great.

Audio: LED responds to live input

My friend wants the LED to respond to live input such as his stereo at home. Being a terrible python programmer but at least middling linux techie, I see a way to accomplish it without his having to touch the sample program by employing an old Unix trick: named pipes. So I create this script, drjexample.sh which combines all the knowledge we gained above into one simple script:

                    

#!/bin/sh
# DrJ 6/21
# make the LED react to live sounds by use of a USB microphone
# I am too lazy to look up how to make the python program read from STDIN so i will just
# make the equivalent thing by creating test.wav as a nmed pipe. It's an old linux trick.

rm test.wav; mkfifo test.wav

# background the python program. It will patiently wait for input
sudo python spectrum_matrix.py &

# Now run ffmpeg
# see my own post, https://drjohnstechtalk.com/blog/2019/04/live-stream-to-youtube-from-a-raspberry-pi-webcam/
ffmpeg \
-thread_queue_size 4096 \
-f alsa -i plughw:1,0 \
-ac 2 \
-y \
test.wav

So a named pipe is just that. Instead of the pipe character we know and love, you coordinate process output from the first process with process input of the second process by way of this special file. The operating system does all the hard work. But it works just as though you had used the | character.

Best of all, this script actually works, to wit, the LED is now responding to live input. I see it jump when I say test into the microphone. Unknown to me is if it will play for extended periods of time – it would be easy for one process to output faster than the other can input for instance, so a backlog builds up. The responsiveness is good, I would guess around no more than 200 ms lag.

Equipment

We used the equipment from this post, except the Firebeetle. You know, that’s just another reason I consider that post to be a sloppy effort. Who lists a piece of equipment that they don’t use?? And again, next time I would rather search for an LED display from Adafruit. We use an RPi 3 and installed the image on a micro SD card with the new-ish Raspberry Pi imager, which works just great: Introducing Raspberry Pi Imager, our new imaging utility – Raspberry Pi

Oh, plus the soldering iron and solder. And a multi-colored ribbon cable with female couplers at one end and a 2×8 connector on the other. Not sure if that came with the LED or not since I didn’t order it.

Our power supply is about 5 amps and plugs into the hat. We do not need power for the RPi.

References and related

This post makes it seem like a walk in the park. Our experience is not so much. Raspberry Pi Audio Spectrum Display – Hackster.io

People seem to like this Raspberry Pi photo frame post I did which draws photos from your Google drive. https://drjohnstechtalk.com/blog/2020/12/raspberry-pi-photo-frame-using-your-pictures-on-your-google-drive/

Introducing Raspberry Pi Imager, our new imaging utility – Raspberry Pi – for putting the operating system formerly known as Raspbian onto a micro SD card.

test.wav (Use this moo wav file and rename to test.wav): http://soundbible.com/grab.php?id=1778&type=wav

Useful ffmpeg commands: FFmpeg Commands: 31 Must-Haves for Beginners in 2022 – VideoProc

Or 20+ FFmpeg Commands For Beginners – OSTechNix

How I figured out hot to livestream audio to YouTube without video from a RPi using ffmpeg is documented here: https://drjohnstechtalk.com/blog/2019/04/live-stream-to-youtube-from-a-raspberry-pi-webcam/

Github project for this effort (not completely working as of yet):