Our EDI group hails me last Friday and says they can’t reach their VANs, or at best intermittently. What to do, what to do… I go on the offensive and say they have to stop using FTP (and that’s literal FTP, not sftp, not FTPs, just plain old FTP), it’s been out of date for at least 15 years.
But that wasn’t really helping the situation, so I had to dig a lot deeper. And frankly, I was coincidentally having intermittent issues with my scripted speedtests. Could the two be related?
The details
We have a bunch of synthetic monitors we run though that same firewall. They were failing every few minutes, and then became good.
And these FTPs were like that as well. Some would work and then minutes later not work.
The firewall person on call looked at the firewall, saw some of the described traffic passing through, and declared firewall is fine.
So I got a more cooperative firewall colleague on this. And he got a really expert Checkpoint support person on the call. That guy led us to look at SYN DEFENDER which is part of IPS and enabled via fw accel. If it sees too many out of state packets in a given time it will shut down the interface where the problem was observed!
The practical effect is that even if you’re taking traces on the Checkpoint, checking the logs, etc, you won’t see the traffic! So that really throws most firewall admins is this situation is so unusual and they are not trained to look for it.
In this case it was an internal firewall and ir was comfortable to disable SYN DEFENDER on it. All problems went away after that.
Four months later…
Then four months later, after the firewall was upgraded to v 81.10, they must have set SYN DEFENDER (AKA synatk) up all over again. And of course no one was thinking about it or expecting what happened next, which is, these exact same problems started all over again. But there were different firewall colleagues involved, none with any first-hand experience of the issue. Then I got involved and just sort of tackled my way through it in a trouble-shooting session. No one was placing any judgments (my-stuff-is- fine,-yours-must-be-broken kind of thinking). Then I eventually recalled the old problem, and looked up this post to help name it – SYN DEFENDER – so that that would be meaningful to the firewall colleague. Yup, he took it from there. And we were good. I admonished the on-call guy who totally missed it, and he humbly admitted to not being familiar with this feature and how does it work. So I will explain it to him.
These are probably the defaults as we haven’t messed with them. Right now you see it’s disabled. It spontaneously re-enabeld itself after only a few days, and the problems started all over again.
A complex environment produces some too-strange-to-be-true type of issues. Yesterday was one of those days. Let me try to set this up like a script from a play.
The setting
A non-descript server room somewhere in the greater New York City area.
The equipment
A generic security appliance we’ll call ThousandEyes PX, just to make up a name.
Cisco Nexus 7K plus a FEX
The players
Dr John – the protagonist
PCT – a generic network vendor
Florence Ranjard – an admin of ThousandEyes PX in France
Shake Abel – a server room resource in PA
Cloud Johnson – someone in Request Management
Bill Otto – a network guy at heart, forced to deal with his now vendor-managed network via ITIL
The processes
ITIL – look it up
Scene 1
An email from Dr John….
Hi Bill,
Thanks.
Well that’s messed up, as they say. I wouldn’t believe it if I hadn’t seen it for myself. Someone, “stole” our port and assigned it to a different device on a different vlan – despite the fact that it was in active use!
I guess I will try to “steal” it back, assuming I can find the IT Catalog article, or maybe with the help of Cloud.
Fortunately I have console access to the Fireeye. I artificially introduced traffic, which I see reflected in the port statistics. So I know the ThousandEyes is still connected to this port, despite the wrong vlan and description.
Regards,
Dr John
Scene 2
One Week earlier
Siting at home due to Covid, Florence realizes she can no longer access the management port of her group’s ThousandEyes security appliance located in another continent. She beings to investigate and even contacts the vendor…
I began to implement the autmoation of speedtest checks. I was running the jobs every 10 minutes, but we noticed something flaky in the results. On the hour and on the half hour the tests seemed to be garbage. What’s going on?
Our findings
Well, if you use a scheduler to run a speedtest every 10 minutes, it will start exactly on the hour and exactly on the half-hour, amongst other start times. We were running it eight times to test eight different paths. Only the last two were returning reliable results. The early ones were throwing errors. So I introduced an offset to run the jobs at 2,12,22,32,42,52 minutes. And with this offset, the results became much more reliable.
The inevitable conclusion is that too many other people are running tests exactly on the hour and half hour. A single run takes roughly 30 seconds to complete. And it must be that the servers which speedtest rely on are simply overwhelmed and refuse to do more tests.
I don’t think we ask a lot of our switches. A packet comes in on this port and goes out on this other port. That sort of thing. But, apparently, when the switch is a Cisco Nexus, that is asking too much. Maybe it will go to that other port. Let’s see, it depends who sent it. So then again, maybe not. So maybe device A can ping device B. Device B cannot ping device A. But it can ping device C and device C can ping A. Or maybe Jack can ping device 1 but not device 2 on the same subnet. Yet Jill can ping device 2 but not device 1!
Yes. Those are not theoretical examples, but, sadly, actual examples lifted from recent massive debugging sessions I’ve participated in, which, eventually, focused on the Nexus switch not treating all traffic as it should. Another example: larger packets were not getting through on the same vlan.
Sometimes the Cisco support engineer is too clever by half. We had one find a few errors on supervisor board 5 so we switched to board 6. That did nothing. We finally noticed that all problems were related to FEXes connected to slot 8. As a test we plugged a FEX into a different slot and, voila, things began to work better. But the Cisco guy did not see any errors on slot 8.
Well, you only get the same Cisco engineer for a limited time. Then their shift turns over and you get to explain the whole problem all over again to the next one. Yeah, supposedly they’re briefed, but ni reality not so much. So finally the second engineer was a little more pragmatic than the first too-clever guy. This is a quote from her: “I do not see errors on that module, but logically, it has to be the problem.” This of course is after we presented all the evidence to her. So we ordered an RMA, replaced that board and yes everything began to work.
I’ve been involved in two such massive debugging session over the last four months. No one from the Cisco side is saying it, but I will. These Nexus switches define flows behind the scenes. They can probably do some very clever things with these flows. But it is no longer a simple switch. And, sadly, when the flows don’t all work, no one at Cisco is prepared to look for that issue or even consider it as a possibility. They look for obscure errors. And, heaven forbid, they are, I guess, incapable of proving to themselves their switch as at fault – eating packets – by doing what any first semester network class would have, namely, mirroring some of the ports and examining the traffic for themselves. It is instead up to the customer to prove the switch is at fault.
So in my first debugging session, which lasted about 16 hours, Cisco did not really find the errors that would lead them to believe slot 8 was a problem. Yet its replacement fixed everything. In fairness, in the second debugging session, which only lasted five hours or so, they did see errors which justified a replacement. But at no time did they ever use the word flow or offer in detail how the errors they saw could have produced the weird results we were seeing (that was the Jack and Jill example from above).
Conclusion
We guys who run servers would like to think a switch is a switch is a switch. But Cisco is requiring us to up our game, even if they don’t up theirs. Their Nexus switches absolutely can eat some of your network traffic while passing other over the same ports. I’d like to call that flows, even if they refuse to use that term.
Answer: the one where their switch eats the DHCPDISCOVER packets. And the amazing thing is they never learn. And the second amazing thing is that they actually don’t apply the most basic networking debugging techniques when such a problem occurs. I’m talking your basic, DHCPDISCOVER packet goes to your switch, same DHCPDISCOVER packet never arrives to the DHCP server on same switch. We know it to be the case, but, to help convince yourself that your switch is eating the packets, do networky things like create a span port of the DHCP server’s port to prove to yourself that no DHCP requests are coming in. And yet, they are never prepared to do that, to propose that. So instead indirect proxies are used to draw the conclusion.
I’ve been involved in three of four such debugging sessions. They take hours. I took notes when it happened again this weekend. I guess that setup is pretty typical of how it plays out. A data center was moved, including a DHCP server. The new data center has a MAN network to the old one. All IPs were preserved. When they turned on the moved DHCP server DHCP lease were no longer getting handed out. In fact it was worse than that. With the moved DHCP sever turned off, most DHCP leases were working. But with it on, that’s when things really began to go south!
Here’s the switch port they noted for the iDRAC:
sh run int gi1/0/24
Building configuration…
Current configuration : 233 bytes
!
interface GigabitEthernet1/0/24
description --- To-cnshis01-iDRAC - iDRAC
switchport access vlan 202
switchport mode access
logging event link-status
speed 100
duplex full
spanning-tree portfast
ip dhcp snooping trust
end
The first line of course if the IOS command. OK, so they had that on the iDRAC, right. But on the actual server port they had this:
sh run int gi1/0/23
Building configuration…
Current configuration : 258 bytes
!
interface GigabitEthernet1/0/23
description --- To-cnshis01-Gb1 - Gb1
switchport access vlan 202
switchport mode access
logging event link-status
spanning-tree portfast
service-policy input PMAP_COS_REMARK_IN
service-policy output PMAP_COS_OUT
end
I basically told them cheekily up front that this is usually a network switch problem and that they have to play with the DHCP snooping enable setting.
And I have to say that the usual hours of debugging were short-circuited this time as they seemed to believe me, and simply experimented by adding
ip dhcp snooping trust
to the DHCP server’s main port. We immediately began seeing DHCPDISCOVER pakcets come in to the DHCP server, and the team testified that people were getting leases.
Final mystery explained
Now why were things behaving really badly – no leases – when the DHCP server was up but no DHCPDISCOVER requests were getting to it? I have the explanation for that as well. You see there is a standby DHCP server which is designed for failure of the primary DHCP server. But not for this type of failure! That’s right. There is an out-of-band (by that I mean not carried over DHCP ports like UDP port 67) communication between standby and primary which tells the standby Hey, although you got this DHCPDISCOVER request, ignore it becasue the primary is active and will serve it! And meanwhile, as we have said, the primary wasn’t getting the requests at all. Upshot: no one gets leases.
Just to mention it
My second-to-last debugging session of this sort was a little different. There they mentioned that there was a “global setting” which governed this DHCP snooping on the switch. So they had to do something with that (enable or disable or something). So there was no issue with the individual switch ports. For me that’s just a variation on the same theme.
2023 debugging session
Well, nothing has really changed two years after I originslly posted this article and I get on these troubleshooting sessions with the vendor, people from the firewall team, vendor management people – it’s quite an affair. And as before it always takes a minimum of a couple hours for the network vendor to find their mistakes in their configuration. I have just done two of these sessions in the last week.
The last one was a tad different. There was a firewalled segment. The PCs behind it were not receiving IPs from the dhcp server. It is worth mentioning. Someone suggested a traceroute from the dhcp server to this subnet. And then a traceroute to another subnet (non-firewalled) at the same site which was working. They looked completely different after the first few hops! So about an hour after that they found that they had forgotten to add a route for this subnet pointing to the firewall. And that makes sense in that on the dhcp server – unlike in most cases – I was see the DHCP DISCOVER and it was replying with a DHCP OFFER. But that DHCP OFFER was simply not getting to the firewall at the site.
Cute Mnemonic to remember the four DHCP phases
Can you never remember the phases of the DHCP protocol like me? Then remember only this: DORA.
Discover
Offer
Request
Ack
What’s the idea behind this feature?
Having done a total of zero minutes of research on the topic, I will anyway weigh in with my opinion! Suppose someone comes along and plugs in a consumer grade home router into your network. It’s probably going to act as a rogue DHCP server. Imagine the fun trying to debug that situation? We’ve all been there… These rogue devices are probably fairly common. So if your corporate switch doesn’t suppress certain DHCP packets from ports where they are not expected, then this rogue device will begin to take down your subnet and totally bewilder everyone. I imagine this setting that is the topic of this blog post stems from trying to suppress all unknown DHCP packets in advance. Its just that sometimes the setting is taken too far and, e.g., a firewall which relays DHCP requests is also getting its DHCP packets suppressed.
Conclusion
I normally would have presented this as part of my IT Detective series. But I feel this is more like a lament about the sad state of affairs with our network providers. And though I’ve seen this issue about four times in the past 12 months, they always act like they have no idea what we’re talking about. They’ve never encountered this problem. They have no idea how to fix it. And they have no idea how to further debug it.. What steps does the customer wish?
A publicity-adverse colleague of mine wrote this amazing program. I wanted to publish it not so much for what it specifically does, but as well for the programming techniques it uses. I personally find i relatively hard to look up concepts when using TCL for an F5 iRule.
# RULE_INIT is executed once every time the iRule is saved or on reboot. So it is ideal for persistent data that is shared accross all sessions.
# In our case it is used to define a template with some variables that are later substituted
when RULE_INIT {
# "static" variables in iRules are global and read only. Unlike regular TCL global variables they are CMP-friendly, that means they don't break the F5 clustered multi-processing mechanism. They exist in memory once per CMP instance. Unlike regular variables that exist once per session / iRule execution. Read more about it here: https://devcentral.f5.com/s/articles/getting-started-with-irules-variables-20403
#
# One thing to be careful about is not to define the same static variable twice in multiple iRules. As they are global, the last iRule saved overwrites any previous values.
# Originally the idea was to load an iFile here. That's also the main reason to even use RULE_INIT and static variables. The reasoning was (and I don't even know if this is true), that loading the iFile into memory once would have to be more efficient than to do it every time the iRule is executed. However, it is entirely possible that F5 already optimized iFiles in a way that loads them into memory automatically at opportune times, so this might be completely unnecessary.
# Either way, as you can tell, in the end I didn't even use iFiles. The reason for that is simply visibility. iFiles can't be easily viewed from the web UI, so it would be quite inconvenient to work with.
# The template idea and the RULE_INIT event stayed, even though it doesn't really serve a purpose, except maybe visually separating the templates from the rest of the code.
#
# As for the actual content of the variable: First thing to note is the use of {} to escape the entire string. Works perfectly, even though the string itself contains braces. TCL magic.
# The rest is just the actual PAC file, with strategically placed TCL variables in the form of $name (this becomes important later)
set static::pacfiletemplate {function FindProxyForURL(url, host)
{
var globalbypass = "$globalbypass";
var localbypass = "$localbypass";
var ceglobalbypass = "$ceglobalbypass";
var zpaglobalbypass = "$zpaglobalbypass";
var zscalerbypassexception = "$zscalerbypassexception";
var bypass = globalbypass.split(";").concat(localbypass.split(";"));
var cebypass = ceglobalbypass.split(";");
var zscalerbypass = zpaglobalbypass.split(";");
var zpaexception = zscalerbypassexception.split(";");
if(isPlainHostName(host)) {
return "DIRECT";
}
for (var i = 0; i < zpaexception.length; ++i){
if (shExpMatch(host, zpaexception[i])) {
return "PROXY $clientproxy";
}
}
for (var i = 0; i < zscalerbypass.length; ++i){
if (shExpMatch(host, zscalerbypass[i])) {
return "DIRECT";
}
}
for (var i = 0; i < bypass.length; ++i){
if (shExpMatch(host, bypass[i])) {
return "DIRECT";
}
}
for (var i = 0; i < cebypass.length; ++i) {
if (shExpMatch(host, cebypass[i])) {
return "PROXY $ceproxy";
}
}
return "PROXY $clientproxy";
}
}
set static::forwardingpactemplate {function FindProxyForURL(url, host)
{
var forwardinglist = "$forwardinglist";
var forwarding = forwardinglist.split(";");
for (var i = 0; i < forwarding.length; ++i){
if (shExpMatch(host, forwarding[i])) {
return "PROXY $clientproxy";
}
}
return "DIRECT";
}
}
}
# Now for the actual code (executed every time a user accesses the vserver)
when HTTP_REQUEST {
# The request URI can of course be used to differentiate between multiple PAC files or to restrict access.
# So can basically any other request attribute. Client IP, host, etc.
if {[HTTP::uri] eq "/proxy.pac"} {
# Here we set variables with the exact same name as used in the template above.
# In our case the values come from a data group, but of course they could also be defined
# directly in this iRule. Using data groups makes the code a bit more compact and it
# limits the amount of times anyone needs to edit the iRule (potentially making a mistake)
# for simple changes like adding a host to the bypass list
# These variables are all set unconditionally. Of course it is possible to set them based
# on for example client IP (to give different bypass lists or proxy entries to different groups of users)
set globalbypass [ class lookup globalbypass ProxyBypassLists ]
set localbypass [ class lookup localbypassEU ProxyBypassLists ]
set ceglobalbypass [ class lookup ceglobalbypass ProxyBypassLists ]
set zpaglobalbypass [ class lookup zpaglobalbypass ProxyBypassLists ]
set zscalerbypassexception [ class lookup zscalerbypassexception ProxyBypassLists ]
set ceproxy [ class lookup ceproxyEU ProxyHosts ]
# Here's a bit of conditionals, setting the proxy variable based on which virtual server the
# iRule is currently executed from (makes sense only if the same iRule is attached to multiple
# vservers of course)
if {[virtual name] eq "/Common/proxy_pac_http_90_vserver"} {
set clientproxy [ class lookup formauthproxyEU ProxyHosts ]
} elseif {[virtual name] eq "/Common/testproxy_pac_http_81_vserver"} {
set clientproxy [ class lookup testproxyEU ProxyHosts]
} elseif {[virtual name] eq "/Common/proxy_pac_http_O365_vserver"} {
set clientproxy [ class lookup ceproxyEU ProxyHosts]
} else {
set clientproxy [ class lookup clientproxyEU ProxyHosts ]
}
# Now this is the actual magic. As noted above we have now set TCL variables named for example
# $globalbypass and our template includes the string "$globalbypass"
# What we want to do next is substitute the variable name in the template with the variable values
# from the code.
# "subst" does exactly that. It performs one level of TCL execution. Think of "eval" in basically
# any language. It takes a string and executes it as code.
# Except for "subst" there are two in this context very useful parameters: -nocommands and -nobackslashes.
# Those prevent it from executing commands (like if there was a ping or rm or ssh or find or anything
# in the string being subst'd it wouldn't actually try to execute those commands) and from normalizing
# backslashes (we don't have any in our PAC file, but if we did, it would still work).
# So what is left that it DOES do? Substituting variables! Exactly what we want and nothing else.
# Now since the static variable is read only, we can't do this substitution on the template itself.
# And if we could it wouldn't be a good idea, because it is shared accross all sessions. So assuming
# there are multiple versions of the PAC file with different proxies or bypass lists, we would
# constantly overwrite them with each other.
# The solution is simply to save the output of the subst in a new local variable that exists in
# session context only.
# So from a memory point of view the static/global template doesn't really gain us anything.
# In the end we have the template in memory once per CMP and then a substituted copy of the template
# once per session. So as noted earlier, could've probably just removed the entire RULE_INIT block,
# set the template in session context (HTTP_REQUEST event) and get the same result,
# maybe even slightly more efficient.
set pacfile [subst -nocommands -nobackslashes $static::pacfiletemplate]
# All that's left to do is actually respond to the client. Simple stuff.
HTTP::respond 200 content $pacfile "Content-Type" "application/x-ns-proxy-autoconfig" "Cache-Control" "private,no-cache,no-store,max-age=0"
# In this example we have two different PAC files with different templates on different URLs
# Other iRules we use have more differentiation based on client IP. In theory we could have one big iRule
# with all the PAC files in the world and it would still scale very well (just a few more if/else or switch cases)
} elseif { [HTTP::uri] eq "/forwarding.pac" } {
set clientproxy [ class lookup clientproxyEU ProxyHosts]
set forwardinglist [ class lookup forwardinglist ProxyBypassLists ]
set forwardingpac [subst -nocommands -nobackslashes $static::forwardingpactemplate]
HTTP::respond 200 content $forwardingpac "Content-Type" "application/x-ns-proxy-autoconfig" "Cache-Control" "private,no-cache,no-store,max-age=0"
} else {
# If someone tries to access a different path, give them a 404 and the right URL
HTTP::respond 404 content "Please try http://webproxy.drjohns.com/proxy.pac" "Content-Type" "text/plain" "Cache-Control" "private,no-cache,no-store,max-age=0"
}
}
While watching Mad Men last night on IMDB we saw a terribly annoying audio delay – probably about three seconds after the video. Then it cuts to the commercials and they were perfectly in sync. Then back to the show – still a 2 – 3 -second delay. Is it the brand of TV? We use a Firestick plus a Samsung LCD TV.
The solution
Well, my first inclination was to look for audio settings on either the TV or the Amazon Firestick which might be set to delay audio. I had such a setting on an old sound system, though I think it was only for a sub-second delay. But, there are no such settings on either Firestick or Samsung TV, so that’s not it.
An Internet search proved not too useful.
What I eventually realized – the in-sync commercials was a hint – is that I could rewind the show a tiny bit, and that might pop it back into sync. And…it did!
How it happens
I think it happens when I pause a show one night, to return to it a later time. It remembers where I was, which is great. But it sometimes gets out of sync this way. I have seen this with Netflix and IMDB. The common element seems to be the Firestick.
#!/bin/sh
# DrJ 1/2021
# call this from cron once a day to refesh random slideshow once a day
NUMFOLDERS=20
DEBUG=1
HOME=/home/pi
RANFILE=$HOME/random.list
REANFILE=$HOME/rean.list
DISPLAYFOLDER=$HOME/Pictures
DISPLAYFOLDERTMP=$HOME/Picturestmp
EXIFTMP=$HOME/EXIFtmp
EXIF=$HOME/EXIF
TXTDIR=$HOME/picstxt
MSHOW=$HOME/mediashow
MSHOW2=$HOME/mediashowtmp2
MSHOW3=$HOME/mediashowtmp3
SLEEPINTERVAL=1
STARTFOLDER="MaryDocs/Pictures and videos"
echo "Starting master process at "`date`
cd $HOME
rm -rf $DISPLAYFOLDERTMP
mkdir $DISPLAYFOLDERTMP
#listing of all Google drive files starting from the picture root
# this takes a few minutes so we may want to skip for debugging
if [ "$1" = "skip" ]; then
if [ $DEBUG -eq 1 ]; then echo SKIP Listing all files from Google drive; fi
else
if [ $DEBUG -eq 1 ]; then echo Listing all files from Google drive; fi
rclone ls remote:"$STARTFOLDER" > files
# filter down to only jpegs, lose the docs folders and the tiny JPEGs
if [ $DEBUG -eq 1 ]; then echo Picking out the JPEGs and losing the small images; fi
egrep '\.[jJ][pP][eE]?[gG]$' files |awk '$1 > 11000 {$1=""; print substr($0,2)}'|grep -i -v /docs/ > jpegs.list
fi
# check if we got anything. If our Internt dropped there may have been a problem, for instance
flines=`cat files|wc -l`
if [ $flines -lt 60 ]; then
echo "rclone did not produce enough files. Check your Internet setup and rclone configuration."
echo Only $flines files in the file listing - not enough - so pausing 60 seconds and starting over... at `date`
# start a new job and kill ourselves!
nohup $HOME/master3.sh > master.log 2>&1 &
exit
fi
# throw NUMFOLDERS or so random numbers for picture selection, select triplets of photos by putting
# names into a file
if [ $DEBUG -eq 1 ]; then echo "\nGenerate random filename triplets"; fi
./random-files3.pl -f $NUMFOLDERS -j jpegs.list -r $RANFILE
# copy over these 60 jpegs
if [ $DEBUG -eq 1 ]; then echo "\nCopy over these random files"; fi
cat $RANFILE|while read line; do
if [ $DEBUG -eq 1 ]; then echo filepath is $line; fi
rclone copy remote:"${STARTFOLDER}/$line" $DISPLAYFOLDERTMP
sleep $SLEEPINTERVAL
done
# do a re-analysis to push pictures further apart in time
if [ $DEBUG -eq 1 ]; then echo "\nRe-analyzing pictures for their timestamps"; fi
cd $DISPLAYFOLDERTMP; $HOME/reanalyze.pl
# copy over just the new pictures that we determined were needed
if [ $DEBUG -eq 1 ]; then echo "\nCopy over the needed replacement files"; fi
cat $REANFILE|while read line; do
if [ $DEBUG -eq 1 ]; then echo filepath is $line; fi
rclone copy remote:"${STARTFOLDER}/$line" $DISPLAYFOLDERTMP
sleep $SLEEPINTERVAL
done
# QC: toss out the pics which are not actually JPEGs
if [ $DEBUG -eq 1 ]; then echo "\nQC: Toss out the pics which are not actually JPEGs"; fi
cd $DISPLAYFOLDERTMP; ../QC.pl
# save EXIF metadata for later
if [ $DEBUG -eq 1 ]; then echo "\nSave EXIF metadata for later"; fi
cd $DISPLAYFOLDERTMP; $HOME/get-all-EXIF.sh
rm -rf $EXIF;mv $EXIFTMP $EXIF
# analyze EXIF info to extract most interesting things
if [ $DEBUG -eq 1 ]; then echo "\nAnalyze EXIF data"; fi
rm -rf $TXTDIR; $HOME/analyze.sh
# rotate pics as needed
if [ $DEBUG -eq 1 ]; then echo "\nRotate the pics which need it"; fi
cd $DISPLAYFOLDERTMP; $HOME/rotate-as-needed.sh
# resize pics
if [ $DEBUG -eq 1 ]; then echo "\nSize all pics to the display size"; fi
$HOME/resize.sh
# create text info + images
if [ $DEBUG -eq 1 ]; then echo "\nEmbed pic info"; fi
$HOME/embedpicinfo.sh
cd ~
# kill any old slideshow
if [ $DEBUG -eq 1 ]; then echo Killing old fbi slideshow; fi
sudo pkill -9 -f fbi
pkill -9 -f m3.pl
# remove old pics
if [ $DEBUG -eq 1 ]; then echo Removing old pictures; fi
rm -rf $DISPLAYFOLDER
mv $DISPLAYFOLDERTMP $DISPLAYFOLDER
cp $MSHOW3 $MSHOW
touch refresh
#run looping fbi slideshow on these pictures
if [ $DEBUG -eq 1 ]; then echo Start "\nfbi slideshow in background"; fi
cd $DISPLAYFOLDER ; nohup ~/m3.pl >> ~/m3.log 2>&1 &
if [ $DEBUG -eq 1 ]; then echo "And now it is "`date`; fi
#!/usr/bin/perl
use Getopt::Std;
my %opt=();
#
# assumption is that we are runnin this from a directory containing pictures
$tier1 = 100; $tier2 = 200; $tier3 = 300; # secs
$DEBUG = 1;
$HOME = "/home/pi";
# pics are here
$pNames = "$HOME/reanpicnames";
$ranfile = "$HOME/random.list";
$reanfile = "$HOME/rean.list";
$origfile = "$HOME/jpegs.list";
$mshowt = "$HOME/mediashowtmp";
$mshow2 = "$HOME/mediashowtmp2";
open(REAN,">$reanfile") || die "Cannot open reanalyze file $reanfile!!\n";
$ms = `cat $mshowt`;
print "Original media show: $ms\n" if $DEBUG;
@lines = split('\0',$ms);
$Pdate = $Phr = $Pmin = $Psec = 0;
$diff = 9999;
for($i=0;$i<@lines;$i++){
$date = 0;
$secs = $ymd = 0;
$_ = $lines[$i];
$file = $_;
# ignore pictures with names like 20130820_180050.jpg
next if /\d{8}_\d{4}/;
open(ANAL,"$HOME/getinfo.py \"$file\"|") || die "Cannot open file: $file!!\n";
print "filename: $file\n" if $DEBUG;
while(<ANAL>){
#extract date and time from remaining pictures, if possible
# # DateTimeOriginal = 2018:08:18 20:16:47
# print STDERR "DATE: $_" if $DEBUG;
if (/date/i && $date++ < 1) {
print "date match in getinfo.pyoutput: $_" if $DEBUG;
($ymd,$hr,$min,$sec) = /(\d{4}:\d\d:\d\d) (\d\d):(\d\d):(\d\d)/;
$secs = 3600*$hr + 60*$min + $sec;
print "file,secs,ymd,i: $file,$secs,$ymd,$i\n" if $DEBUG;
$YMD[$i] = $ymd;
$SECS[$i] = $secs;
}
} # end loop over analysis of this pic
} # end loop over all files
# now go over that
$oldfolder = 0;
for($i=1;$i<@lines;$i++){
$folder = int($i/3) + 1;
next unless $folder != $oldfolder;
print "analyzing results. folder no. $folder\n" if $DEBUG;
# analyze pics in triplets
# center pic
$j = ($folder - 1)*3 + 1;
for ($o=-1;$o<2;$o+=2){
$k=$j+$o;
print "j,k,o: $j,$k,$o\n" if $DEBUG;
next unless $SECS[$j] > 0 && $YMD[$j] == $YMD[$k] && $YMD[$j] > 0;
print "We have non-0 dates we're dealing with\n" if $DEBUG;
$file = $lines[$k];
chomp($file);
$diff = abs($SECS[$j] - $SECS[$k]);
print "diff: $diff\n" if $DEBUG;
next unless $diff < $tier3;
# the closer the files are together the more we push away
$bump = 1 if $diff < $tier3;
$bump = 2 if $diff < $tier2;
$bump = 3 if $diff < $tier1;
# get full filepath
$filepath = `grep \"$file\" $ranfile`;
chomp($filepath);
# now use that to search within the jpegs file listing
$prog = $o < 0 ? "head" : "tail";
$newfilepath = `grep -C$bump "$filepath" $origfile|$prog -1`;
($newfile) = $newfilepath =~ /([^\/]+)$/;
chomp($newfile);
print "file,filepath,newfile,newfilepath,bump: $file,$filepath,$newfile,$newfilepath,$bump\n" if $DEBUG;
print REAN $newfilepath;
# we'll get the new pictures over in a separate step to keep this more atomic
$ms =~ s/$file/$newfile/;
}
$oldfolder = $folder;
} # end loop over pics
# print out new mediashow pics in order
print "Printing new mediashow: $ms\n" if $DEBUG;
open(MS,">$mshow2") || die "Cannot open mediashow $mshow2!!\n";
print MS $ms;
close(MS)
#!/usr/bin/perl
# kick out the non-JPEG files - sometimes they creep in
$DEBUG = 1;
$HOME = "/home/pi";
$mshow2 = "$HOME/mediashowtmp2";
$mshow3 = "$HOME/mediashowtmp3";
$ms = `cat $mshow2`;
@pics = split('\0',$ms);
foreach $file (@pics) {
print "file is $file\n" if $DEBUG;
#DSC00185.JPG: JPEG image data, JFIF standard 1.01...
$res = `file "$file"|cut -d: -f2`;
if ($res =~ /JPEG/i){
print "This file is indeed a JPEG image\n" if $DEBUG;
} else {
print "Not a JPEG image! We have to remove this file form the mediashow\n" if $DEBUG;
$ms =~ s/$file\0//;
}
}
# print out new mediashow pics in order
print "Printing new mediashow: $ms\n" if $DEBUG;
open(MS,">$mshow3") || die "Cannot open mediashow $mshow3!!\n";
print MS $ms;
close(MS);
#!/bin/sh
# DrJ 1/2021
# preserve EXIF info of all the images because our rotate step removes it
# and we will use it in subsequent steps
# assumption is that our current directory is the one where we want to read files
EXIFTMP=~/EXIFtmp
mkdir $EXIFTMP
ls -1|while read line; do
echo file is "$line"
~/getinfo.py "$line" > $EXIFTMP/"$line"
done
#!/bin/sh
# DrJ 1/2021
# try to extract date, file and folder name and even GPS info, create jpegs with info
# for each image
# assumption is that are current directory is the one where we want to alter files
HOME=/home/pi
TXTDIR=$HOME/picstxt
# it's assumed EXIF info for each pic has already been extracted and put into EXIF diretory
EXIF=$HOME/EXIF
mkdir $TXTDIR
cd $EXIF
ls -1|while read line; do
echo file is "$line"
echo -n "$line"|../analyzeDate.pl > "$TXTDIR/${line}"
echo -n "$line"|../analyzeGPS.pl >> "$TXTDIR/${line}"
done
#!/bin/sh
# DrJ 12/2020
# some of our downloaded files will be sideways, and fbi doesn't auto-rotate them as far as I know
# assumption is that our current directory is the one where we want to alter files
ls -1|while read line; do
echo file is "$line"
o=`~/getinfo.py "$line"|grep -ai orientation|awk '{print $NF}'`
echo orientation is $o
if [ "$o" -eq "6" ]; then
echo "90 clockwise is needed, o is $o"
# rotate and move it
~/rotate.py -90 "$line"
mv rot_"$line" "$line"
elif [ "$o" -eq "8" ]; then
echo "90 counterclock is needed, o is $o"
# rotate and move it
~/rotate.py 90 "$line"
mv rot_"$line" "$line"
elif [ "$o" -eq "3" ]; then
echo "180 rot is needed, o is $o"
# rotate and move it
~/rotate.py 180 "$line"
mv rot_"$line" "$line"
fi
done
#!/bin/sh
# DrJ 2/2021
# To combat the RPi's inherent sluggish performance we'll downsize the pictures in advance to save fbi the effort
#
# on the pidisplay fbset gives:
#mode "800x480"
# geometry 800 480 800 480 32
# timings 0 0 0 0 0 0 0
# rgba 8/16,8/8,8/0,8/24
#endmode
displaywidth=`fbset|grep geometry|awk '{print $2}'`
displayheight=`fbset|grep geometry|awk '{print $3}'`
ls -1|while read line; do
echo file is "$line"
# this will create a new image with same name prepended with txt_
~/embedpicinfo.py $displaywidth $displayheight "$line"
done
#!/usr/bin/python3
# call with two arguments: degrees-to-rotate and filename
import PIL, os
import sys
from PIL import Image
# first do: pip3 install piexif
import piexif
degrees = int(sys.argv[1])
pic = sys.argv[2]
picture= Image.open(pic)
# see https://github.com/hMatoba/Piexif for piexif writeup
# this method of preserving EXIF info does not always work, and
# causes script to crash when it fails!
##exif_dict = piexif.load(picture.info["exif"])
##exif_bytes = piexif.dump(exif_dict)
## both rotate and preserve EXIF data
##picture.rotate(degrees,expand=True).save("rot_" + pic,"jpeg", exif=exif_bytes)
# rotate (which will blow away EXIF info, sorry...)
picture.rotate(degrees,expand=True).save("rot_" + pic,"jpeg")
#!/usr/bin/python3
# DrJ 2/2021
import PIL, os
import sys
from PIL import Image
# somewhat inspired by http://www.riisen.dk/dop/pil.html
# arguments:
# <width> <height> file
# with and height should be provided as values in pixels
# image file should be provided as argument.
# A pidisplay is 800x480
displaywidth = int(sys.argv[1])
displayheight = int(sys.argv[2])
smallscreen = 801
imageFile = sys.argv[3]
im1 = Image.open(imageFile)
narrowmax = .76
blowupfactor = 1.1
# take less from the top than the bottom
topshare = .3
bottomshare = 1.0 - topshare
# for DrJ debugging
DEBUG = True
if DEBUG:
print("display width and height: ",displaywidth,displayheight)
def imgResize(im):
width = im.size[0]
height = im.size[1]
if DEBUG:
print("image width and height: ",width,height)
# If the aspect ratio is wider than the display screen's aspect ratio,
# constrain the width to the display's full width
if width/float(height) > float(displaywidth)/float(displayheight):
if DEBUG:
print("In section width contrained to full width code section")
widthn = displaywidth
heightn = int(height*float(displaywidth)/width)
im5 = im.resize((widthn, heightn), Image.ANTIALIAS) # best down-sizing filter
else:
heightn = displayheight
widthn = int(width*float(displayheight)/height)
if width/float(height) < narrowmax and displaywidth < smallscreen:
# if width is narrow we're losing too much by using the whole picture.
# Blow it up by blowupfactor% if display is small, and crop most of it from the bottom
heightn = int(displayheight*blowupfactor)
widthn = int(width*float(heightn)/height)
im4 = im.resize((widthn, heightn), Image.ANTIALIAS) # best down-sizing filter
top = int(displayheight*(blowupfactor - 1)*topshare)
bottom = int(heightn - displayheight*(blowupfactor - 1)*bottomshare)
if DEBUG:
print("heightn,top,widthn,bottom: ",heightn,top,widthn,bottom)
im5 = im4.crop((0,top,widthn,bottom))
else:
im5 = im.resize((widthn, heightn), Image.ANTIALIAS) # best down-sizing filter
im5.save("resize_" + imageFile)
imgResize(im1)
#!/usr/bin/perl
# show the pics ; rotate the screen as needed
# for now, assume the display is in a neutral
# orientation at the start
use Time::HiRes qw(usleep);
$DEBUG = 1;
$delay = 6; # seconds between pics
###$delay = 4; # for testing
$mdelay = 200; # milliseconds
$mshow = "$ENV{HOME}/mediashow";
$pNames = "$ENV{HOME}/pNames";
# pics are here
$picsDir = "$ENV{HOME}/Pictures";
$refreshFile = "$ENV{HOME}/refresh";
chdir($picsDir);
$cn = `ls -1|wc -l`;
chomp($cn);
print "$cn files\n" if $DEBUG;
# throw up a first picture - all black. Trick to make black bckgrd permanent
system("sudo fbi -a --noverbose -T 1 $ENV{HOME}/black.jpg");
# see if this is a new batch of pictures
$refresh = (stat($refreshFile))[9];
$now = time();
$diff = $now - $refresh;
print "refresh,now,diff: $refresh, $now, $diff\n" if $DEBUG;
if ($diff < 100){
system("sudo fbi -a --noverbose -T 1 $ENV{HOME}/newslideshowintro.jpg");
sleep(25);
}
system("sudo fbi -a --noverbose -T 1 $ENV{HOME}/black.jpg");
system("sleep 1; sudo killall fbi");
# start infinitely looping fbi slideshow
for (;;) {
# then start slide show
# shell echo cannot work with null character so we need to use a file to store it
system("sudo xargs -a $mshow -0 fbi --noverbose -1 -T 1 -t $delay ");
###system("sudo xargs -a $mshow -0 fbi -a -1 -T 1 -t $delay "); # for testing
# fbi runs in background, then exits, so we need to monitor if it's still alive
for(;;) {
open(MON,"ps -ef|grep fbi|grep -v grep|") || die "Cannot launch ps -ef!!\n";
$match = <MON>;
if ($match) {
print "got fbi match\n" if $DEBUG > 1;
} else {
print "no fbi match\n" if $DEBUG;
# fbi not found
last;
}
close(MON);
print "usleeping, noexist is $noexit\n" if $DEBUG > 1;
usleep($mdelay);
} # end loop testing if fbi has exited
} # close of infinite loop
Optional script
mshowtmp.pl (revision not yet reflected in the tar file)
# display some ip info first
@reboot sleep 15;ip a|grep wlan|sudo tee -a /dev/console > /dev/null
@reboot sleep 22; ./m3.pl >> m3.log 2>&1
# reboot if we can’t reach the Internet
19 5 */2 * * curl google.com > /dev/null 2>&1 || sudo reboot
26 5 */2 * * ./master3.sh >> master.log 2>&1
That will refresh the slideshow every two days, which we found is a good interval for our lifestyle – some days you don’t get around to viewing them. If you want to refresh every day just change ‘*/2″ to ‘*’.
And… that’s it!
Reminder
Don’t forget to make all these files executable. Something like:
$ chmod +x *.pl *.py *.sh
should do it.
My equipment
RPi 3 running Raspbian Lite, OS version “Bullseye,” though older versions also work well, just accommodating the appropriate packages which have changed over time.
Pi Display. The Pi Display resolution is 800×480, so pretty small.
HDMI display such as a TV as alternate to a Pi Display. This does work! I just tested this in Jan, 2022. My Sony TV display resolution is 1920 x 1080.
Pre-install
There are a few things you’ll need (accurate statement as of OS Bullseye, Jan 2022) such as these system packages: fbi, file, rclone, and these python modules: pip, Pillow, and piexif. That’s mostly described in my previous post so I won’t repeat it here. Basically the system package you install with apt-get. After installing pip you use it to install Pillow and piexif.
Getting started
To see how badly things are going for you (hey, I like to be cautiously pessimistic) after you’ve created all these files and have installed rclone, do a
$ ./master3.sh
If you have your rclone file listing (which takes a long time) and want to focusing on debugging the rest of it, do a
$ ./master3.sh skip
Discussion
In this version of Raspberry Pi photo frame I’ve made more effort to force time separation between the randomly selected photos. But, that’s not all. I blow up pictures taken in a narrow (portrait) mode (see next paragraph). And I do some fancy analysis to determine filename, folder, date, time and even location of the pictures. And there’s more. I create an alternate version of each photo which embeds this info at the bottom – in anticipation of my even more fancy remote-controlled slideshow! I am afraid to overwrite what I have previously posted because that by itself is a complete solution and works quite well on its own. So this can be considered worthy of folks looking for a little more challenge to get better results.
The fancyresize.py script is designed around my small PiDisplay which has a horizontal resolution of only 800 pixels. It blows up a narrow, portrait-format picture only if the detected display has a horizontal resolution of no more than 800 pixels. It blows the picture up by 10%, chops off 3% from the top, 7% from the bottom, because that yields optimal results in my experience. If you like that approach but are using a larger HDMI display, you could edit the “801” in that file to make it a larger number (bigger than your display, like 5000).
Show pictures with embedded info
This process is not streamlined. But it can be cool to do it by hand. You could follow these steps.
$ ./mshowtmp.pl; mv mediashowtmp2 mediashow
If you wait the whole cycle the next time around it should display the pictures with the embedded info at the bottom. If you’re impatient, do this:
I’m seeing this while transferring the pictures. Guess I’ll have to slow down the transfer. Not sure. Still figuring this out.
Fun Fact
You know how those old digital cameras created files prefixed with DSC, like DSC00102.JPG? If you read the JPEG spec, which is a pretty dense document, you learn that DSC stands for Digital Still Camera.
Concept for tossing out pictures of documents
We sometimes take pictures of documents, or computer screens, or a slide at a presentation, or a historical marker. They don’t make for compelling slideshow material. Well, the historical markers are debatable since they have character. Anyway, I am looking at using an old open source program called tesseract to do OCR (optical character recognition) on all the photos to help identify those containing a lot of words so they can be excluded. I’ll include that if I determine it to be a good approach.
Installing a searchable dictionary on Raspberry Pi
To install a word dictionary that you can do simple searches against on an RPi, try:
$ sudo apt-get install wamerican
or maybe
$ sudo apt-get install wamerican-huge
Those will produce simple wordlists, not actual dictionaries with definitions as you might have expected. They go into /usr/share/dict, e.g., usr/share/dict/american-english-huge.
The dict program is quite nice. apt-get install dict. Then you run it like this
$ dict neume
and it shoots back definitions and cites sources for those definitions. The drawback for my purposes is that it uses your Internet connection and I’m trying to build a photo frame that doesn’t rely too much on the Internet after the photos themselves are fetched.
git clone https://github.com/thortex/rpi3-tesseract.git cd rpi3-tesseract cd release ./install_requires_related2leptonica.sh ./install_requires_related2tesseract.sh ./install_tesseract.sh
But you’re gonna need git first:
$ sudo apt-get install git
RPi lost Wifi
This could be a whole separate post. In the course of my hard work my RPi just would not acquire an IP address on wlan0.
Here’s a great command to see all the SSIDs it knows about:
$ sudo iwlist wlan0 scan > scan.log
Then you can inspect scan.log in an editor. Turns out the one SSID it needed wasn’t in the list. Turns out I had reserved a DHCP entry for it in my router. My router was simply not cooperating, it seems – the RPi wasn’t doing anything wrong. I was almost ready to re-install the whole thing and waste hours… My router is an older model Linksys WRT1200AC. I removed the DHCP reservation on the router, then did a
$ sudo service networking stop; sudo service networking start
on the RPi, and…all was good! Its assigned IP won’t change that often, I can always check the router to see what it is. The management software with the Linksys is quite good.
I’m still having IP issues. So I added an additional crontab entry to display IP info of the WiFi adapter for a few seconds before the slideshow kicks in. This will tell me if it at least has an IP (which it often doesn’t)! It’s a pretty clever idea if I say so myself – the way I managed to do it that is. It’s not in the tar file.
RPi partially blown up
I’ve been running the photo frame for about two years now. All the residents love to see when the pictures refresh what the new slideshow brings. But in all the work I’ve done here and there I’ve partially blown up the RPi. Symptoms: running curl produces a segmentation fault; running crontab -e produces crontab: “/usr/bin/sensible-editor” exited with status 2; and then there’s the fact I lose my IP after a few days. I bet there’s a lot else that’s wrong too, but the sldieshow stuff keeps chugging along, amazingly. I’m too unmotivated (lazy) to fix all these problems, except the IP thing. That prevents slideshow refreshes. So I’ve decided to script a reboot command to run before the slideshow refresh. The resulting conditional reboot is now incorporated into the crontab entries shown earlier.
And by the way, I did fix this problem by re-imaging the micro SD card. That brought a new problem which is that the display blanked out afew only a few seconds. I bought a new display (turns out I didn’t need to), still had the problem, then figured out how to fix it. I wrote up the fix in this post.
Conclusion
A more advanced treatment of photos is shown in this post than I have done previously. It is fairly robust and will withstand quite a few user errors in my experience. The end result will be an interesting display of your photos, randomly selected but in small groupings.
m3.pl refers to a black.jpg and a newslideshowintro.jpg file. It’s not a disaster to not have those, but the overall experience will be slightly better. Here’s black.jpg:
And the beautiful newslideshowintro.jpg I created is at the top of this blog post.
Tesseract, an surprisingly old and surprisingly good OCR open-source OCR program, is basically impossible to compile for RPi. Fortunately, someone has done it for us. This page has the instructions: https://github.com/thortex/rpi3-tesseract
I am assembling a lot of different ideas I have to do some more cool things than was possible with my original Raspberry Pi photo frame effort although that contained a lot of original ideas as well.
Skillset
Intermediate or better linux skills required. Beginners/newbies: please do not attempt as you will encounter insurmountable problems and be left in a trail of tears. I don’t have time to help.
I will be using this remote control which I can attest is indeed compatible with the RPi:
But it has no CTRL key! I can’t survive without that. It seems aimed at media consumers. I guess those types never need that key. At this point in time my idea is to use the arrow keys + Enter button as a familiar way to navigate around a simple menu I plan to create, plus perhaps the menu key. the pre-defined keys fbi has created for navigation are simply not intuitive, nor are they adequate for the tasks I have in mind.
Rii key
Value
up arrow
103
down arrow
108
right arrow
106
left arrow
105
Menu
127
OK
28
Some interesting keys and their values when monitored
In my basic photo frame approach I had a simpler rotate image python script. This python program below rotates pictures by the specified amount and preserves the EXIF tags. It doesn’t update the orientation tag after rotating because for the fbi display program I use that doesn’t matter. I call it rotate.py.
#!/usr/bin/python3
# call with two arguments: degrees-to-rotate and filename
import PIL, os
import sys
from PIL import Image
# first do: pip3 install piexif
import piexif
degrees = int(sys.argv[1])
pic = sys.argv[2]
picture= Image.open(pic)
# see https://github.com/hMatoba/Piexif for piexif writeup
exif_dict = piexif.load(picture.info[“exif”])
exif_bytes = piexif.dump(exif_dict)
# both rotate and preserve EXIF data
picture.rotate(degrees,expand=True).save(“rot_” + pic,”jpeg”, exif=exif_bytes)
As it says in the code you need to install piexif in addition to Pillow:
I set for myself a problem that is much more difficult than anything I’ve tackled. I wanted to post a preliminary solution, but I don’t want to do constant re-writes which are time-consuming. I have a lot of code re-writes to do. I do have a menu system working, and a couple functions implemented to date. But there is so much more to do to even get to an alpha version.
To be continued…
Pie-in-the-sky To-Do list
Use of deep learning AI to toss out images which are poor quality
Use Facebook API to identify people in the images
Commercialization of idea
Smarthome enablement, e.g., hey Alexa, pause that picture and tell me about it, etc
Breaking those pie-in-the-sky ideas down, I believe the RPi is way vastly underpowered to do any serious image analysis; Facebook facial recognition is creepy and an invasion of privacy; commercialization sounds great on paper but in reality is a time sink doing unpleasant things for little or no profit in the end; and lastly Smarthome enablement is actually the most achievable and was my original thinking before I landed on the remote control. I may or may not get back to it one day.
This example will probably get used in my advanced Raspberry Pi photo frame treatment. I use image display software, fbi, which is designed to take key presses in order to take certain actions, such as, advance to the next picture, display information about the current picture, etc. qiv works similarly. But I am running an automated picture display, so no one is around to type into the keyboard. hence the need for software emulation of the physical keyboard. I’ve always believed it must be possible, but never knew how until this week.
I am not a python programmer, but sometimes you gotta use whatever language makes the job easier, and I only know how to do this in python, python3 specifically.
This will probably only make sense for Raspberry Pi owners.
Setup
I believe this will work best if your Raspberry Pi has only a keyboard and not a mouse hooked up because in that case your keyboard ought to be mapped to /dev/input/event0. But it’s easy enough to change that. To see whether your keyboard is /dev/input/event0 or /dev/input/event1 or some other, just cat one of those files and start typing. You should see some junk when you’ve selected the right /dev/input file.
#!/usr/bin/python3
# inject a single key, acting like a software keyboard
# DrJ 12/20
import sys
from evdev import UInput, InputDevice, ecodes as e
from time import sleep
# set DEBUG = True to print out more information
DEBUG = False
sleepTime = 0.001 # units are secs
# dict of name mappings. key is how we like to enter it, value is what is after KEY_ in evdev ecodes
d = {
'1' : '1',
'2' : '2',
'3' : '3',
'4' : '4',
'5' : '5',
'6' : '6',
'7' : '7',
'8' : '8',
'8' : '8',
'9' : '9',
'0' : '0',
'a' : 'A',
'b' : 'B',
'c' : 'C',
'd' : 'D',
'e' : 'E',
'f' : 'F',
'g' : 'G',
'h' : 'H',
'i' : 'I',
'j' : 'J',
'k' : 'K',
'l' : 'L',
'm' : 'M',
'n' : 'N',
'o' : 'O',
'p' : 'P',
'q' : 'Q',
'r' : 'R',
's' : 'S',
't' : 'T',
'u' : 'U',
'v' : 'V',
'w' : 'W',
'x' : 'X',
'y' : 'Y',
'z' : 'Z',
'.' : 'DOT',
',' : 'COMMA',
'/' : 'SLASH',
'E': 'ENTER',
'S': 'RIGHTSHIFT',
'C': 'LEFTCTRL'
}
# https://python-evdev.readthedocs.io/en/latest/tutorial.html
inputchars = sys.argv
ltrs = inputchars[1]
# get rid of program name
keybd = InputDevice("/dev/input/event0")
for ltr in ltrs:
ui = UInput.from_device(keybd, name="keyboard-device")
if DEBUG: print(ltr)
mappedkey = d[ltr]
key = "KEY_" + mappedkey
if DEBUG: print(key)
if DEBUG: print(e.ecodes[key])
ui.write(e.EV_KEY, e.ecodes[key], 1) # KEY_ down
ui.write(e.EV_KEY, e.ecodes[key], 0) # KEY_ up
ui.syn()
sleep(sleepTime)
ui.close()
And it gets called like this:
$ sudo ./keyinject.py my.injected.letters
or
$ sudo ./keyinject.py ./m2.plE
to run the m2.pl script in the current directory and have it behave as though it were launched from a console terminal. The “E” is the ENTER key.
Interesting observations
There really is no such thing as a separate “k” key and “K” key (lower-case versus upper-case). There is only a single key labelled “K” on a keyboard. It’s a physical layer versus logical layer type of thing. The k and K are characters.
In the above program I did some of the keys – the ones I will be needing, plus a few bonus ones. I do need the ENTER key, and I can’t think of a way to convey that to this program, so to send ENTER you would do
$ sudo ./keyinject.py ENTER
But I was able to have these characters represent themselves: . , / so that’s not bad.
Prerequisites
You will need pyhon3 version > 3.5, I think. And the evdev package. I believe you get that with
$ sudo pip3 install evdev
And if you don’t have pip3 you can install that with
$ sudo apt-get install python3-pip
Reading keyboard input
Of course the opposite of simulating key presses is reading what’s been typed from an actual keyboard. That’s possible too with this handy evdev package. The following program is not as polished as the writing program, but it gives you the feel for what to do. I call it evread.py.
#!/usr/bin/python3
# https://python-evdev.readthedocs.io/en/latest/tutorial.html
import asyncio
from evdev import InputDevice, categorize, ecodes
import evdev,re
#/dev/input/event4 MemsArt MA144 RF Controller System Control usb-0000:01:00.0-1.4/input2
#/dev/input/event3 MemsArt MA144 RF Controller Consumer Control usb-0000:01:00.0-1.4/input2
#/dev/input/event2 MemsArt MA144 RF Controller usb-0000:01:00.0-1.4/input1
#/dev/input/event1 MemsArt MA144 RF Controller usb-0000:01:00.0-1.4/input0
#/dev/input/event0 iTalk-02 usb-0000:01:00.0-1.3/input2
devices = [evdev.InputDevice(path) for path in evdev.list_devices()]
#for device in devices:
# print(device.path, device.name, device.phys)
for device in devices:
path = device.path
# we only are interested in 0 or 1
if re.search(r'event[01]',path):
print(path)
if re.search('MemsArt',device.name): Riipath = path
# You can hardcode '/dev/input/event0' if you like and do not have an Rii remote ctrlr
dev = InputDevice(Riipath)
# following line is optional - it takes away the keybd from fbi!
# there is also a dev.ungrab()
dev.grab()
async def helper(dev):
async for ev in dev.async_read_loop():
print(repr(ev))
loop = asyncio.get_event_loop()
loop.run_until_complete(helper(dev))
Note the presence of the dev.grab(). That permits your program to be the exclusive reader of keyboard input, shutting out fbi. It can be commented out if you want to share.
How to interpret the output of evread.py
Say I press and hold the Enter button. I get this output from evread.py.
The 4, 4, longnumber – You always seem to get somethiing like that. Not sure about it.
The 1, 28, 1 – means I’ve pressed the ENTER key. 28 is for ENTER. other keys will have other values assigned.
0, 0, 1 – not really sure. Continuation, or something.
1, 28, 2 – I think I know this. It means I’m continuing to hold down the ENTER button.
1, 28, 0 – I have released the ENTER key
These get generated pretty frequently, perhaps a few a second.
Conclusion
We have created an example program, mostly for Raspberry Pi, though easily adapted to other linux environments, that injects keyboard presses, via a python3 program, as though those keys had been typed by someone using the physical keyboard such that, graphics programs which rely on this, such as fbi or qiv (and probably others – vlc?), can be controlled through software.
We have also provided a basic python program for reading key presses from the actual keyboard. I plan to use these things for my advanced RPi photo frame project.