Categories
Admin Perl

Elegant code, poor results

Intro
A colleague of mine is an awesome Perl programmer. It may be a dying language but for those of us who know it it can still do some great things in our hands. So this guy, let’s call him Stain, wrote some code to analyze large proxy logs in parallel by using threads. Using threads presumably makes the analysis program run a lot faster since you’re throwing more CPUs at a problem that is CPU bound. It would have taken me years to write but I bet he did it in a couple days because he’s so experienced. I was eager to reap the benefits of his labor but was disappointed with the results I was getting…

The details
What I need the program for is very basic – just looking for a character string in the lnies of a proxy log. In fact my needs are so basic I don’t need a program at all. I can cobble together my analysis with Linux command-line utilities like a

$ gunzip -c proxy_log.gz|grep drjohnstechtalk > /tmp/drjs

And that’s what I was doing before his program came along. when I run his program with eight threads on a server with 16 cores, it should finish a lot faster than my command line, but it wasn’t. So I rolled up my sleeves to look into his code to see what the problem is. Running his code with a single thread showed that it ran much, much slower than my command-line commands. I focused on that problem as the root of the issue.

Here is the essence of his code for the analysis:

#!/usr/bin/perl
# DrJ research labs...
use strict;
use Compress::Zlib;
use threads;
use threads::shared;
use File::Find;
my @searchStrings = ("drjohnstechtalk");
my @matches;
my $lines;
&searchGZip("/tmp/p-02.gz");
#----------------------------------------------------------------------------
# searchGZip
#----------------------------------------------------------------------------
sub searchGZip{
        my $file = shift(@_);
        my $line;
        my $gz;
        my $loglines = 0;
 
        if($gz = gzopen($file, "rb")){
 
                while($gz->gzreadline($line)){
                        $loglines++;
                        if(&matchLine($line)){
                                push(@matches,$line);
                        }
                }
                $lines += $loglines;
                $gz->gzclose;
        }else{
                #&addAppLogMsg("Could not open file $file: $gzerrno\n");
                print "Cannot read file $file\n";
        }
}
 
#----------------------------------------------------------------------------
# matchLine
#----------------------------------------------------------------------------
sub matchLine{
        my $line = shift;
 
        foreach(@searchStrings){
 
                return 1 if $line =~ /$_/
        }
        return 0;
}

He’s using functions I never even head of, like gzopen and gzreadline, apprently from the Compress::Zlib package. Looks elegant, right? Yes. But slow as a dog.

Version 2
So I began to introduce more shell commands into the Perl since shell by itself is faster. In this version I’ve gotten rid of those unfamiliar gzopen and gzreadlikne functions in favor of what I know:

#!/usr/bin/perl
# DrJ research labs...
use strict;
use Compress::Zlib;
use threads;
use threads::shared;
use File::Find;
my @searchStrings = ("drjohnstechtalk");
my @matches;
my $lines;
&searchGZip("/tmp/p-02.gz");
#----------------------------------------------------------------------------
# searchGZip
#----------------------------------------------------------------------------
sub searchGZip{
        my $file = shift(@_);
        my $line;
        my $gz;
        my $loglines = 0;
 
        open(PXY,"gunzip -c $file|");
        while() {
                        $loglines++;
        $line = $_;
                        if(&matchLine($line)){
                                push(@matches,$line);
                        }
                $lines += $loglines;
        }
}
 
#----------------------------------------------------------------------------
# matchLine
#----------------------------------------------------------------------------
sub matchLine{
        my $line = shift;
 
        foreach(@searchStrings){
 
                return 1 if $line =~ /$_/
        }
        return 0;
}

Version 3
Not satisfied with the time, I got rid of the function call for each line in this version.

#!/usr/bin/perl
# DrJ research labs...
use strict;
use Compress::Zlib;
use threads;
use threads::shared;
use File::Find;
my @searchStrings = ("drjohnstechtalk");
my @matches;
my $lines;
&searchGZip("/tmp/p-02.gz");
#----------------------------------------------------------------------------
# searchGZip
#----------------------------------------------------------------------------
sub searchGZip{
        my $file = shift(@_);
        my $line;
        my $gz;
        my $loglines = 0;
 
        open(PXY,"gunzip -c $file|");
        while() {
                        $loglines++;
        $line = $_;
           push(@matches,$line) if $line =~ /$searchStrings[0]/;
                $lines += $loglines;
        }
}

That was indeed still faster, but not as fast as the command line.

Version 4
I wondered if I could do better still by also using the shells built-in match command, grep. That leads to this version:

#!/usr/bin/perl
# DrJ reseach labs...
use strict;
use Compress::Zlib;
use threads;
use threads::shared;
use File::Find;
my @searchStrings = ("drjohnstechtalk");
my @matches;
my $lines;
&searchGZip("/tmp/p-02.gz");
#----------------------------------------------------------------------------
# searchGZip
#----------------------------------------------------------------------------
sub searchGZip{
        my $file = shift(@_);
        my $line;
        my $gz;
        my $loglines = 0;
 
        open(PXY,"gunzip -c $file|grep $searchStrings[0]|");
        while() {
        $line = $_;
           push(@matches,$line);
        }
}

Here’s table with the performance summaries.

Version changes Time (wall clock)
Original 63.2 s
2 gunzip -c instead of gzopen/gzreadline 18.6 s
3 inline instead of function call 14.0 s
4 grep instead of perl match operator 10.8 s

So with these simple improvements the timing on this routine improved from 63.2 s to 10.8 s – a factor of six!!

I incorporated my changes into his original program and now it really is a big help when it runs multi-threaded. I can run eight threads and finish an analysis about six times as quick as searching from the command-line. That’s a significant speed-up which will make us more productive.

Conclusion
Some elegant perl code is taken apart and found to be the cause of a significant slow-down. I show how we rewrote it in a more primitive style that results in a huge performance gain – code that runs six times faster!

Categories
Admin Python Raspberry Pi

Building a Four Monitor Media Show using Raspberry Pis

Intro
This is the paper a student wrote under my guidance.

Building a Four Monitor Media Show using Raspberry Pis

The first page
4-monitor-media-display

Link to full article

References
My write-up concerning our novel use of the Pi Presents program, which has a different emphasis and no pictures.

Categories
Linux Perl

HTTP replay user-agent

Intro
When debugging F5’s Web Application Firewall (AKA ASM) you get all the HTTP request headers in the GUI which you can cut and paste. Sometimes it is not so obvious how to fix a particular SupportID and you don’t want to bother the user to reproduce the error. This is code that allows you to scrape the headers and send them back, assuming you have a Linux server on the Internet. It’s not quite exact – Perl wants to add a TE header and maybe some others. More importantly, F5 only gives you around the first 5 KB of the headers, and often the real headers + data can be much longer than that. But It’s pretty good at preserving the basic headers and the POSTed data, if any.

The details

#!/usr/bin/perl
# DrJ - 6/2015
# take waf Full Request as input from STDIN and spit it out
use LWP;
$DEBUG = 1;
# example:
#POST /cognosbi/cgi-bin/cognos.cgi HTTP/1.1
#Accept: */*
#Accept-Language: en-us
#Referer: https://ag-intelligence.drj.com/cognosbi/pat/rsapp.htm
#soapaction: http://www.ibm.com/xmlns/prod/cognos/reportService/201109/
#Content-Type: text/xml; charset=utf-8
#Accept-Encoding: gzip, deflate
#User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/6.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .
NET4.0C; .NET4.0E; Ion 3.3.0.125; InfoPath.3; MDPA101; OWASMIME/4.0200)
#Host: ag-intelligence.drj.com
#Content-Length: 161667
#DNT: 1
#Connection: Keep-Alive
#Cache-Control: no-cache
#Cookie: c8server=https%3A%2F%2Fag-intelligence.drj.com%3A443%2Fcognosbi%2Fcgi-bin%2Fcognos.cgi; mob_lang=en; TSe36f05=b6a15d33eff8797d98bc75f22d2c85e1f
20fa2f8dda9abed5570682f728766c9684dc6d2c7f603263f299dc03470ed0e5dd2c529d1af78e016f94f63ba1abc528f04a493fe2f9ff9dd4ea3b99a99de5024487a2a9a99de5024487a2a9
a99de5024487a2a9a99de5024487a2a; cc_state="null"; cam_passport=MTsxMDE6ZmVjNDEzYzQtMGVjOC03YjgyLTJlNTgtYjgxNWFhNTE0ZDQxOjMzNjQ5NDA0MDQ7MDszOzA7; CRN=col
umnsPerPage%3D3%26contentLocale%3Den-us%26showHiddenObjects%3Dtrue%26skin%3Dcorporate%26linesPerPage%3D35%26showWelcomePage%3Dtrue%26http%3A%2F%2Fdevelo
per.cognos.com%2Fceba%2Fconstants%2FsystemOptionEnum%23accessibilityFeatures%3Dfalse%26listViewSeparator%3Dbackground%26displayMode%3Dlist%26timeZoneID%
3DEST%26http%3A%2F%2Fdeveloper.cognos.com%2Fceba%2Fconstants%2FbiDirectionalOptionEnum%23biDirectionalFeaturesEnabled%3Dfalse%26format%3DHTML%26automati
cPageRefresh%3D30%26productLocale%3Den%26useAccessibilityFeatures%3Dfalse%26showOptionSummary%3Dtrue%26; cea-ssa=false; usersessionid="AQgAAAA/tHlVAAAA
AoAAACSXrdg581uvb1aFAAAAFJ3FMiV1XFHYYjmBPlmT/mXlCpnFAAAAFlzIt8pvs4xewrB8PBrB1omTHOq"; userCapabilities=f%3Bfdbffc6d%3Be000002f%3Bff077efa%26ARQAAABSdxTI
ldVxR2GI5gT5Zk%2F5l5QqZwtel%2FW97E5sO8sAbwnsxaQA5mLc; cc_session="s_cc:|s_conf:na|s_sch:td|s_hd:sa|s_serv:na|s_disp:na|s_set:|s_dep:na|s_dir:na|s_sms:dd
|s_ct:sa|s_cs:sa|s_so:sa|e_hp:CAMID(*22DRJ*3au*3aagarwap*22)|e_proot:Public*20Folders|prootid:i1E4054857A7F43398F82386788807209|e_mroot:My*20Folders|mr
ootid:i45E22CE51A2D4F9DA668810E67B44CD2|e_mrootpath:CAMID(*22DrJ*3au*3aagarwap*22)*2ffolder*5b*40name*3d*27My*20Folders*27*5d|e_user:John*20Doe|
e_tenantDisplayName:|e_showTenantInfo:true|cl:en-us|dcid:i1E4054857A7F43398F82386788807209|show_logon:true|uig:|ui:|rsuiprofile:all|lch:f|lca:f|ci:f|wri
te:true|eom:0|pp:3364940404|null:|cachestamp:2015-06-04T13:04:34|null:"; BIGipServerag-intelligence.drj.us.secure=1090633994.47873.0000; TS83f7ea=c439d
424e740dc4c1ed042aa7bafa59df20fa2f8dda9abed5570851f1e66c5c4bbcf97c1
 
#<SOAP-ENV:Envelope SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:
SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:bus='http://developer.cognos.com/schemas/bibus/3/' xmlns:rns1='http://developer.cognos.com/schemas/rep
ortService/1'><SOAP-ENV:Header><bus:biBusHeader xsi:type="bus:biBusHeader"><bus:CAM xsi:type="bus:CAM"><authenticityToken xsi:type="xsd:base64Binary">Vj
FZcyLfKb7OMXsKwfDwawdaJkxzqg==</authenticityToken></bus:CAM><bus:CAF xsi:type="bus:CAF"><contextID xsi:type="xsd:string">CAFW00000070Q0FGQTNjMDAwMDAwD
GQUFBQUZKM0ZNaVYxWEZIWVlqbUJQbG1ULW1YbENwbld5amlFRU5ObWJJcTZHaEdLNTdjVWZ0Zy1GWV8zOTgxNzd8cnM_</contextID></bus:CAF><bus:userPreferenceVars SOAP-ENC:arra
yType="bus:userPreferenceVar[]" xsi:type="SOAP-ENC:Array"><item><bus:name xsi:type="xsd:string">productLocale</bus:name><bus:value xsi:type="xsd:string"
>en</bus:value></item><item><bus:name xsi:type="xsd:string">contentLocale</bus:name><bus:value xsi:type="xsd:string">en-us</bus:value></item></bus:userP
referenceVars><bus:dispatcherTransportVars xsi:type="SOAP-ENC:Array" SOAP-ENC:arrayType="bus:dispatcherTransportVar[]"><item xsi:type="bus:dispatcherTra
nsportVar"><name xsi:type="xsd:string">rs</name><value xsi:type="xsd:string">true</value></item></bus:dispatcherTransportVars></bus:biBusHeader></SOAP-E
NV:Header><SOAP-ENV:Body><rns1:update><object xsi:type="bus:report"><searchPath><value xsi:type="xsd:string">/content/folder[@name=&apos;Dashboards and
Portals&apos;]/folder[@name=&apos;DRJOPS (DrJ)&apos;]/folder[@name=&apos;Dashboard Reports&apos;]/report[@name="Sales"]</value></searchPath><specific
ation><value xsi:type="xsd:string" xml:space="preserve">&lt;report xmlns=&quot;http://developer.cognos.com/schemas/report/9.0/&quot; useStyleVersion=&qu
ot;10&quot; expressionLocale=&quot;en-us&quot;&gt;
#&lt;modelPath&gt;/content/folder[@name=&apos;Shared Packages&apos;]/package[@name=&apos;DrJ_Dashboard_Cube&apos;]/model[@name=&apos;model&apos;]&lt;
/modelPath&gt;
#&lt;drillBehavior modelBasedDrillThru=&quot;true&quot;/&gt;
#&lt;layouts&gt;
#&lt;layout&gt;
#&lt;reportPages&gt;
 
#&lt;page name=&quot;DrJ Dashbord&quot;&gt;
#&lt;pageBody&gt;
#&lt;contents&gt;&lt;table&gt;&lt;style&gt;&lt;defaultStyles&gt;&lt;defaultStyle refStyle=&quot;tb&quot;/&gt;&lt;/defaultStyles&gt;&lt;CSS value=&quot;f
ont-family:Arial;font-size:8pt;font-weight:bold&quot;/&gt;&lt;/style&gt;&lt;tableRows&gt;&lt;tableRow&gt;&lt;tableCells&gt;&lt;tableCell colSpan=&quot;1
&quot;&gt;&lt;contents&gt;&lt;table&gt;&lt;style&gt;&lt;defaultStyles&gt;&lt;defaultStyle refStyle=&quot;tb&quot;/&gt;&lt;/defaultStyles&gt;&lt;CSS valu
e=&quot;border-collapse:collapse;width:100%;text-align:center&quot;/&gt;&lt;/style&gt;&lt;tableRows&gt;&lt;tableRow&gt;&lt;tableCells&gt;&lt;tableCell&g
t;&lt;contents&gt;&lt;textItem&gt;&lt;dataSource&gt;&lt;staticValue&gt;In Thousands, Data As of: &lt;/staticValue&gt;&lt;/dataSource&gt;&lt;style&gt;&lt
;CSS value=&quot;color:#0D3F53&quot;/&gt;&lt;/style&gt;&lt;/textItem&gt;&lt;textItem&gt;&lt;dataSource&gt;&lt;reportExpression&gt;CubeDataUpdatedOn ([Gr
ower_Dashboard].[Year])&lt;/reportExpression&gt;&lt;/dataSource&gt;&lt;style&gt;&lt;CSS value=&quot;color:#0D3F53&quot;/&gt;&lt;/style&gt;&lt;/textItem&
gt;&lt;/contents&gt;&lt;style&gt;&lt;CSS value=&quot;text-align:left;vertical-align:middle;font-family:Arial;font-size:9pt;font-weight:bold;border-top-s
tyle:none;border-right-style:none;border-bottom-style:none;border-left-style:none&quot;/&gt;&lt;/style&gt;&lt;/tableCell&gt;&lt;tableCell&gt;&lt;content
s&gt;&lt;image&gt;
#&lt;dataSource&gt;
#&lt;staticValue&gt;../samples/images/Dashboard_Excel.png&lt;/staticValue&gt;
#&lt;/dataSource&gt;
#&lt;conditionalStyles&gt;&lt;conditionalStyleCases refVariable=&quot;For ReportOutput&quot;&gt;&lt;conditionalStyle refVariableValue=&quot;1&quot;&gt;&
lt;CSS value=&quot;display:none&quot;/&gt;&lt;/conditionalStyle&gt;&lt;/conditionalStyleCases&gt;&lt;conditionalStyleDefault/&gt;&lt;/conditionalStyles&
gt;&lt;reportDrills&gt;&lt;reportDrill name=&quot;Excel&quot;&gt;&lt;drillLabel&gt;&lt;dataSource&gt;&lt;staticValue/&gt;&lt;/dataSource&gt;&lt;/drillLa
bel&gt;&lt;drillTarget method=&quot;execute&quot; outputFormat=&quot;spreadsheetML&quot; showInNewWindow=&quot;true&quot;&gt;&lt;reportPath path=&quot;/
content/folder[@name=&apos;Dashboards and Portals&apos;]/folder[@name=&apos;USCROP (Grower)&apos;]/folder[@name=&apos;Dashboard Reports&apos;]/report[@n
ame=&apos;Sales&apos;]&quot;&gt;&lt;XMLAttributes&gt;&lt;XMLAttribute name=&quot;ReportName&quot; value=&quot;Sales&quot; output=&quot;no&quot;/&gt;&lt;
XMLAttribute name=&quot;class&quot; value=&quot;report&quot; output=&quot;no&quot;/&gt;&lt;/XMLAttributes&gt;&lt;/reportPath&gt;&lt;drillLinks&gt;&lt;dr
illLink&gt;&lt;drillTargetContext&gt;&lt;parameterContext parameter=&quot;Select Area&quot;/&gt;&lt;/drillTargetContext&gt;&lt;drillSourceContext&gt;&lt
;parameterContext parameter=&quot;Select Area&quot;/&gt;&lt;/drillSourceContext&gt;&lt;/drillLink&gt;&lt;drillLink&gt;&lt;drillTargetContext&gt;&lt;para
meterContext parameter=&quot;Select DrJIdea Specialist&quot;/&gt;&lt;/drillTargetContext&gt;&lt;drillSourceContext&gt;&lt;parameterContext parameter=
&quot;Select Innovation Specialist&quot;/&gt;&lt;/drillSourceContext&gt;&lt;/drillLink&gt;&lt;drillLink&gt;&lt;drillTargetContext&gt;&lt;parameterContex
t parameter=&quot;AreaTgtBy&quot;/&gt;&lt;/drillTargetContext&gt;&lt;drillSourceContext&gt;&lt;parameterContext parameter=&quot;AreaTgtBy&quot;/&gt;&lt;
/drillSourceContext&gt;&lt;/drillLink&gt;&lt;drillLink&gt;&lt;drillTargetContext&gt;&lt;parameterContext parameter=&quot;ISTgtBy&quot;/&gt;&lt;/drillTar
getContext&gt;&lt;drillSourceContext&gt;&lt;parameterContext parameter=&quot;ISTgtBy&quot;/&gt;&lt;/drillSourceContext&gt;&lt;/drillLink&gt;&lt;drillLin
k&gt;&lt;drillTargetContext&gt;&lt;parameterContext parameter=&quot;Grower Type&quot;/&gt;&lt;/drillTargetContext&gt;&lt;drillSourceContext&gt;&lt;param
eterContext parameter=&quot;Grower Type&quot;/&gt;&lt;/drillSourceContext&gt;&lt;/drillLink&gt;&lt;/drillLinks&gt;&lt;/drillTarget&gt;&lt;/reportDrill&g
t;&lt;/reportDrills&gt;&lt;/image&gt;&lt;image&gt;
#&lt;dataSource&gt;
#&lt;staticValue&gt;../samples/images/Dashboard_PDF.png&lt;/staticValue&gt;
#&lt;/dataSource&gt;
#&lt;conditionalStyles&gt;&lt;conditionalStyleCases refVariable=&quot;For ReportOutput&quot;&gt;&lt;conditionalStyle refVariableValue=&quot;1&quot;&gt;&
lt;CSS value=&quot;display:none&quot;/&gt;&lt;/conditionalStyle&gt;&lt;/conditionalStyleCases&gt;&lt;conditionalStyleDefault/&gt;&lt;/conditionalStyles&
gt;&lt;reportDrills&gt;&lt;reportDrill name=&quot;PDF&quot;&gt;&lt;drillLabel&gt;&lt;dataSource&gt;&lt;staticValue/&gt;&lt;/dataSource&gt;&lt;/drillLabe
l&gt;&lt;drillTarget method=&quot;execute&quot; outputFormat=&quot;PDF&quot; showInNewWindow=&quot;true&quot;&gt;&lt;reportPath path=&quot;/content/fold
er[@name=&apos;Dashboards and Portals&apos;]/folder[@name=&apos;DRJOPS (Grower)&apos;]/folder[@name=&apos;Dashboard Reports&apos;]/report[@name=&apos;Sa
les&apos;]&quot;&gt;
@lines = <STDIN>;
for $line (@lines) {
  $_ = $line;
  chomp;
  if (/^(GET|POST)\s+(\/.*)\s(http|HTTP)/) {
    $uri = $2;
    $method = $1;
    next;
  } elsif (/^User-Agent:/ && ! $datasec) {
    ($agent) = /^User-Agent:\s+(.+)$/;
    print "User-Agent: $agent\n" if $DEBUG;
    next;
  } elsif (/^Content-Length:\s/ && ! $datasec) {
# LWP calculates its own content-length header
      next;
  } elsif  (! $datasec) {
# add header to hash
      ($hdr,$val) = /^(\S+):\s+(.+)$/;
      print "hdr,val: $hdr, $val\n" if $DEBUG;
      $myhash{$hdr} = $val if $hdr;
  }
  if (/^Host: (\S+)/i && ! $datasec) {
    $host = $1;
  }
  $data .= $line if $datasec;
# test if we are at the end of the headers
  $datasec = 1 if /^$/;
}
 
# Create a user agent object
use LWP::UserAgent;
$ua = LWP::UserAgent->new;
$ua->agent($agent);
$req = HTTP::Request->new($method => "https://$host$uri");
# enter all the headers
foreach $key (keys %myhash) {
  $val = $myhash{$key};
  print "key,val: $key, $val\n" if $DEBUG;
  $req->header($key => $val);
}
print "data: $data\n" if $DEBUG;
$req->content($data);
# Pass request to the user agent and get a response back
my $res = $ua->request($req);
 
# Check the outcome of the response
if ($res->is_success) {
 print $res->content;
} else {
  print $res->status_line, "\n";
}

Conclusion
We’ve created a Perl-based HTTP user agent to preserve headers. It’s not perfect but it’s a help, for instance in debugging am F5 web application firewall policy.

Categories
Python Raspberry Pi

Raspberry Pi visual alerting with Pibrella

I just came across this article: http://www.itworld.com/article/2919325/personal-technology/website-monitoring-with-a-raspberry-pi-for-nighttime-alerts.html?phint=newt%3Ditworld_linux&phint=idg_eid%3Db4ed050ca62ad3f6d5c69e8448d945d7#tk.ITWNLE_nlt_linux_2015-05-19

I wish I had thought of it first! I’m in a similar situation – I constantly get emails and TXT messages overnight so I have to put my phone in airplane mode. A visual alerting might just help give me an early warning for come critical failures.

The Pibrella module that lights up and buzzes has its own web site, pibrella.com.

If I do anything with it I’ll be sure to post it here.

Categories
DNS Linux Network Technologies Perl

Announcing a simple DNS web interface and code

Intro
For demonstration purposes I’ve written a WEB interface to do DNS queries. This can be used for light querying. Once it gets abused I will pull it from the web site.

Motivation
Some large enterprises are behind not only a corporate firewall, but also confined to a private namespace with no access to Internet name resolution. Users in such situations can use one of the many available tools to do DNS resolution through the web, but they all want to throw advertising at you and it’s not clear which can be trusted not to load you up with spyware. I am offering this ad-free DNS lookup using my position on the Internet as a trusted source.

And if you’re lucky and looking for code to do this yourself, you might find it. But nowhere will you find a site that’s running its own published code for DNS resolution. Except here.

The code
Admittedly very simple-minded, but hopefully not fatally flawed, here it is in Perl.

#!/usr/bin/perl
use CGI;
$query = new CGI;
%allowedArgs = (domainname =&gt; 'dum',type =&gt; 'dum',short =&gt; 'dum');
#
print "Content-type: text/html\n\n";
print "
\n";
foreach $key ($query->param) {
  exit(1) unless defined $allowedArgs{$key};
  exit(1) if $query->param($key) !~ /^([a-zA-Z0-9\.-]){2,256}$/;
  print "$key " . $query->param($key) . "\n";
}
# possible keys: domainname, type
$domainname = $query->param(domainname);
$type     = $query->param(type);
$type = "any" unless $type;
# argument validation checks
exit(1) if $domainname !~ /^([a-zA-Z0-9\.-]){2,256}$/ || $domainname =~ /\.\./ ||  ! $domainname;
exit(1) if $type !~ /^([a-zA-Z]){1,8}$/;

# short answer?
$short = "+short" if defined $query->param(short);

# authoritative request?
if (defined $query->param(authoritative)) {
# this will be a lot more complicated and so is not implemented. Perhaps someday if there is a request...
}

open(DIG,"dig $short $type $domainname|") || die "Cannot run dig!!\n";
while() {
  print ;
}

Yes it’s very old-school. I do not even use a DNS package. Why bother? It’s not rocket science. There’s a lot more to argument validation than it looks like – you would not believe the evil things people send to your web server. So you have to vigilant about injection attacks or shelling out by use of unexpected characters.

Usage

2020 Update

This URL has been deactivated since I moved to my new server. I’ll have to see if there’s time and interest to restore this functionality.

example 1

https://drjohnstechtalk.com/cgi-bin/digiface.cgi?domainname=johnstechtalk.com&type=a

domainname johnstechtalk.com
type a
 
; &lt;&lt;&gt;&gt; DiG 9.8.2rc1-RedHat-9.8.2-0.17.rc1.el6_4.4 &lt;&lt;&gt;&gt; a johnstechtalk.com
;; global options: +cmd
;; Got answer:
;; -&gt;&gt;HEADER&lt;&lt;- opcode: QUERY, status: NOERROR, id: 8711
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
 
;; QUESTION SECTION:
;johnstechtalk.com.		IN	A
 
;; ANSWER SECTION:
johnstechtalk.com.	3600	IN	A	50.17.188.196
 
;; Query time: 10 msec
;; SERVER: 172.16.0.23#53(172.16.0.23)
;; WHEN: Mon May  4 14:59:05 2015
;; MSG SIZE  rcvd: 51

example 2
https://drjohnstechtalk.com/cgi-bin/digiface.cgi?domainname=drjohnstechtalk.com&short

domainname drjohnstechtalk.com
short 
50.17.188.196

Familiarity with dig will help you determine the best switches to use as you can see that at the end of the day it is merely calling dig and sending back that output with a minimum of html markup. This will make it easy to parse the output programatically.

Conclusion
A simple DNS web interface is being announced today. Both the service and the code are being made available. The service may be pulled once it becomes abused.

References

2024 update

I learned about this basic but useful web interface to dig today: https://www.digwebinterface.com/
A nice, not too commercial web interface to dig and traceroute that is more user-friendly than mine is http://www.kloth.net/services/dig.php
The dig man pages can be helpful.

Got a geoDNS entry? Although this link has ads, it’s quite interesting because it sends your query to open DNS servers around the world: https://dnschecker.org/.

You can explore some details behind Google’s public resolving server 8.8.8.8 by using the web site: https://dns.google.com/. It’s quite helpful.

I won’t paste the link to my service but you can see what it is from the examples above.

There’s a simple but effective DIG available for your Android smartphone from the Playstore. That’s DNS debugger from TurboBytes. No obnoxious ads and yet no cost.

Of course if you are on the Internet and have access to dig, Google’s DNS servers are available for you to use directly.

Want to learn if the Great Friewall of China is clobbering the expected DNS result? The site https://viewdns.info/chinesefirewall/ is designed to do just that.

Categories
Admin DNS

Example of case-sensitive DNS usage

Intro
From RFC 1035, written in November, 1987:


Note that while upper and lower case letters are allowed in domain
names, no significance is attached to the case. That is, two names with
the same spelling but different case are to be treated as if identical.

The details
Now fast forward in time 27 years. I learned that Cisco IP Phones, when resolving the Call manager name, require that the DNS name for the Cisco Unified Call Manager be in the same exact upper or lower case as what is configured into the phone.

Suppose your Call Manager’s hostname was configured as CUCM.drjohnstechtalk.com and your DNS servers behaved like this:

> dig CUCM.drjohnstechtalk.com @208.109.255.46

; <<>> DiG 9.9.4-P2 <<>> CUCM.drjohnstechtalk.com @208.109.255.46
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15899
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 1
;; WARNING: recursion requested but not available
 
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;CUCM.drjohnstechtalk.com.      IN      A
 
;; ANSWER SECTION:
cucm.drjohnstechtalk.com. 3600  IN      A       50.17.188.196

Well, every application that is compliant with this 27-year-old DNS standard would work just fine. But Cisco phone’s will not. If they were configured to use CUCM.drjohnstechtalk.com and your DNS server spits back the answer to an A (address record) query, changing the FQDN to lower-case, it won’t “find” the call manager and won’t boot! So it’s a garbage implementation of DNS.

Shame on Cisco!

I happened to hear about this problem today, so it can occur under those very special circumstances outlined above. We can’t merely say it is only theoretical. However, mitigating circumstances abound that will make this a rarely observed problem.

Mitigation
Newer DNS servers actually spit back the FQDN in the exact same case as it received in the original query. I’m not sure at this point if this is an option or simply a change in behaviour that occurred at some point in the evolution of the ISC BIND resolver. It would be interesting to see when this behaviour changed.

The other mitigation, if you do have the older DNS servers that spit back the FQDN in lower-case is to configure the hostname in your zone file using upper case to agree with the upper-case version you’ve configured on the phone. With either of these mitigations the DNS server response will look like this:

; <<>> DiG 9.9.4-P2 <<>> CUCM.drjohnstechtalk.com @208.109.255.46
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15899
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 1
;; WARNING: recursion requested but not available
 
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;CUCM.drjohnstechtalk.com.      IN      A
 
;; ANSWER SECTION:
CUCM.drjohnstechtalk.com. 3600  IN      A       50.17.188.196

and the phone will be happy, seeing the case matched and will be able to contact the Call Manager so it can finish booting.

Conclusion
Cisco of all companies has built in to its IP Phones a bad DNS resolver that is case-sensitive. There are some mitigations which can be done while waiting for them to fix this embarrassing bug.

Second example from VMWare circa June 2020
The VMWare Horizon Client v 5.4 has a similar issue. If you use a proxy PAC file with contents like *.drjohnstechtalk.com DIRECT, that may not work for this client if the DNS entry for the hostname was entered in upper case! For instance HostName.DRJOHNSTECHTALK.COM. In that case it acts with case-sensitivyt and ignores the PAC file entry which it should have used to know to make DIRECT (without the aid of a proxy) HTTP connection. Very unfortunate.

References
RFC 1035 – things were so much simpler then!
ISC BIND web site.

Categories
Admin

Bitninja – is it legit?

Does anyone know if Bitninja is a legitimate service? They purport to provide security services. I recently got an email from them with a link to some supposedly bad URLs that prove that a PC has malware. I’m a little skeptical. To get full details I have to pay. To request a delisting I have to pay.

To me it smells like some of those thinly veiled extortionist schemes that I come across in the mail world.

The particulars in this case consist of stripped access information (I can’t think of any good reason to strip some of the most useful information away) which look like this:

Bitninja stripped listing
Bitninja stripped listing

There is only one thing in my logs that that could be. They got the time very wrong. The host is www.casarivercentury.org. It kind of looks like it may be a honeypot that they managed to sneak by Google and enticed some unlucky souls to use. Or not.

Or maybe I’ve got it all wrong. Does anyone else have experience with them?

Categories
Admin Raspberry Pi

Can’t ssh to Raspberry Pi

Intro
I did not experience this problem but it happened to a friend of mine – more than once. I’m not sure what he’s doing during installation of the OS to cause this but I know the solution.

Also there is some really misguided information out there on the Internet. Pure ignorance talking. There’s some accurate discussion as well. By re-posting correct information I hope to increase the average correct treatment. I have nothing original to say per se.

The details
The problem is that you can”t log in to your Raspberry Pi via ssh. See if the ssh daemon is running:

$ sudo service ssh status

sshd is running.

Probably it is and you see the response above.

Try to log in locally:

$ ssh localhost

If you get something like this:

Read from socket failed: Connection reset by peer

then you have the problem which can be fixed by this procedure below.

The fix

$ sudo dpkg-reconfigure openssh-server

Or more generally:

$ sudo rm /etc/ssh/ssh_host_* && sudo dpkg-reconfigure openssh-server

There must be some way to screw up the Raspbian installation that produces this particular problem.

And all this is assuming you already had the presence of mind to enable ssh in raspi-config. That of course is a prerequisite.

Conclusion
You should be able to ssh (remote shell access) to your Raspberry Pi. If you cannot the recipe above is probably the needed fix. This discussion is generally valid for all Debian installations, except for the comment about raspi-config!

References
The correct discussion on this topic in the official Raspberry Pi forum is here.
I’ve published quite a few articles now on Raspberry Pi.
A digital photo frame is described in this article.
Using a Raspberry Pi as a router is described here.

Categories
Raspberry Pi

True (horror) story about the Banana Pi

Intro
The laughably named Banana Pi is an unabashed knock-off of the Raspberry Pi by some folks in China. The specs look pretty good on paper and I guess it is cheap. But don’t jump on that bandwagon just yet, there are several drawbacks.

Burn baby burn

This anecdote was related to me by an electrical engineer friend of mine. It happened to a friend of his and I trust my source for this story. His friend built a qaud copter and wanted an on-board cpu to control it. So he put on a Banana Pi. While flying the quad copter the Banana Pi spontaneously burst into flames!!! Yes, it burned up. But wait, there’s more. He actually called support. They didn’t address any real questions he raised but this is what the support person had to say: “Raspberry Pis burn up, too!”

By the way the CanaKit I recently bought which included the new Raspberry Pi 2 comes with two heat sinks you attach to the CPU and another chip. It seems like a good idea to have that to keep things cool.

The other drawback of the Banana Pi is that there is a much smaller user community offering helpful, free advice and project ideas (such as my blog).

Conclusion
I recommend against purchasing the Banana Pi. Look into a Raspberry Pi model 2 instead. With 5 million Pis sold you can see there’s something going on there worth joining. The Pi 2 seems to run all the software the older models do, and has the same size. It’s just faster.

References
A kit on Amazon currently costs $70. http://smile.amazon.com/Guide–Clear-Case–Power-Supply–WiFi-Dongle–Kingston-Adapter–HDMI/dp/B00MV6TAJI/ref=sr_1_4?ie=UTF8&qid=1424894339&sr=8-4&keywords=raspberry+Pi+2

Categories
Linux Python Raspberry Pi

What I’m working on: a Raspberry Pi digital photo frame

Intro
The idea is that for a display kiosk let’s have a Raspberry Pi drive a display like one of those electronic picture frames. Power the thing up, perhaps plug in a flash drive, leave off the mouse and keyboard, but have a display attached, and get it to where it just automatically starts a slideshow without more fuss.

Some discarded options
Obviously this is not breaking new ground. you can find many variants of this on the Internet. An early-on approach that caught my eye is flickrframe. I read the source code to learn that at the end of the day it relies on the fbi program (frame buffer imageviewer). I thought that perhaps I could rip out the part that connects to Flickr but it seemed like too much trouble. At the end of the day it’s just a question of whether to use fbi or not.

Then there’s Raspberry Pi slideshow. That’s a quite good write-up. That’s using pqiv. I think that solution is workable.

But the one I’m focusing on uses qiv. You would have thought that pqiv would rely on qiv (quick image viewer) but it appears not to. So qiv is a separate install. qiv has lots of switches so it’s been written with this kind of thing in mind it seems.

What it looks like so far

#!/bin/sh
# -f : full-screen; -R : disable deletion; -s : slideshow; -d : delay <secs>; -i : status-bar;
# -m : zoom; [-r : ranomdize]
# this doesn't handle filenames with spaces:
##cd /media; qiv -f -R -s -d 5 -i -m `find /media -regex ".+\.jpe?g$"`
# this one does:
if [ "$1" = "l" ]; then
# print out proposed filenames
  cd /media; find . -regex ".+\.jpe?g$"
else
  sleep 5
  cd /media; find . -regex ".+\.jpe?g$" -print0|xargs -0 qiv -f -R -s -d 5 -i -m
fi

The idea being, why not make a slideshow out of all the pictures found on a flash drive that’s been inserted into the Pi? That’s how a standard picture frame works after all. It’s a very convenient way to work with it. That’s the aim of the above script.

Requirements update
OK. Well this happens a lot in IT. We thought we were solving one problem but when we finally spoke with the visual arts team they had something entirely different in mind. They want to mix in movies as well. fbi, pqiv or qiv don’t handle movies. I have mplayer and vlc from my playing around with Raspberry Pi camera. mplayer runs like a dog on the movie files I tried, perhaps one frame update every two seconds. After more searching around I came across omxplayer. That actually works pretty well. It on the other hand doesn’t seem up to the task of handling a mixed multimedia stream of stills and movies. But it did handle the two movies types we had: .mov and .mp4 movie files. omxplayer is written specifically for the Pi so it uses its GPU for frame acceleration. mplayer just seems to rely on the CPU which just can’t keep up on a high-def quality movie. So as a result omxplayer will only play through a true graphical console. It doesn’t even bother you to get your DISPLAY environment variable set up correctly – it’s just going to send everything to the head display.

Overheard recently
And when using my TV as display omxplayer put out the sound, too, perfectly synchronized and of high quality.

I was thinking if we should kludge stitching together qiv and omxplayer. You know letting one lapse and starting up the other to transition from a still to a movie, but I don’t know how to make the transition smooth. So i searched around yet some more and found pipresents. I believe it is a python framework around omxplayer. It’s pretty sophisticated and yet free. It’s actually aimed at museums and can include reactions to pressed buttons as you have at museum displays. So far we got the example media show to loop through – it demonstrates a high-quality short movie and a still plus some captions at the beginning.

Pipresents isn’t perfect however
I quickly found some problems with pipresents so I went the official route and posted them to the github site, not really knowing what to expect. The first issue is that you are not allowed to import .mov files! That makes no sense since omxplayer plays them. So I post this bug and that very same day the author emails me back and explains that you simply edit pp_editor.py line 32 and add .mov as an additional video file type! Sure enough, that did it. Then I found that it wasn’t downsampling my images. These days everyone has a camera or phone that takes mmulti-megapixel images far exceeding a cheap display’s 1280×1024 resolution. So you only see a small portion of your jpeg. I just assumed pipresents would downsample these large pictures because the other packages like qiv do it so readily. Again the same day the author gets back to me and says no this isn’t supported – in pipresents. But there is a solution: I should use pipresents-next! It’s officially in beta but just about ready for production release. I don’t think I’ll go that route but it’s always nice to know your package continues to be developed. I’ve written my own downsampler which I will provide later on.

Screen turns off
The pipresents has a command-line switch, -b, to prevent screen blanking. But I think in general it’s better to not use that switch and instead disable screen blanking in general.

$ sudo nano /etc/kbd/config
– change BLANK_TIME=30 to BLANK-TIME=0
– and change POWERDOWN_TIME=30 to POWERDOWN_TIME=0
$ sudo nano /etc/lightdm/lightdm.conf
– below the [SeatDefault] line create this line:
xserver-command=X -s 0 dpms

How to get started with PiPresents
$ wget https://github.com/KenT2/pipresents/tarball/master -O – | tar xz

There should now be a directory ‘KenT2-pipresents-xxxx’ in your home directory. Rename the directory to pipresents:

$ mv KenT2* pipresents

To save time make sure you have two terminal windows open on your Pi and familiarize yourself with how to cut and paste text between them. Then from the one window you can:

$ cd pipresents; more README.md

while you execute the commands you’ve cut and paste from that window into the other, e.g.,

$ sudo apt-get install python-imaging
etc.

What happens if you forget to install the unclutter package
Not much. It’s just that you will see a mouse pointer in the center of the screen which won’t go away, which is not desirable for black box operation.

Python image downsizing program
This is also known as downsampling. Amazingly, you really don’t find a simple example program like this when you do an Internet search, at least not amongst the first few hits. I needed a program to reduce the large images to the size of the display while preserving the aspect ratio. My display, a run-of-the-mill Acer v173, is 1280 x 1024 pixels. Pretty standard stuff, right? yet the Pi sees it as 1232 x 992 pixels! Whoever would have thought that possible? And with no possible option to change that (at least from the GUI). So just put in the appropriate values for your display. This program just handles one single image file. also note that if it’s a small picture, meaning smaller than the display, it will be blown up to full screen and hence will make a thumbnail image look pixelated. The match doesn’t distinguish small from large images but I fel that is fine for the most part. So without further chatting, here it is. I called it resize3.py:

import Image
import sys
# DrJ 2/2015
# somewhat inspired by http://www.riisen.dk/dop/pil.html
# image file should be provided as argument
# Designed for Acer v173 display which the Pi sees as a strange 1232 x 992 pixel display
# though it really is 1 more run-of-the-mill 1280 x 1024
 
imageFile = sys.argv[1]
im1 = Image.open(imageFile)
 
def imgResize(im):
# Our display as seen by the Pi is a strange 1232 x 992 pixels
    width = im.size[0]
    height = im.size[1]
 
# If the aspect ratio is wider than the display screen's aspect ratio,
# constrain the width to the display's full width
    if width/float(height) > 1232.0/992.0:
      widthn = 1232
      heightn = int(height*1232.0/width)
    else:
      heightn = 992
      widthn  = int(width*992.0/height)
 
    im5 = im.resize((widthn, heightn), Image.ANTIALIAS) # best down-sizing filter
 
    im5.save("resize/" + imageFile)
 
imgResize(im1)

As I am not proficient in python I designed the above program to minimize file handling. That I do in a shell script which was much easier for me to write. Together they can easily handle downsampling all the image files in a particular directory. I call this script reduce.sh:

#!/bin/sh
echo "Look for the downsampled images in a sub-directory called resize
echo "JPEGs GIFs and PNGs are looked at in the current directory
mkdir resize 2>/dev/null
ls -1 *jpg *jpeg *JPG *png *PNG *gif *GIF 2>/dev/null|while read file; do
  echo downsampling $file
# downsample the image file
  python ~/resize3.py "$file"
done

Stopping the slideshow
Sometimes you just need to stop the thing and that’s not so easy when you’ve got it in blackbox mode and running at startup.

If you’re lucky enough to have a keyboard attached to the Pi we found that

<Alt> F4

from the keyboard stops it.

No keyboard? We assigned Our Pi a static IP address and leave an ethernet cable attached to it. Then we put a PC on the same subnet and ssh to it, e.g., using putty or teraterm. Then we run this simple kill script, which I call kill.sh:

#!/bin/sh
pkill -f pipresents.py
pkill omxplayer

Digital photo frame projects morphs to museum-style kiosk display
At times I was tempted to throw out this pipresents software but we persisted. It has a different emphasis from a digital photo frame where you plug in a USB stick and don’t care about the order the pictures are presented to you. pipresents is oriented towards museums and hence is all about curated displays, where you’ve pored over the presentation order and selected your mix of videos and images. And in the end that better matched our requirements.

The manual is wanting for clarity
It’s nice that a PDF manual is included, but it’s a pain to read it to extract the small bits of information you actually need. Here’s what you mostly need to know. An unattended slideshow mixture of images and videos is what he calls a mediashow. Make your own profile to hold your mediashow:

$ cd pipresents; python pp_editor.py

This brings up a graphical editor. Then follow these menus:

Profile|New from template|Mediashow

Choose a short easy-to-type name such as drjmedia.

Click on media.json and then you can start adding images and movies. These are known as “tracks.”

Remove the example track.

Add your own images and movies.

Do a Profile|Validate

There is no Save! Just kill it.

And to run it full screen from your home directory:

$ python pipresents/pipresents -ftop -p drjmedia

Autostarting your mediashow
The instructions provided in the manual.pdf worked on my older Pi, but not on the B+ model Pis. So to repeat it here, modifying it so that it is more correct (the author doesn’t seem comfortable with Linux). Manual.pdf has:

$ mkdir -p ~/.config/lxsession/LXDE
$ cd !$; echo "python pipresents/pipresents.py -ftop -pdrjmedia" > autostart
$ chmod +x autostart

And as I say this worked on my model B Pi, but not my B+. The following discussion about autostarting programs is specific to operating systems which use the LXDE desktop environment such as Raspbian. On the B+ this fairly different approach worked to get the media show automatically starting upon boot:

$ cd /etc/xdg/autostart

Create a file pipresents.desktop with these lines:

[Desktop Entry]
Type=Application
Name=pipresents
Exec=python pipresents/pipresents.py -ftop -pdrjmedia
Terminal=true

But I recommend this approach which also works:

$ mkdir ~/.config/autostart

Place a pipresents.desktop file in this directory with the contents shown above.

More sophisticated approach for better black box operations
We find it convenient to run pp_editor in a virtual display created by vnc. Then we still don’t need to attach keyboard or mouse to the Pi. But the problem is that pipresents will also launch in the vnc session and really slow things down. This is a solution I worked out to have only one instance of pipresents run, even if others X sessions are launched on other displays. Note that this is a general solution and applies to any autostarted program.

The main idea is to test in a simple shell script if our display is the console (:0.0) or not.

I should interject I haven’t actually tested this but I think it’s going to work! Update: Yes, it did work!

Put startpipresents.sh in /home/pi with these contents:

#!/bin/bash
# DISPLAY environment variable is :0.0 for the console display
echo $DISPLAY|grep :0 > /dev/null 2>&1
if [ "$?" == "0" ]; then
#  matched. start pipresents in this xsession, but not any other one
  python pipresents/pipresents.py -ftop -pdrjmedia
fi

Then pipresents.desktop becomes this:

[Desktop Entry]
Type=Application
Name=pipresents
Exec=/home/pi/startpipresents.sh
Terminal=true

To install the vnc server:

$ sudo apt-get install tightvncserver

And to auto-launch it make a vnc.desktop file in ~/.config/autostart like this:

[Desktop Entry]
Type=Application
Name=vncserver
Exec=/home/pi/startvncserver.sh
Terminal=false

and put this in the file /home/pi/startvncserver.sh:

#!/bin/bash
# DISPLAY environment variable is :0.0 for the console display
echo $DISPLAY|grep :0 > /dev/null 2>&1
if [ "$?" == "0" ]; then
#  matched. start vncserver in this xsession, but not any other one
  vncserver
fi

You need to launch vncserver by hand once to establish the password.

And we may as well pre-launch the pp_editor because we’re likely to need that. So make a file in the home directory called startppeditor.sh with these contents:

#!/bin/bash
# DISPLAY environment variable is :1.0 for the vnc display
echo $DISPLAY|grep :1 > /dev/null 2>&1
if [ "$?" == "0" ]; then
#  matched. start ppeditor in this xsession, but not any other one
  python pipresents/pp_editor.py
fi

and in ~/.config/autostart a file called ppeditor.desktop with these contents:

[Desktop Entry]
Type=Application
Name=ppeditor
Exec=/home/pi/startppeditor.sh
Terminal=true

Similarly we can pre-launch an lxterminal because we’ll probably need one of those. Here’s an example startlxterminal.sh:

#!/bin/bash
# DISPLAY environment variable is :1.0 for the vnc display
echo $DISPLAY|grep :1 > /dev/null 2>&1
if [ "$?" == "0" ]; then
#  matched. start a large lxterminal in this xsession, but not any other one
  lxterminal --geometry=100x40
fi

and the autostart file:

[Desktop Entry]
Type=Application
Name=lxterminal
Exec=/home/pi/startlxterminal.sh
Terminal=true

A note about Powerpoint slides
With a Macbook we were able to read in a Powerpoint slideshow and export it to JPEG images, one image per slide. That was pretty convenient. We have done the same directly from Microsoft Powerpoint – it’s a save option.

A note about Mpeg4 videos
Some videos overwhelm these older Pis that we use. Maybe on the Pi 3 they’d be OK? A creative student would hand us his 2 minute movie in mpeg4 format. The Pi would never be able to display it. We learned you can reduce the resolution to get the Pi to display it. A student was doing this on his Macbook, but when he left i had to figure out a way.

The original mpeg4 video had resolution of 1920 x 1080. I wanted to have horizontal resolution of no more than 1232, but maybe even smaller, while preserving the aspect ratio (widescreen format).

I used good ole’ Microsoft Movie Maker. I don’t think it’s available any longer except from dodgy sites, but in the days of Windows 7 you could get it for free through Windows Live Update. Then, if you upgraded that Windows 7 PC to Windows 10, it allowed you to keep Movie Maker. That’s the only way I know of. Not that it’s a good program. It’s not. Very basic. But it does permit resizing a video stream to custom resolution, so I have to give it that. I tried various resolutions nd played them back. i finally settled on the smallest I tried: 800×450. In fact I couldn’t really tell the difference in video quality between all the samples. And of corse 800×450 made for the smallest file. So we took that one. Fortunately, pipresents blew it up to occupy the full screen width (1232 pixels) while preserving the aspect ratio. So it looks great and no further action was needed.

The sound of silence
You want the video sound to come out the stereo mini-jack because you’re not using an HDMI monitor? PiPresents tries to send audio out through HDMI by default so you won’t hear the sounds if you have a VGA monitor. But you can change that. If you want to do this in raw omxplayer the switch which sends the sound out through the mini-jack is:

omxplayer -o local

In pipresents this option is available in the pp_editor. It’s a property of the profile. So you edit the profile, look for omx-audio, and change its value in the drop-down box from hdmi to local. That’s it!

A word about DHCP
We use a PC to connect to the four Pis. They are connected to a hub and there is an Ethernet cable connected to the hub and ready to be connected to a PC with an Ethernet port. The Pis all have private IP addresses: 10.31.42.1, 10.31.42.2, 10.31.42.3 and 10.31.42.4. For convenience, we set up a DHCP server on Pi 1 so that when the PC connects, it gets assigned an IP address on that subnet. DHCP is a service that dynamically assigns IP addresses. Turns out this is dead easy. You simply install dnsmasq (sudo apt-get install dnsmasq) and make sure it is enabled. That’s it! More sophisticated setups require modification of the file /etc/dnsmasq.conf, but for our simple use case that is not even needed – it just picks reasonable values and assigns an appropriate IP to the laptop that allows it to communicate to any of the four Pis.

References and related
I worked on this project with a student. Building a Four Monitor Media Show using Raspberry Pis
Pipresents has its own wordpress site.

LXDE has its own official site.
Read about a first look at the custom-built 7″ Raspberry Pi touch display in this blog post.

An alternative slideshow program to pipresents is to leverage qiv. I put something together and demo it in this post, but with a twist: I pull all the photos from my own Google Drive, where I store 40,000+ pictures!