Categories
Admin Linux

Things that went wrong during the HP SiteScope upgrade

Intro
These are my notes of all the stupid things I did during my attempt to upgrade to HP SiteScope version 11.24 in response to a security problem in earlier releases. When you don’t do these things very often you totally forget what you did last time!

The details
I managed to download the “patch,” SIS_00314, from the HP Passport site. That was relatively straightforward.

First mistake: winging it
After baking up my configuration I looked into the zip file and saw an rpm that seemed like the thing I needed: packages/HPSiS1124Core-11.24.241-Linux2.4.rpm. So I decided to just install it directly. That created deep directories like this one:

/opt/HP/SiteScope/installation/HPSiS1124/flvr/SiteScope

and it really looked like it wasn’t going to do anything. And when I tried that with the Java package all kinds of dependent libraries were not found. So I removed that package and went back to the manual.

So I identified the included deployment guide and skimmed through that for inspiration. It seems you should drive the installation through the file HPSiS1124_11.24_setup.bin. So I tried it. First I had to set up my X-Windows stuff:

# vncserver :2
# export DISPLAY=:2.0
(then connect to that display using VNC client)
# xhost +
and to test it (the following command should pop up a new window):
# gnome-terminal

Then finally:

# ./HPSiS1124_11.24_setup.bin

Preparing to install...
Extracting the JRE from the installer archive...
Unpacking the JRE...
Extracting the installation resources from the installer archive...
Configuring the installer for this system's environment...
 (i) Checking display ...
 (-) Display may not be properly configured
Please make sure the display is set properly...

I don’t know what went wrong, but I knew a GUI install was out of the question without a lot of digging. I also knew about silent or console installs. So I looked to that part of the deployment manual.

# ./HPSiS1124_11.24_setup.bin -i silent

Preparing to install...
Extracting the JRE from the installer archive...
Unpacking the JRE...
Extracting the installation resources from the installer archive...
Configuring the installer for this system's environment...
Preparing SILENT Mode Installation...
 
===============================================================================
                                       (created with InstallAnywhere by Zero G)
-------------------------------------------------------------------------------
 
=======================================================
 
Installer User Interface Mode Not Supported
 
Unable to load and to prepare the installer in console or silent mode.
 
=======================================================

I learned this happens if you run silent mode but have a DISPLAY environment variable set! Just unset DISPLAY and re-run.

But that didn’t work out for me either. In /tmp/HPOvInstaller/HPSiS1124_11.24 the file HPSiS1124_11.24_2014.08.11_15_39_HPOvInstallerLog.txt showed:


2014-08-11 15:39:40,065 INFO – Checking free disk space… [FAILED]

However since it was a silent installation it never complained! I only knew there was a problem when I restarted SiteScope and saw it was still showing me version 11.23.

I cleared some more space and the install finally completed successfully.

Conclusion
We ran into quite a few problems doing a simple minor HP SiteScope upgrade, most of our own making. But we persevered and are now running version 11.24.

References
Yearning for the old Freshwater SiteScope? You’ll want to read this blog posting and comments.
The Secunia advisory.

Categories
Admin Network Technologies

Routing based on source MAC address

Intro
As I am not a true network specialist but more a security operations specialist, I am always amazed by discovering network things that they probably teach in networking 101 and seem obvious to those in the know.

I had a “revelation” (to me anyways) like that lately.

The details
For all these years I’ve dabbled setting up interfaces, creating static routes, setting up RIP advertisements, reading BGP configurations, solving martian source problems, or slightly harder things like network address translation, secure network address translation and vpn tunnels, I thought I knew all the mainly relevant things there is to know about how to get packets to where you want them.

For a typical, non-routing device, I had always thought that the only choice on how to send packets out is

– local subnet if destination is on a subnet of one of the interfaces
– static route if configured for that destination IP
– default gateway if configured

And that’s that, right?

This limited number of route selection methods is very important in some architectural designs I’ve been involved with. For instance, an Intranet which has “borrowed” large swaths of valid Internet address space pretty much necessitates a two-stage proxy approach if using explicit proxy settings. Or so I thought.

What I learned
Someone pointed out to me a feature on Bluecoat proxy called return-to-sender. Normally it is disabled. But when enabled, what it does is (to me, because I never thought it possible) amazing: it sends its response packets back to the MAC address of the inbound sender! Thus it will shortcut all the routing decisions mentioned above and use the MAC address of the host which sent it a packet to which it is responding.

If I had know that was possible I might never have implemented a two-stage proxy.

I decided to try the ultimate worst-case scenario:

– use of proxy with this feature enabled and
– proxy client on same internal subnet as proxy server, 10.11.12.0/24
– source IP of proxy client = my AWS server, 50.17.188.196
– requested web site: my AWS web site at http://50.17.188.196/

In other words, steal my own IP, put it on the Intranet, and make the proxy route packets back to me in response while simultaneously connecting to that exact same IP on the Internet.

How I configured the VIP
This is very old school, but I did one of these numbers ona SLES11 system:

$ sudo ifconfig eth0:0 50.17.188.196 up

And that was sufficient to add that IP to the eth0 interface.

Did it work? Yes, it did. Amazing.

I only needed one single request to verify that it worked. Here is that request, using wget from the Linux host which is on the same subnet.

export http_proxy=http://10.11.12.13:8080/
wget --bind-address=50.17.188.196 http://50.17.188.196/

And the proxy log line this produced:

2014-08-04 13:55:08 3 50.17.188.196 - "none" 200 TCP_HIT GET text/html http 50.17.188.196 80 / - - "Wget/1.11.4" OBSERVED 10.11.12.13 769 158 - -

Other appliances can do this, too
On F5 BigIP this feature is also supported. It is called auto last hop and can be configured either globally or for an individual virtual server.

To be continued…

Categories
Admin Internet Mail Spam

Gmail: not as much a white-glove service as you thought

Intro
I have a pretty high regard for Google and their Gmail email service. They really seem to strive to keep its reputation sterling. But lately a persistent spam has been coming in to me from one of their users and no action is being taken, so I am beginning to wonder.

The details

It’s not that I don’t get spam from Gmail account holders. I do. That’s not shocking as I get hundreds of spam each day since my address is available from whois registrations on hundreds of domains, amongst other readily available sources.

How do I know it’s a real Gmail user and not someone spoofing the sender address? These two headers tell me:

Received-SPF: pass (google.com: domain of [email protected] designates 209.85.160.44 as permitted sender) client-ip=209.85.160.44;
Received: from mail-pb0-f44.google.com ([209.85.160.44]) (using TLSv1) by drjohnstechtalk.com, etc.

In other words one of the received headers provided by a trusted server gives me the IP of the sending server (209.85.160.44), which is in Google’s directly allocated IP space.
This can be confirmed at arin.net.

The email itself looks like this:

From: "Tom Zhu" <[email protected]>
 
Dear Sir/Madam,
 
We are owner of your_domain.mx and Presently we would like to know if you have an
interest in buying it. We are looking to sell this domain for 2,000 Euro.
It has been listed on Sedo.com. You can buy it through the following link:
 
https://sedo.com/search/details.php4?domain=your_domain.mx
...

But instead of your_domain the email mentions a specific copyrighted domain name.

I’ve received it over 10 times from the same sender. The sender is a cybersquatter sending repeated, unsolicited spam. If that doesn’t constitute a violation of their Terms of Use then I don’t know what does. I’ve filed no fewer than five formal abuse complaints to Google over the course of the several months. The Gmail abuse link is in the references. But they keep coming in so I know Google has taken no action whatsoever. And of course I have never heard back from them.

I’ve filed lots of other abuse complaints about other Gmail senders as well, but those others seem to be one-off spams and I don’t get additional emails from them. Yes it takes time to fill out the abuse form, but I do it for the overall good of the Internet. We technical people have a responsibility to make our world better…

Conclusion
I am personally miffed and professionally concerned that Google Gmail may not be operating as clean a shop (white glove) as we all had thought. Here I’ve documented a specific case of documented abuse about which they have done nothing for months on end.

References
Gmail abuse link.
ARIN’s IP lookup service is here.
A detailed look at how enom has handled some spam/domain complaints is written up here.
My successful fight to conquer scads of Chinese language spam is documented here.

Categories
Admin Linux Network Technologies

The IT Detetecive Agency: the case of the unreliable FTP

Intro
So one of my power users complains that his FTPs to a particular site fail frequently, but not always. I rolled up my sleeves and set to work. The thing I do best is find the essence of a problem – what is the bare minimum sequence of events that reproduces it. I’m still getting my head around this one and I haven’t cracked the case yet, but I’ve learned about a few obscure packet generation tools.

The details
I may flesh this out later. The essence of the thing is that a packet trace (using tcpdump) shows that randomly no SYN-ACK packet is returned for our SYN packet to the FTP server on port 21. The FTP server resides on the Amazon cloud, but on the West Coast. We are on the East Coast. Not that that matters.

So I learned to reproduce the problem myself with the built-in ftp client. But I wanted even more control.

Packet generation tools
My trajectory went kind of like this:

ftp -> ping -> nmap -> hping3 -> mausezahn (-> scapy)

I had to compile mausezahn but I did manage to get it to work. I guess the developer has passed away. It doesn’t offer complete control over tcp packet generation, but nearly so.

I just discovered scapy. It appears to offer complete control over packet generation, including the tcp options such as mss, but the proof is in the pudding and I haven’t had time to check it out.

See the references fo links to further information about where to find these packages.

Preliminary findings
I began to see that with nmap and hping3 I was getting SYN-ACKs back consistently. What’s the difference between their SYN packets and ftp’s? They don’t use any options whereas my ftp client does.
And that is the essence of the problem. A tcp SYN packet which sets options like SACK, wscale and MSS is not being responded to around 30% of the time. No options set? SYN-ACKs come back 100% of the time. Pings are answered 99 – 100% of the time. mausezahn (mz) allows to set the window size. The window size is irrelevant.

Is it one particular tcp option that is the culprit, or just the fact of using any of them? Unfortunately that’s where you reach the limits of mz. mz only allows you to turn on or off all options. scapy promises to be more granular. So at least with mz by itself I can turn of/off the problem at will. That is getting to the essence of the problem.

Another wrinkle? Only certain source IPs have the problem! I have an identical system using a different ISP and it works all day long.

Conclusion
A lot of work and only modest progress to show for it. I need cooperation of the ftp administrator to do a simultaneous trace. Either the packet never gets to him, or his infrastructure discards it, or he responds but I never see the response. A two-sided trace will narrow down which of these three things is happening.

But I did learn that fine-control packet generation is a bit difficult to come by, which comes as a surprise in this day and age. You have to do some work to get full control over your packets.

I have nos stomach for writing my own C++ code to have total control.

It’s still an open case.

References
nmap.org talks about nmap. nmap is a pretty standard package available for major distributions. But it is not sufficiently configurable.
I’ve written about hping3 before, showing how to compile it.
I used this site for mausezahn source code and documentation.
scapy is well-documented here.

Categories
Admin Linux Network Technologies

Querying AD via LDAP – reference documentation

Intro
Suppose you managed to stuff the user’s ID into the description field of every computer object. Then independently the name of the computer object appears in a log such as a web server log and you want to know the user associated with that.

These examples show how to get at that description field from the computer object name.

The details

ldapsearch is a useful tool. I have two versions of it installed on Windows 7 and different Unix/Linux versions. The syntax is slightly different in all cases. Let’s assume the AD domain DRJOHNSAD is mapped to DNS domain drjohnsad.drjohns.net, and the user is drj. Then we have:

Linux
> ldapsearch -h drjohnsad.drjohns.net -b dc=drjohnsad,dc=drjohns,dc=net -D ‘drjohnsad\drj’ -W cn=computerName description

The -W switch prompts for the password. That is a nice switch, and not available in all versions of ldapsearch. If not, use -w password instead. drjohnsad\drj needs the single quote to prevent the “\” character from being treated as a special character by the shell. Windows doesn’t need that.

Windows 7 CMD Window

Oracle-provided ldapsearch

> ldapsearch -h drjohnsad.drjohns.net -b dc=drjohnsad,dc=drjohns,dc=net -D drjohnsad\drj -q cn=computerName description

So -q is used to prompt for a password instead of Linux’s -W.

Lotus Notes ldapsearch

> ldapsearch -h drjohnsad.drjohns.net -b dc=drjohnsad,dc=drjohns,dc=net -D drjohnsad\drj -w password cn=computerName description

You gotta put in the password on the command line.

Of course Windows also has applications which can be used for ldap queries in a GUI, but I don’t use them.

Conclusion
The syntax for a simple ldap query against an AD domain controller is shown.

Categories
Admin Web Site Technologies

Getting WebDav to work through Basic Authentication and HTTP

Intro
What I’m about to describe is not a recommended setup, but if like me you’re dealing with legacy infrastructure, well, sometimes you just gotta make things work as they are.

So imagine on your Intranet you have a WebDAV server running HTTP, not HTTPS, and using Basic Authentication. And you want to work with the files on a Windows 7 workstation. Read on to see how we got this combination to play nice together.

The details
First I gave myself access to a WebDAV resource on the server.

Being a Unixy type of guy, I then tried a Linux command-line program. I chose cadaver. See the webdav.org site for other clients.

I had to compile and install cadaver but that was no problem. Here is my test session:

> cadaver http://iwwwd.drj.com/webdav.ear/
Authentication required for CORP LDAP webDAV on server `iwwwd.drj.com’:
Username: drj
Password:
dav:/webdav.ear/> ls
Listing collection `/webdav.ear/’: succeeded.
Coll: drJ Application.ear 0 Sep 17 2012
Coll: drj app Application.ear 0 Nov 30 2013

dav:/webdav.ear/>

So, in other words, it worked! cadaver is kind of nice because it puts you into a shell and has commands similar to FTP, so it is a (mostly) familiar environment.

Now why was my Windows 7 PC giving me such a hard time?

Windows 7 by default only allows for Basic authentication against HTTPS webDAV servers.

To enable Basic authentication on the client computer, follow these steps:
1) Click Start , type regedit in the Start Search box, and then click regedit.exe in the Programs list.
2) Locate and then click the following registry key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\WebClient\Parameters
3) Double-click the BasicAuthLevel registry key.
4) In the Value data box, type 2, and then click OK.
5) Exit Registry Editor, and then restart the computer.

More info can be found in this Microsoft knowledge base article: http://support.microsoft.com/kb/841215

http-webdav

Conclusion
Though not recommended for security reasons, it’s good to know there is a way to map a webdav drive when just HTTP (not HTTPS) is being used.

Categories
Admin Network Technologies Raspberry Pi Security

Bridging with the Raspberry Pi

Intro
Now I’m into playing with networking stuff on the Pi. So for a small investment I got a USB to ethernet adapter – $25 from Amazon. My first few experiments with it – turning it into a bridge – were largely successful.

The details
You need the bridge-utils package:

$ sudo apt-get install bridge-utils

For me it was easiest to connect to my Pi via the Wifi adapter I have on it (see this post describing how I used that to make a router). Then I could blow up the wired ethernet without disrupting my connection.

The first thing I noticed after plugging in the adapter is that it was automagically recognized and, for instance, the ifconfig -a command now shows an eth1 device. So no device driver needed to be installed, which was pretty sweet.

Listing the USB devices now looks like this:

$ lsusb

Bus 001 Device 002: ID 0424:9512 Standard Microsystems Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp.
Bus 001 Device 004: ID 0b95:1780 ASIX Electronics Corp. AX88178
Bus 001 Device 005: ID 7392:7811 Edimax Technology Co., Ltd EW-7811Un 802.11n Wireless Adapter [Realtek RTL8188CUS]

In short I want to take my two wired ports, eth0 and eth1, bridge them and send real traffic through them to see what happens. Bridging here means connecting two separate networks at a layer-2 level, so MAC addresses are proxied through the bridge and layer-3 communications occur transparently over the bridge as if it weren’t even there.

So…

$ sudo brctl addbr br0

creates a bridge named br0.

$ sudo brctl addif br0 eth0
$ sudo brctl addif br0 eth1

Adds those two physical ports to the bridge.

By the way I got all this from this reliable Linux Foundation source.

Now if I got it right, I should be able to unplug any wired device and put the Pi in between the switch and the device, no matter what its network is!

So I plugged my PC into one port, the other port into the switch the PC had been plugged into, and…

Well, for instance, showmacs output looks like this:

$ sudo brctl showmacs br0

port no mac addr                is local?       ageing timer
  1     00:00:00:0c:c8:1f       no                 1.30
  1     00:01:00:02:c8:1f       no               197.23
  2     00:02:cf:80:cc:99       no                 0.07
  2     00:90:a9:bb:3d:76       no                 0.94
  1     5e:00:00:fb:c8:1f       no               134.11
  1     5e:00:00:fd:c8:1f       no                57.49
  1     5e:7f:ff:fa:c8:1f       no                29.55
  2     64:66:b3:3b:bd:51       no                12.34
  1     8c:ae:4c:ff:27:69       yes                0.00
  2     b8:27:eb:dd:21:03       yes                0.00
  1     c8:1f:66:00:63:fe       no                 0.15

00:02:cf:80:cc:99 belongs to my upstream router, which is plugged into eth0, so I conclude that port 2 is eth0 and port 1 must be eth1. Maybe the port number was determined by the order in which I added the interface to the bridge?

Then I ran speedtest on my PC – there was no measurable slowdown in speed.

I loaded up the Pi with a cpu-intensive job:

$ yes > /dev/null

and re-ran speedtest. Still no hit to performance.

A brief review
I don’t think everyone will have appreciated what we’ve accomplished so let me review. We have created a passive, stealth man-in-the-middle (MITM) device with a management interface!

There are lots of uses for that, some not so noble. A more nefarious usage would be to connect up to a device and record all traffic passing back and forth to it (tcpdump -i br0). But there are also lots of good things we can achieve as well. For instance, Intrusion Detection System or Intrusion Prevention System (IDS/IPS). Firewall. Transparent proxy filter. IDS/IPS may be possible with Snort. I have to ask one of my security pals if that’s still the popular open source choice for IPS. Five years ago it was the package of choice.

What’s on my plate
Now I’d like to tinker with the Pi so that I can in fact slow down traffic and emulate slower connections! But I don’t know how yet…

I need to show how to make the bridging permanent using /etc/network/interfaces file.

To be continued…

Categories
Admin Internet Mail Linux

Sendmail: getting mailertable, smarttable and virtusertable to play nice together

Intro
Remember when I posted about some obscure but cool sendmail features? Well in that posting I mentioned having trouble with a catch-all mailertable entry co-existing with a smarttable. I recently got an incentive to work through that and I am sharing my results here. Plus use of virtusertable thrown in for good measure.

The setup
I have a Secure Mail Gateway (SMGW). Some of my users use it, most do not. So for the few who do I wish to divert my outgoing mail, if it sent by one of them, to the SMGW1. Everything else that’s outbound gets forwarded to an Internet-facing relay2. And for inbound mail I also want to divert their email over to it so it can do S/MIME or PGP decryption3. All other inbound emails should go to my native email system4.

So consider the outbound stream, requirement 1. Sender-based routing. That’s the normal thing in sendmail. That’s why you use smarttable, which Andrzej Filip has developed. See my references for more details on that. But my mailertable has a catch-all entry like:

.              relay:emgw.drj.com

I found if I got rid of this entry my smarttable entries began to work, but they stopped working if I put it back. I rolled up my sleeves and tried to understand ruleset 0. I became pretty convinced that mailertable runs before smarttable, and that if mailertable made a successful lookup of the recipie4nt domain that’s it. You’re done. Then I read a very brief comment in Andrzej’s writeup of smarttable. He bascially said mailertable runs first, and I guess form that brief comment you’re supposed to know that this setup I am trying will never work. And yet I have a colleague with a similar setup who says it does work, which got my competitive juices flowing – if the other guy can do it, then I know it can be done and I can do it as well! I knew I had to eliminate the catch-all entry, but still needed a catch-all feature. What to do? Fudge DNS to the point where all TLDs have a fake MX entry pointing at my external mail server?? Sounds too kludgy.

I was reading here and there in the Sendmail book and the cf/README file when SMARTHOST caught my eye. A smarthost can be defined to deliver all email to a specified relay. I always viewed it as an alternative to the mailertable, where you can specify much more specific delivery rules. But maybe they could co-exist? Mailertable for domain-specific delivery instructions, and smarthost for everything else? Yes, you can indeed do that! And with the catchall entry gone from mailertable does smarttable begin to work? Yes! It does! So our outbound stream is in good shape and all requirements are met. Each user of the SMGW is entered in the smarttable:

[email protected]    relay:smgw.drj.com
[email protected]    relay:smgw.drj.com
etc.

The mc configuration for smarthost looks like this:

dnl smarthost: to take care of the everything-else delivery case - DrJ 2/28/14
define(`SMART_HOST',`internet-facing-relay.drj.com')

Inbound
For inbound we can use virtusertable to rewrite the user domain to a fictional domain, then a mailertable entry which describes that this fictional domain should be routed over to the SMGW! Like this.

Virtusertable

and

Mailertable

drj.com   relay:native-mail-system-gw.drj.com
.smgw     relay:smgw.drj.com

Although this sounded good on paper, I found it was not enough by itself. In addition I needed to throw in a virtuser domain file which included the domain drj.com.

The mc configuration for virtusertable and virtuserdomain look like this.

dnl
dnl add virtusertable to do recipient re-writing to accomodate SMGW routing -DrJ 3/3/14
FEATURE(`virtusertable',`hash -o /etc/mail/virtusertable')dnl
dnl virtuser domain file location - DrJ 2/27/14
VIRTUSER_DOMAIN_FILE(`/etc/mail/virtuserdomains')dnl

virtuserdomains is a standard text file which contains drj.com.

Testing
It is helpful to know how to debug this stuff. Smarttable is probably the hardest. Here’s how to see the before and after effect that smarttable has:

$ sendmail -Csendmail-test.cf -bt<<END
> 3,0 [email protected]
> [email protected]
> 3,0 [email protected]
> END

The 3,0 lines show the result of running rulesets 3 and 0, I guess. The .Df…line defines the sender, so the last 3,0 line shows how delivery to the sender is now altered if the defined sender exists in the smarttable table.

Regular addresses can be tested with the same -bt switch if you want to see the gory details, or simply -bv:

$ sendmail -Csendmail-test.cf -bt< 3,0 [email protected]
> END

$ sendmail -Csendmail-test.cf -bv [email protected]

It’s assumed you put your sendmail config in sendmail-test.cf to not interfere with production.

Even if all the tests succeed, what I found is that smarthost did not take effect dynamically. I needed to re-start sendmail.

Conclusion
By digging into the innards of sendmail we learned enough to see how things should work together and found that it is indeed possible for smarttable, virtusertable and mailertable to peacefully co-exist, but only with a helping of smarthost and virtuserdomains!

References
I describe smarttable here.
Andrzej’s smarttable page is here.

Categories
Admin Linux Network Technologies Raspberry Pi

Using your Raspberry Pi as a router

Intro
Most Raspberry Pi router HowTos describe how to make the Pi act like an Access Point. That can be very useful, and kind of tricky. Here I’ve turned wireless and wired network roles around and show how to make it route traffic from a wired LAN over its WiFi connection.

I bought the EDIMAX nano USB adapter to play around with wireless on my Pi. It wasn’t long before the network lover in me realized, Hey I got an ethernet port on one end, a wireless on the other – that sounds like a potential router – let’s have some fun! So I hooked up my cantankerous Sony Blueray player to it. The thing has always been touchy about using wireless but it also has a wired LAN connection option – in a room where I don’t have a wired ethernet network available. Enter the routing Pi…

WiFi on the Pi
There are many ways to get the thing going. The important thing to note is that this EDIMAX is tested and is known to be compatible with the Raspberry Pi. I guess they even included the appropriate driver for it, because there is no need at any time to read the mini-CD that the EDIMAX comes with and try to pull of, or worse, compile, a Linux driver like I initially feared. I think you merely need to do one of these numbers:

$ sudo apt-get update

to ensure you have the latest of everything on your Pi and that includes the driver for this WiFi adapter.

Also note that the Pi, being a tiny computer, is perfectly complemented by this nano adapter. The thing is so tiny you barely have enough surface area to pull it out of the USB slot.

As I said there are probably many ways to get your WiFi going. Being a command line lover I present that way, and even then there are alternative setups to choose from, of which I present only one here.

If you do one of these numbers:

$ cd /etc/network; sudo nano interfaces

I would recommend to add a line towards the top:

auto wlan0

and at the bottom:

# following http://antael.blogspot.com/2013/01/wifi-is-live-and-kicking.html
# to see Wifi signal strength and available signals run iwlist wlan0 scan
#allow-hotplug wlan0
iface wlan0 inet static
wpa-ssid 
wpa-psk 
wpa-key_mgmt WPA-PSK
address 192.168.0.90
gateway 192.168.0.254
netmask 255.255.255.0
broadcast 192.168.0.255

substitute the name of your WiFi SSID and the password (WPA Pre-Shared Key) in place of <MYSSID> and <MYSSIDPASSWORD>.

I prefer static IPs to dynamic ones. That way I ssh to the IP and it is the same every time.

As I am setting up a router this is not my complete interfaces file, just the part I want to emphasize for now. The complete file is listed below.

A word on what we are setting out to do – our network architecture

OK. So I am going to put the Pi next to the Blueray Player. The Pi will connect to my TP-LINK WiFi network. The Pi’s ethernet interface will connect to the LAN port on the Blueray player using a standard ethernet patch cable (these days it is not necessary to use a crossover cable as there’s always a device that auto-senses how it is wired). I will create a new network for this interface on the Pi, and make sure I’ve assigned a valid IP on this network to the Blueray player. The network I choose is 10.31.42.0/24 – something completely different from anything I may already have in use at home.

The Blueray player can also be assigned a static IP address. I give it 10.31.42.13 and the Pi I assign 10.31.42.11. The gateway of the Blueray player is our routing Pi, so that makes the gateway IP 10.31.42.11.

But I have another requirement: I want to conveniently put the Pi back on my wired home network in case something goes wrong with wireless. so I want to keep its valid static IP (192.168.2.100) which it used to have when it was wired to my switch. The way to accomplish all this is to use virtual IPs on the eth0 interface. See the full /etc/network/interfaces file below which shows eht0 still assigned to IP 192.168.2.200, but as well a virtual interface eth0:0 assigned to 10.31.42.11.

Convenient way to test things
Initially I was focused on the architecture I’ve outline above and getting all my interfaces up and running. And I could run, for instance,

$ sudo service networking restart

to make my changes to the interfaces file be dynamically enabled, and I can do a

$ ifconfig -a

to show all my interfaces, their IPs and other information. And I especially like

$ iwlist wlan0 scan

to show the available WiFi networks and their signal strength. But how do I know that routing is working?? Here’s how. You know the ping command, right? Normally you do

$ ping 8.8.8.8

to show you can reach the Internet. Why? because that’s a valid IP address on the Internet that responds to PING – thank you Google – and it’s easy to remember!

But since our default route is out of our WiFi-connected interface, the IP it picks for the source of that PING is the IP assigned to wlan0, namely 192.168.0.90. There’s no Pi-routing involved in that so far. But now, you can choose a different source IP for your pings. So we pick our new virtual IP, 10.31.42.11 like this:

$ ping -I 10.31.42.11 8.8.8.8

but instead of a nice result like

PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_req=1 ttl=44 time=172 ms
64 bytes from 8.8.8.8: icmp_req=2 ttl=44 time=92.7 ms
64 bytes from 8.8.8.8: icmp_req=3 ttl=44 time=69.6 ms
64 bytes from 8.8.8.8: icmp_req=4 ttl=44 time=70.6 ms
64 bytes from 8.8.8.8: icmp_req=5 ttl=44 time=70.3 ms
64 bytes from 8.8.8.8: icmp_req=6 ttl=44 time=70.5 ms
^C
--- 8.8.8.8 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5006ms
rtt min/avg/max/mdev = 69.676/91.121/172.740/37.409 ms

we don’t get much encouragement:

PING 8.8.8.8 (8.8.8.8) from 10.31.42.11 : 56(84) bytes of data.
^C
--- 8.8.8.8 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3000ms

The ^C above means <CTRL-C> was typed in at the keyboard.

So we’re seeing 100% packet loss and we can safely conclude no routing is occurring. By the way, as a reality and typo check on this approach, try to specify as source your wlan0 IP, as in

$ ping -I 192.168.0.90 8.8.8.8

This should work.

Turn on routing
I feared it would be mess to get routing turned on, but it turns out to be pretty tidy, considering. Probably your Pi is already enabled for routing. Mine was. Just

$ cat /proc/sys/net/ipv4/ip_forward

and make sure the value is 1. That means it is set up to do routing. If you’re not so lucky and you have 0, enable routing:

$ sudo sysctl -w net.ipv4.ip_forward=1

But of course there’s more we have to do or else our source ping would have worked!

We need to turn on network address translation. This is the part I was worried about, never having done it before on Linux, but only with expensive products like commercial firewalls. But it’s really not bad at all.

I created a file iptables-NAT in the home directory of the pi user with these contents:

#!/bin/sh
# This is a one-time script - DrJ
# 2/2014
# explained nicely in http://www.karlrupp.net/en/computer/nat_tutorial
# and seems to even work!
 
# flush old iptables stuff. Need to specify nat table specifically
iptables -t nat -F
iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE
 
# make it permanent
#/usr/sbin/service iptables save
 
# list it
iptables -t nat -L
 
# persist it:
 
iptables-save &gt; /etc/iptables.conf
 
# note that in /etc/network/interfaces we added this line to read in
# iptables.conf upon reboot:
# pre-up iptables-restore &lt; /etc/iptables.conf

and ran it:

$ cd; sudo ./iptables-NAT

It outputs this:

Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination
 
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
 
Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
 
Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
MASQUERADE  all  --  anywhere             anywhere

Now run our source ping:

$ ping -I 10.31.42.11 8.8.8.8

PING 8.8.8.8 (8.8.8.8) from 10.31.42.11 : 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_req=1 ttl=44 time=76.3 ms
64 bytes from 8.8.8.8: icmp_req=2 ttl=44 time=71.4 ms
^C
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 71.471/73.922/76.373/2.451 ms

Voila! It worked. We now are NATing our source address of 10.31.42.11 to our wlan IP of 192.168.0.90 before sending the packet along to the next router. But as skeptics, we’d like to see some more empirical proof that all that is really happening. You can. Here’s how using tcpdump.

The thing is that tcpdump doesn’t have full access to pre-processing, it only has “post processing” access. So we need to hook up the Pi to a device we are going to route traffic for, in my case the Sony Blueray player. I don’t have much control, but I can do a network diagnostics which I know tries to reach the configured DNS server (because it complained that it could not reach the DNS server 8.8.8.8 during one of my early tests. So tee up tcpdump to monitor all interfaces’ traffic to 8.8.8.8 like this:

$ sudo tcpdump -n -i any host 8.8.8.8

Then run ping (or a DNS query) to 8.8.8.8 on that device, and voila, this is the result:

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
20:22:09.041922 IP 10.31.42.13.54693 &gt; 8.8.8.8.53: 5+ A? m.root-servers.net. (36)
20:22:09.042144 IP 192.168.0.90.54693 &gt; 8.8.8.8.53: 5+ A? m.root-servers.net. (36)
20:22:09.163618 IP 8.8.8.8.53 &gt; 192.168.0.90.54693: 5 1/0/0 A 202.12.27.33 (52)
20:22:09.163777 IP 8.8.8.8.53 &gt; 10.31.42.13.54693: 5 1/0/0 A 202.12.27.33 (52)

The -i any switch told it to listen on all interfaces. Unfortunately the output doesn’t show which interface, but you can easily deduce it. the traffic comes in from my Blueray player at 10.31.42.13 on eth0, and then leaves the Pi on wlan0 having had its source IP translated to 192.168.0.90. And the reverse happens to the response from 8.8.8.8. In fact I see from this tcpdump output that the Blueray player does not do a PING, it does an actual DNS query to 8.8.8.8. Makes sense since it is a DNS server.

Be persistent
But a reboot will undo this nice NATing we have achieved. Try it. So we have to find a way to make our iptables entry persist across reboots. I’m sure there are many ways to do this. I liked this one. You add this line to the end of your interfaces file:

pre-up iptables-restore &lt; /etc/iptables.conf

And of course we already anticipated this manner of proceeding in our iptables-NAT file when we included this line at the bottom:

iptables-save &gt; /etc/iptables.conf

Now reboot again, and try that source ping as the first command you issue. Now it should work.

Ready for the real test
Now I can hook up my Blueray player with some confidence that at least the networking should be working. Assign its IP (10.31.42.13 as mentioned above), its gateway (the Pi’s eth0:0 virtual IP, namely 10.31.42.11 in our example), a DNS server (8.8.8.8, of course!) and see if:

– we can ping 10.31.42.13 from the Pi
– the Blueray player’s network tester shows OK (there aren’t many fine debugging utilities, just this single “network test”)

If the Blueray player can’t reach the configured DNS server then it its test fails.

Why this big long explanation for something that no one else in the world wants to do?
All this sounds like a very, very specific application that doesn’t apply to anyone else. But once you understand some of the core networking principles involved that I’ve touched on here, you will begin to realize that this can be very easily generalized to something much more applicable and powerful: a full-blown replacement for a standard wireless router such as my TP-LINK nano router. After all, we’ve set up almost all the utilities and facilities that you get from a standard router (NATing and routing), perhaps with one glaring exception: a DHCP service. I personally didn’t want one, but nothing prevents you from setting that up on the wired side. I give some suggestions below on how to do that in the next section.

A few words on a DHCP service
I did get that running on the Pi as well, though for an entirely different purpose. I used dnsmasq:

$ sudo apt-get install dnsmasq

and I think if you edit /etc/dnsmasq.conf and put these lines at the bottom:

interface=eth0
dhcp-range=10.31.42.14,10.31.42.254,10h
dhcp-option=3,10.31.42.11

I think you will pretty much have a working DHCP service as well and could use that instead of assigning static IPs.

What I fear about the DHCP service – and I think I have seen this – is that if I put the Pi back to the wired network, its DHCP server competes with the normal router’s DHCP service, and I start to lose connectivity to my devices! So be careful. If devices like your PC start to pick up 10.31.42.x addresses on a network where 192.168.2.x is expected, there could be some connectivity troubles ahead until you disconnect the Pi!

Finally, the full /etc/network/interfaces file

auto lo
auto eth0
auto eth0:0
auto wlan0
 
iface lo inet loopback
# DrJ change: make IP static
# somewhat inspired by http://www.techiecorner.com/486/how-to-setup-static-ip-in-debian/ - DrJ 1/8/13
#iface eth0 inet dhcp
iface eth0 inet static
address 192.168.2.100
#gateway  192.168.2.1
netmask 255.255.255.0
network 192.168.2.0
broadcast 192.168.2.255
 
# for network 3142
iface eth0:0 inet static
address 10.31.42.11
netmask 255.255.255.0
network 10.31.42.0
broadcast 10.31.42.255
 
# following http://antael.blogspot.com/2013/01/wifi-is-live-and-kicking.html
# to see Wifi signal strength and available signals run iwlist wlan0 scan
#allow-hotplug wlan0
iface wlan0 inet static
wpa-ssid 
wpa-psk 
wpa-key_mgmt WPA-PSK
address 192.168.0.90
gateway 192.168.0.254
netmask 255.255.255.0
broadcast 192.168.0.255
# wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
#iface default inet dhcp
 
# to read in iptable configuration. See ~pi/iptables-NAT
# added - DrJ 3/2/14
pre-up iptables-restore &lt; /etc/iptables.conf

Conclusion
Yes, we got our Pi to act as a router, and it wasn’t too bad. We demonstrated it for this specific application, but also showed how the simple addition of a DHCP service would make this a very general solution.
Did it help with the problem at hand – getting smoother streaming on the Sony Blueray player? Well, actually, yes, it did seem to help.

References and related

An idea for temporarily replacing a busted home router with a Raspberry Pi router which uses your hotspot is described in this article.

Getting started on a Pi without a dedicated console is described here.
A look at playing around with real-time video using the Pi’s camera is described here.
A more esoteric project, using the Pi to monitor your home’s Internet/power connection, is presented here.
Turning your Raspberry Pi into a transparent networking bridge is described here.
Interested to run some real networking protocols on your Pi like RIP, OSPF or BGP? I suggest to look into Quagga. Quagga is a networking suite. In full disclosure I haven’t had time or motivation to experiment with it myself (yet).

Categories
Admin Linux

Linux command line tips

Intro
I love the Linux command line. I don’t know how to categorize the tips I have collected so I’m dumping them here for the time being.

The newer netstat
In the old days I used to constantly do

$ netstat -an|grep LISTEN|grep <PORT>

to see what tcp ports I had processes listening on. While that is very helpful, it doesn’t tell you the listening process. A better approach:

$ lsof -i tcp:<PORT>

where is 22, 80 or whatever you want to look for. lsof does show the process.

Repeat last argument

I am constantly doing things like

$ grep <string> <big-file> > /tmp/results

and then I want to have a look at that file but I don’t want to type in

$ more /tmp/results

so what do I do?

I type this to give me the last argument:

$ more !$

My fingers have memorized that pattern and so I do it without conscious thought, and you’re holding down the SHIFT key so it can be typed very quickly.

But sometimes I can’t wait to type even that. I want to collect the grep results into a file in case there were a lot of matches, but I also want to see the matching results on my screen as soon as they are available. What do I do? Use tee, as in:

$ grep <string> <big-filez> |tee /tmp/results|more

Cool, huh?

More Obscure stuff
A cool blog with lots of good, obscure tips, generally more technical than the ones above, is https://die-computerhilfe.de/blog

To be continued…

Conclusion