Author: john

Categories
DNS

The IT detective agency: rogue IPv6 device messes up DHCP for entire subnet

Intro
This was a fascinating case insofar as it was my first encounter with a real life IPv6 application. So it was trial by fire.

The details
I think the title of the post makes clear what happened. The site people were saying they can ping hosts by IP but not by DNS name. So basically nothing was working. I asked them to do an ipconfig /all and send me the output. At the top of the list of DNS servers was this funny entry:

IPv6 DNS server shows up first

I asked them to run nslookup, and sure enough, it timed out trying to talk to that same IPv6 server. Yet they could PING it.

The DNS servers listed below the IPv6 one were the expected IPv4 our enterprise system hands out.

My quick conclusion: there is a rogue host on their subnet acting as an IPv6 DHCP server! It took some convincing on my part before they got on board with that idea.

But I goofed too. In my haste to move on, I confused an IPv6 address with a MAC address. Rookie IPv6 mistake I suppose. It looked strange, had letters and even colons, so it kind of looks like a MAC address, right? So I gave some quick advice to get rid of the problem: identify this address on the switch, find its port and disable it. So the guy looked for this funny MAC address and of course didn’t find it or anything that looked like it.

My general idea was right – there was a rogue IPv6 DHCP server.

My hypothesis as to what happened
The PCs have both an IPv4 as well as an IPv6 stack, as does just about everyone’s PC. These stacks run independently of each other. Everyone blissfully ignores the IPv6 communication, but that doesn’t mean it’s not occurring. I think these PCs got an IPv4 IP and DNS servers assigned to them in the usual way. All good. Then along came a DHCPv6 server and the PC’s IPv6 stack sent out a DHCPv6 request to the entire subnet (which it probably is doing periodically all along, there just was never a DHCPv6 server answering before this). This time the DHCPv6 server answered and gave out some IPv6-relevant information, including a IPv6 DNS server.

I further hypothesize that what I said above about the IPv4 and IPv6 stacks being independent is not entirely true. These stacks are joined in one place: the resolving nameservers. You only get one set of resolving namesevrers for your combined IPv4/IPv6 stacks, which sort of makes sense because DNS servers can answer queries about IPv6 objects if they are so configured. So, anyway, the DHCPv6 client decides to put the DNS server it has learned about from its DHCPv6 server at the front of the existing nameserver list. This nameserver is totally busted, however and sits on the request and the client’s error handling isn’t good enough to detect the problem and move on to the next nameserver in the list – an IPv4 nameserver which would have worked just great – despite the fact that it is designed to do just that. And all resolution breaks and breaks badly.

What was the offending device? They’re not saying, except we heard it was a router, hence, a host introduced by the LAN vendor who can’t or won’t admit to having made such an error, instead making a quiet correction. Quiet because of course they initially refused the incident and had us look elsewhere for the source of the “DHCP problem.”

Alternate theory
I see that IPv6 devices do not need to get DNS servers via DHCPv6. They can use a new protocol, NDP, neighbor discovery protocol. Maybe the IPv6 stack is periodically trying NDP and finally got a response from the rogue device and put that first on the list of nameservers. No DHCPv6 really used in that scenario, just NDP.

Useful tips for layer 2 stuff
Here’s how you can find the MAC of an IPv6 device which you have just PINGed:

netsh interface ipv6 show neighbors

from a CMD prompt on a Windows machine.

In Linux it’s

ip ‐6 neigh show

Conclusion
Another tough case resolved! We learn some valuable things about IPv6 in the process.

References and related
I found the relevant commands in this article: https://www.midnightfreddie.com/how-to-arp-a-in-ipv6.html

Categories
Web Site Technologies

Where is my IP without the aggressive ads

Intro
To locate where any IP address is located – known as geoip – you can do a simple duckduckgo search and get an idea, but you may also get sucked into one of those sites that provides a service while subjecting you to a lot of advertising. So I prefer to have the option to go to the source.

For that I kind of like this site: https://www.maxmind.com/en/geoip-demo

Maxmind also has a free downloadable database of all IPs known as GeoLite2. If I get time I may explore using it.

References and related
https://www.maxmind.com/en/geoip-demo

Categories
Consumer Tech

What credit card fraud looks like

Intro
A lost credit card. Or was it misplaced? Months later a whole bunch of “modest” charges appear all at once, a couple days after a few lower-value test charges were made.

Thank goodness I had the presence of mind to lower my alert limit on transactions from $200 to $50. I was not too late to have all the charges disputed despite the test charges being two days old.

What fraudulent credit card charges look like

I do not know how these charges were created – what is Google/Walmart or Google/Target.

I can guess how my address was matched to the card – it’s a sufficiently uncommon last name available from simple public records.

Conclusion
Well, now we know the card was lost or stolen, not misplaced. After all these years that’s the first time that has happened to us. We will not be responsible for the disputed charges.

Categories
Uncategorized

NJ homeowners: how to sell your SRECs from your solar panels

Intro
I was an enthusiast and got solar panels on my roof while there was still a tax credit for doing so. But then i became lazy and didn’t want to bother selling the SRECs I was awarded. Here is what I did.

The details
I got a recommendation from a friend who found a legitimate company who will buy my SRECs with a process so simple no registration is required! And, their prices seem competitive.

Here are the CEPS I’ve accumulated on the PJM-EIS web site. And no, I don’t really know how to use the site other than to report my generation. I just wasn’t that interested.

CEPS from Dr John’s home solar system

CEPS is a synonym for SRECs. SREC is a solar renewable energy credit. It’s a unit of measure = 1 Kilowatt Hour of generation by your system.

Here is the web site of the company I will sell them to: http://njsrec.com/

And their instructions – clearly written for someone not overly familiar with using a computer as everything is spelled out:

NJSREC.COM instructions

I haven’t sold them yet because I will have another one by tomorrow so I’ll wait for that one and bundle them all together. They get credited to your account on the last day of the month. My friend uses them however so I know they are to be trusted. They will simple send you a check in the mail for your CEPS after you follow those simple instructions!

Conclusion
We recommend NJSREC.COM as the simplest way to sell your SRECs and know you are not being taken advantage of. As of this writing July 2018 a quantity of 4 – 10 CEPS is worth $201 per CEPS. The prices have been going down (mostly) and will continue to go down. So don’t hold on too long, i.e., years.

References and related
The GATS web site is https://www.pjm-eis.com/
The buyer’s web site: http://njsrec.com/

Categories
Admin Linux Network Technologies SLES

Linux tip: how to enable remote syslog on SLES

Intro
I write this knowing I still don’t know anything to speak of about syslog, but, sometimes you gotta act without knowing. I needed to send syslog to somewhere in a big hurry so I figured out the absolute minimum I needed to do to get it running on one of my other systems.

The details
This all started because of a deficiency in the F5 ASM. At best it’s do slow when looking through the error log. But in particular there was one error that always timed out when I tried to bring up the details, a severity 5 error, so it looked pretty important. Worse, local logging, even though it is selected, also does not work – the /var/log/asm file exists but contains basically nothing of interest. I suppose there is some super-fancy and complicated MySQL command you could run to view the logs, but that would take a long time to figure out.

So for me the simplest route was to enable remote syslog on a Linux server and send the ASM logging to it. This seems to be working, by the way.

The minimal steps
Again, this was for Suse Enterprise Linux running syslog-ng.

  1. modify /etc/sysconfig/syslog as per the next step
  2. SYSLOGD_PARAMS=”-r”
  3. modify /etc/syslog-ng/syslog-ng.conf as per the next step
  4. uncomment this line: udp(ip(“0.0.0.0”) port(514));
  5. launch yast (I use curses-based yast [no X-Windows] which is really cantankerous)
  6. go to Security and Users -> Firewall -> Allowed services -> Internal Zone -> Advanced
  7. add udp port 514 as additional allowed Ports in internal zone and save it
  8. service syslog stop
  9. service syslog start
  10. You should start seeing entries in /var/log/localmessages as in this suitably anonymized example (I added a couple line breaks for clarity:
Jul 27 14:42:22 f5-drj-mgmt ASM:"7653503868885627313","50.17.188.196","/Common/drjohnstechtalk.com_profile","blocked","/drjcrm/bi/tjhmore345","0","Illegal URL,Attack signature detected","200021075","Automated client access ""curl""","US","<!--?xml version='1.0' encoding='UTF-8'?-->44e7f1ffebff2dfb-800000000000000044f7f1ffebff2dfb-800000000000000044e7f1ffe3ff2dfb-80000000000000000000000000000000-000000000000000042VIOL_ATTACK_SIGNATURErequest200021075
7VXNlci1BZ2VudDogY3VybC83LjE5LjcgKHg4Nl82NC1yZWRoYXQtbGludXgtZ251KSBsaWJjdXJsLzcuMTkuNyBOU1MvMy4yNy4xIHpsaWIvMS4yLjMgbGliaWRuLzEuMTggbGlic3NoMi8xLjQuMg0KSG9zdDogYWctaW50ZWw=
01638
VIOL_URL","GET /drjcrm/bi/tjhmore345 HTTP/1.1\r\nUser-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.27.1 zlib/1.2.3 libidn/1.18 libssh2/1.4.2\r\nHost: drjohnstechtalk.com\r\nAccept: */*\r\n\r\n"

Observations
Interestingly, there is no syslogd on this particular system, and yet the “-r” flag is designed for syslogd – it’s what turns it into a remote syslogging daemon. And yet it works.

It’s easy enough to log these messages to their own file, I just don’t know how to do it yet because I don’t need to. I learn as I need to. just as I learned enough to publish this tip.

Conclusion
We have demonstrated activating the simplest possible remote syslogger on Suse Linux Enterprise Server.

References and related

Want to know what syslog is? Howtonetwork has this very good writeup: https://www.howtonetwork.com/technical/security-technical/syslog-server/

Categories
Security

Suppress /apple-touch-icon URLs on an F5 ASM

Intro
Displaying the ASM event log is slow – it can take minutes on our older equipment. So anything that helps cut out the clutter in the returned log entries may save precious minutes of, e.g., paging to the next screen (also a minute). At some point I realized the logs were mostly filled with complaints about illegal URLs beginning with /apple-touch-icon… So i found a way to suppress those. This is for version 12.1

The problem

Typical example from a typical WAF log

The details

LTM policy to suppress those entries

How to edit policy
These are policies in the Local Traffic section. It’s not that intuitive. Clicking on the policy name will give you a read-only view and no evident way to switch to an edit mode. What you do is click on Create Draft. That creates a “draft policy” which you can edit. There you can introduce the rule above. Drag it to the top. Hit Save and publish draft and it should go live.

The best way?
It’s debatable if this is the best way to suppress these. if they come from legitmate devices mistakenly asking for these URLs it’s probably nicer to send them a 404 Not found. An iRule would be required for that.

Conclusion
We show how to suppress annoying ASM log entries saying illegal URL, /apple-touch-icon… on an F5 web application firewall. What is producing these URL attempts I just don’t know at this point. I suspect them to be innocuous.

Categories
Raspberry Pi SLES Web Site Technologies

Pi-hole: it’s as easy as pi to get rid of your advertisements

Intro
I learned about pi-hole from Bloomberg Businessweek of all places. Seems right up my alley – uses Raspberry Pi in your home to get rid of advertisements. Turns out it was too easy and I don’t have much to contribute except my own experiences with it!

The details
When I read about it I got to thinking big picture and wondered what would prevent us from running an enterprise version of this same thing? Well, large enerprises don’t normally run production critical applications like DNS servers (which this is, by the way) on Raspberry Pis, which is not the world’s most stable hardware! But first I had to try it at home just to learn more about the technology.

pi-hole admin screen

I was surprised just how optimized it was for the Raspberry Pi, to the neglect of other systems. So the idea of using an old SLES server is out the window.

But I think I got the essence of the idea. It replaces your DNS server with a custom one that resolves normal queries for web sites the usual way, but for DNS queries that would resolve to an Ad server, it clobbers the DNS and returns its own IP address. Why? So that it can send you a harmless blank image or whatever in place of an Internet ad.

You know those sites that obnoxiously throw up those auto-playing videos? That ain’t gonna happen any more when you run pi-hole.

You have to be a little adept at modifying your home router, but they even have a rough tutorial for that.

Installation
For the record on my Rspberry Pi I only did this:
$ sudo su ‐
$ curl ‐sSL https://install.pi‐hole.net | bash

It prompted me for a few configuration details, but the answers were obvious. I chose Google DNS servers because I have a long and positive history using them.

You can see that it installs a bunch of packages – surprisingly many considering how simple in theory the thing is.

Test it
On your Raspberry Pi do a few test resolutions:

$ dig google.com @localhost # should look like it normally does
$ dig pi.hole # should return the IP of your Raspberry Pi
$ dig adservices.google.com # I gotta check this one. Should return IP address of your Pi

It runs a little web server on your Pi so the Pi acts as adservices.google.com and just serves out some white space instead of the ad you would have gotten.

Linksys router
Another word about the home router DHCP settings. You have the option to enter DNS server. So I put the IP address of my raspberry pi, 192.168.1.119. What I expected is that this is the DNS server that would be directly handed out to the DHCP clients on my home network. But that is not the case. Instead it still hands out itself, 192.168.1.1 as DNS server. But in turn it uses the raspberry PI for its resolution. This through me when I did an ipconfig /all on my Windows 10 and didn’t see the DNS server I expected. But it wa all working. About 10% of my DNS queries were pi-holed (see picture of my admin screen above).

I guess pi-hole is run by fanatics, because it works surprisingly well. Those complex sites still worked, like cnn.com, cnet.com. But they probably load faster without the ads.

Two months check up
I checked back with pihole. I know a DNS server is running. The dashboard is broken – the sections just have spinning circle instead of data. It’s already asking me to upgrade to v 3.3.1. I run pihole -up to do the upgrade.

Another little advantage
I can now ssh to my pi by specifying the host as pi.hole – which I can actually remember!

Idea for enterprise
finally, the essence of the idea probably could be ported over to an enterprise. In my opinion the secret sauce are the lists of domain names to clobber. There are five or six of them. Some have 50,000 entries. So you’d probably need a specialized DNS server rather than the default ISC BIND. I remember running a specialized DNS server like that when I ran Puremessage by Sophos. It was optimized to suck in real-time blacklists and the like. I have to dig through my notes to see what we ran. I’m sure it wasn’t dnsmasq, which is what pi-hole runs on the Raspberry Pi! But with these lists and some string manipulation and a simple web server I’d think it’d be possible to replicate in enterprise environment. I may never get the opportunity, more for lack of time than for lack of ability…

Conclusion
Looking for a rewarding project for your Raspberry Pi? Spare yourself Internet advertisements at home by putting it to work.

References and related
The pi-hole web site: https://pi-hole.net/
Another Raspberry Pi project idea: monitor your cable modem and restart it when it goes south.

Categories
CentOS

Docker on CentOS 6: FAIL

Intro
I think I’ve been running CentOS 6 for six years now. With upgrades I’m at v 6.9. It was nice and new six years ago, but not so much anymore. I think it showed its age when I tried to run Docker. It just wouldn’t fly an I don’t feel like debugging it. The conventional wisdom seems to be that it is not possible – the two are incompatible, which further discouraged me.

Here are my notes

followed https://www.liquidweb.com/kb/how-to-install-docker-on-centos-6/ - DrJ 6/18/18
 
my problem:
level=error msg="[graphdriver] prior storage driver \"devicemapper\" failed: Error running DeviceCreate (CreatePool) dm_tas
k_run failed"
https://github.com/moby/moby/issues/13008
 
 
docker -d -D
...
DEBU[0000] Warning: could not change group /var/run/docker.sock to docker: Group docker not found
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)
WARN[0000] You are running linux kernel version 2.6.32-131.17.1.el6.x86_64, which might be unstable running docker. Please
upgrade your kernel to 3.10.0.
DEBU[0000] devicemapper: driver version is 4.20.6
DEBU[0000] Generated prefix: docker-202:65-131224
DEBU[0000] Checking for existence of the pool 'docker-202:65-131224-pool'
DEBU[0000] Pool doesn't exist. Creating it.
DEBU[0000] Error retrieving the next available loopback: open /dev/loop-control: no such file or directory
DEBU[0000] Error retrieving the next available loopback: open /dev/loop-control: no such file or directory
ERRO[0000] [graphdriver] prior storage driver "devicemapper" failed: Error running DeviceCreate (CreatePool) dm_task_run fa
iled
FATA[0000] Error starting daemon: error initializing graphdriver: Error running DeviceCreate (CreatePool) dm_task_run faile
d
john@ip-10-164-80-53:~$ docker -v
Docker version 1.7.1, build 786b29d/1.7.1
 
 
trying:
 
yum install --setopt=obsoletes=0 \
   docker-ce-17.03.2.ce-1.el7.centos.x86_64 \
   docker-ce-selinux-17.03.2.ce-1.el7.centos.noarch
(from https://github.com/docker/for-linux/issues/20)

Conclusion
Running docker on CentOS v 6.9 in 2018 may be impossible or so difficult as to be practically impossible. I give up.

Categories
Linux

Future project idea: Interplanetary file system, IPFS

Intro
If I had more time and more energy, what I’d like is to explore the Interplanetary Filesystem, perhaps put up a server and create some objects. It seems right up my alley as I was an early adapter and put up one of the first web servers on the Internet. IPFS combines a lot of m interests: Linux (it extends the filesystem), web and computer science concepts.

But I don’t have spare time. Maybe later this year…

One year later
OK, I just heard on the Reveal podcast about an invidious use for IPFS. Although I was too busy before, now I’m simply loathe to embrace a technology that’s been embraced by the alt-right. Specifically, i have heard that Rob Monster has stored hate speech manifestos using IPFS and thereby using IPFS to create what is being called an alt-tech stack (their own web browser, social media sites, streaming video sites, etc, etc). It’s a shame.

References and related
Here’s the IPFS web site: https://ipfs.io/
This lecture explains why we need this improved web technology and what it is: https://www.youtube.com/watch?v=HUVmypx9HGI

Categories
Network Technologies Security

LDAP authentication on the F5 BigIP without Access Policy Manager

Intro
I recently received revised guidelines for dmz best practices which mentioned a requirement to implement application-independent authentication using the F5 web application firewall. I had never heard of it and didn’t think it was possible without buying the very expensive APM license. They insisted it was possible and even easy to do. So I investigated and found they were right!

The details
This is a feature added around version 11.4.

On the F5, go to Local Traffic|Profiles|Authentication|Configurations and create a new configuration. Here you put in the essential LDAP information and give these settings a name such as myLDAP. I needed to set Login Attribute to cn. Then go to …Authentication|Profiles and create a new one. Set parent profile as LDAP and associate the configvuration myLDAP to it. Rule can be _sys_auth_ldap.

In the virtual server Properties tab look for the section Authentication Profiles. Pick the profile you created.

That’s it! Your virtual server now has application-independent authentication using your preferred LDAP source.

So far I only tested against an LDAP source that doesn’t require an ldap bind. But I did successfully test against an ldaps source (which runs on port 636 and encrypts the communication using SSL. I got that to work setting SSL to Enabled and essentially taking the other SSL-related default values.

Conclusion
We show how to implement application-independent authentication on an F5 BigIP which only has the local traffic manager (LTM) license. We used an LDAP directory for the authentication source. I believe a certificate mechanism would also have been possible. As it happens our LDAP source was not an Active Directory (AD) tree, but I believe it would be possible to use that as well. We also did not limit access to any specific group, but that is probably possible as well.