Categories
Admin CentOS Linux Raspberry Pi

A few RPM and YUM commands and equivalent on Raspberry Pi

Intro
This post adds nothing to the knowledge out there and readily available on the Internet. I just got tired of looking up elsewhere the few useful rpm and yum commands that I employ. Here’s how I installed a missing binary on one system when I have a similar system that has it.

RPM is the Redhat Package Manager. It is also used on Suse Linux (SLES). A much better resource than this page (Hey, we can’t all be experts!) is http://www.idevelopment.info/data/Unix/Linux/LINUX_RPMCommands.shtml

List all installed packages:

$ rpm −qa
dmidecode-2.11-2.el6.x86_64
libXcursor-1.1.10-2.el6.x86_64
basesystem-10.0-4.el6.noarch
plymouth-core-libs-0.8.3-24.el6.centos.x86_64
libXrandr-1.3.0-4.el6.x86_64
ncurses-base-5.7-3.20090208.el6.x86_64
python-ethtool-0.6-1.el6.x86_64

Same as above – list all installed packages – but list the most recently installed packages first (Wish I had discovered this command sooner)!

$ rpm −qa −−last

libcurl-devel-7.19.7-35.el6                   Mon Apr  1 20:00:47 2013
curl-7.19.7-35.el6                            Mon Apr  1 20:00:47 2013
libidn-devel-1.18-2.el6                       Mon Apr  1 20:00:46 2013
libcurl-7.19.7-35.el6                         Mon Apr  1 20:00:46 2013
libssh2-1.4.2-1.el6                           Mon Apr  1 20:00:45 2013
ncurses-static-5.7-3.20090208.el6             Mon Apr  1 19:59:24 2013
ncurses-devel-5.7-3.20090208.el6              Mon Apr  1 19:58:40 2013
gcc-c++-4.4.7-3.el6                           Fri Mar 15 07:59:36 2013
gcc-gfortran-4.4.7-3.el6                      Fri Mar 15 07:59:34 2013
...

Which package owns a command:

$ rpm −qf `which make`
make-3.81-3.el5

(This was run on an older Redhat 5.6 system which has make.)

Similarly, which package owns a file:

$ rpm −qf /usr/lib64/libssh2.so.1
libssh2-1-1.2.9-4.2.2.1

List files in (an installed) package:
$ rpm −ql freeradius-client-1.1.6-40.1

List files in an rpm package file:
$ rpm −qlp packages/HPSiS1124Core-11.24.241-Linux2.4.rpm

Get history of the package versions on this server:

$ yum history list te-agent

Get history of the list of changes to this package:

$ rpm -q -changelog te-agent

Install a package from a local RPM file:
$ rpm −i openmotif-libs-32bit-2.3.1-3.13.x86_64.rpm

Uninstall a packge:
$ rpm −e package
$ rpm −e freeradius-server-libs-2.1.1-7.12.1

How will you install the missing make in CentOS? Use yum to search for it:

$ yum search make

Loaded plugins: fastestmirror
Determining fastest mirrors
 * base: mirror.umd.edu
 * extras: mirror.umd.edu
 * updates: mirror.cogentco.com
============================== N/S Matched: make ===============================
automake.noarch : A GNU tool for automatically creating Makefiles
...
imake.x86_64 : imake source code configuration and build system
...
make.x86_64 : A GNU tool which simplifies the build process for users
makebootfat.x86_64 : Utility for creation bootable FAT disk
mendexk.x86_64 : Replacement for makeindex with many enhancements
...

How to install it:

$ sudo yum install make.x86_64

Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.umd.edu
 * extras: mirror.umd.edu
 * updates: mirror.cogentco.com
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package make.x86_64 1:3.81-19.el6 will be installed
--> Finished Dependency Resolution
 
Dependencies Resolved
 
===========================================================================================================================
 Package                   Arch                        Version                             Repository                 Size
===========================================================================================================================
Installing:
 make                      x86_64                      1:3.81-19.el6                       base                      389 k
 
Transaction Summary
===========================================================================================================================
Install       1 Package(s)
 
Total download size: 389 k
Installed size: 1.0 M
Is this ok [y/N]: y
Downloading Packages:
make-3.81-19.el6.x86_64.rpm                                                                         | 389 kB     00:00
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : 1:make-3.81-19.el6.x86_64                                                                               1/1
 
Installed:
  make.x86_64 1:3.81-19.el6
 
Complete!

make should now be in your path.

If we were dealing with SLES I would use zypper instead of yum, but the idea of searching and installing is similar.

Debian Linux, e.g. Raspberry Pi

Find which package a file belongs to:

> dpkg -S filepath

List installed packages:

> dpkg -l

List all files belonging to the package iperf3:

> dpkg -L iperf3

Transferring packages from one system to another

When I needed to transfer Debian packages from one system with Internet access to another without, I would do:

apt download apache2

Then sftp the file to the other system and on it do

apt install ./apache2_2.4.53-1~deb11u1_amd64.deb

In fact that only worked after I installed all dependencies. This web of files covered all dependencies:

apache2-bin_2.4.53-1~deb11u1_amd64.deb
apache2-data_2.4.53-1~deb11u1_all.deb
apache2-utils_2.4.53-1~deb11u1_amd64.deb
apache2_2.4.53-1~deb11u1_amd64.deb
libapr1_1.7.0-6+deb11u1_amd64.deb
libaprutil1-dbd-mysql_1.6.1-5_amd64.deb
libaprutil1-dbd-odbc_1.6.1-5_amd64.deb
libaprutil1-dbd-pgsql_1.6.1-5_amd64.deb
libaprutil1-dbd-sqlite3_1.6.1-5_amd64.deb
libaprutil1-ldap_1.6.1-5_amd64.deb
libaprutil1_1.6.1-5_amd64.deb
libgdbm-compat4_1.19-2_amd64.deb
libjansson4_2.13.1-1.1_amd64.deb
liblua5.3-0_5.3.3-1.1+b1_amd64.deb
libmariadb3_1%3a10.5.15-0+deb11u1_amd64.deb
libperl5.32_5.32.1-4+deb11u2_amd64.deb
mailcap_3.69_all.deb
mariadb-common_1%3a10.5.15-0+deb11u1_all.deb
mime-support_3.66_all.deb
mysql-common_5.8+1.0.7_all.deb
perl-modules-5.32_5.32.1-4+deb11u2_all.deb
perl_5.32.1-4+deb11u2_amd64.deb
ssl-cert_1.1.0+nmu1_all.deb

Categories
Admin CentOS Security

How to Set up a Secure sftp-only Service

Intro
Updated Jan, 2015.

Usually I post a document because I think I have something to add. This time I found a link that covers the topic better than I could. I just wanted to have it covered here. What if you want to offer an sftp-only jailed account? Can you do that? How do you do it?

The Answer
Well, it used to be all here: http://blog.swiftbyte.com/linux/allowing-sftp-access-while-chrooting-the-user-and-denying-shell-access/. But that link is no longer valid.

I tried it, appropriately modified for CentOS and it worked perfectly. A few notes. Presumably you will already have ssh installed. Who can imagine a server without it? So there’s typically no need to install openssh-server.

I was leery mucking with subsystem sftp. What if it prevented me from doing sftp to my own account and having full access like I’m used to? Turns out it does no harm in that regard.

Very minor point. His documentation might be good for Ubuntu. To restart the ssh daemon in CentOS/Fedora, I recommend a sudo service sshd restart. Do you wonder if that will knock you out of your own ssh session? I did. It does not. Not sure why not!

These groupadd/useradd/usermod functions are “cute.” I’m old school and used to editing the darn files by hand (/etc/passwd, /etc/group). I suppose it’s safer to use the cute functions – less chance a typo could render your server inoperable (yup, done that).

Let’s call my sftp-only user is joerg.

I did the chown root:root thing, but initially the files weren’t accessible to the joerg user. The permissions were 700 on the home directory, now owned by root. That produces this error when you try to sftp:

$ sftp joerg@localhost
sftp> dir

Couldn't get handle: Permission denied

That’s no good, so I liberalized the permissions:

$ sudo chmod go+rx /home/joerg

My /etc/passwd line for this user looks like this:

joerg:x:1004:901:Joerg, etherip author:/home/joerg:/bin/false

So note the unusual shell, /bin/false. That’s the key to locking things down.

In /etc/group I have this;

joerg:x:1004:

If you want to add the entries by hand to passwd and group then if I recall correctly you run a pwconv to generate an appropriate entry for it in /etc/shadow, and a sudo passwd joerg to set up a desired password.

Does it work? Yeah, it really does.

$ sftp joerg@localhost
Connecting to localhost…
sftponlyuser@localhost’s password:
sftp> pwd
Remote working directory: /
sftp> cd ..
sftp> pwd
Remote working directory: /
sftp> cd /etc
Couldn’t canonicalise: No such file or directory
sftp> ls -l
[shows files in /home/joerg]

Moreover, ssh really is shut out:

$ ssh joerg@localhost
joerg@localhost’s password:

This hangs and never returns with a prompt!

Cool, huh?

Locking out this same account
Now suppose you only intended joerg to temporarily have access and you want to lock the account out without actually removing it. This can be done with:

$ sudo passwd -l joerg

This puts an invalid character in that account’s shadow file entry.

Conclusion
We have an easy prescription to make a jailed sftp-only account that we tested and found really works. Regular accounts were not affected. The base article on which I embellished is now kaput so I’ve added a few more details to make up for that.

Categories
Uncategorized

Experimenting with GoodSync

Intro
I was thinking about doing something with photos – not exactly sure what yet. But since I have this nice decicated server with lots of space, it’s a great place to store my collection of thousands of photos. But how to upload them and keep the server copy in sync?

The traditional Unix approach might be to install rsync, and I suppose I could have gotten it to work. But I decided to see what commercial package was out there. I settled on Goodsync. One of the inducements is that it has support for sftp servers. Or so it says.

Well, I got it to work. I decided to use a private key approach to login. I generated my key pair with cygwin’s openssl. It all seemed fine, however, I couldn’t use that key in GoodSync. Based on something I read in their documentation, I decided it might need a key generated by putty. So I downloaed puttygen and generated another keypair. I then had to make a saved session in putty, which I had never done before, using that keypair. I tested with putty’s psftp -load session. It worked.

So I loaded up that session in GoodSync. It began to work. I could successfully analyze.

I thought JPEG files were compressible. I played around with setting the compress option in putty, but it didn’t seem to matter one bit. Then I ran gzip on one of the files and saw essentially no reduction in size from compression.

So now the files are crawling from my home PC over to the server. It will take days to finish. I bought the professional version of GoodSync and run multiple jobs simultaneously, which is kind of nice and a slightly better use of the bandwidth.

Open issues include: would GoodSync’s native server offer me any advantages? Does it even run on Linux? What program do I use to display the images? Is there a scheduler for GoodSync?

Conclusion
GoodSync seems to be a solid program for syncing files from a PC to an sftp server, although that is not its primary focus, The GUI is nice and makes it easy to set up sync jobs.

Categories
Admin Web Site Technologies

Tipsheet: How to run NTP on F5 BigIP

If you feel you’ve added your ntp servers correctly via the GUI (System|Configuration|Device|NTP), and yet you get an output like this:

# ntpq -p

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 ntp1.drj        .INIT.          16 -    - 1024    0    0.000    0.000   0.000
 ntp2.drj        .INIT.          16 -    - 1024    0    0.000    0.000   0.000

and you observe the time is off by seconds or even minutes, then you may have made the mistake I made. I used fully-qualified domain names (FQDN) for the ntp servers.

Switch from FQDNs to IP addresses and it will work fine:

# ntpq -p

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
*ntp1.drj         10.23.34.1     3 u  783 1024  377    0.951   -8.830   0.775
+ntp2.drj         10.23.35.1     3 u  120 1024  377   20.963   -8.051   5.705

The date command will now give the correct time.

Having correct time is useful for the logging, especially if you are using ASM and are trying to correlate known activity against the reported errors.

Categories
Perl Python

Help with the NPR Weekend Puzzle – and Learning Python

Intro
As I mentioned in my review of Amazon’s Web Services Summit, Python seems to be the vogue scripting language these days. I decided I had better dust off the brain cells and try it out. I am an old Perl stalwart, but one senses that that language has sort of hit a wall after enthusiasm from 10 years ago began to wane. One of my example Perl scripts is provided in my post about turning HP SiteScope into SiteScope Classic. After deciding I needed a Python project only a few days passed before I came across what I thought would be a worthy challenge – simple yet non-trivial. That is the weekend puzzle as I understand it on NPR.

The Details
Start with a one-syllable, four-letter common boy’s name. Adjust all the letters with the ROT-13 cipher, and arrive at a common two-syllable girl’s name? What are the names? I’m pretty sure I could have figured out how to do this in Perl. Python? If it lived up to its hype then it should also be up to the task IMHO.

About the Weekend Puzzle
I listen to it most weekends. I often end up listening to it twice! I always think of whether it is suitable to be programmed or not. Most often I find it is not. I mean not suitable for the simple scripts and such that I write. I’m not talking about IBM’s Jeopardy-playing Watson! But this weekend I feel that the ROT-13 part of the challenge can definitely be aided by programming. Is that cheating? If I “give away” my solution as I am then I remove my unfair advantage in knowing a thing or two about programming!

The program – npr-test.py

#!/usr/bin/python
# drJ test script - 4/2012
# to get input agruments...
import sys
#inputName = raw_input("Enter name to be translated: ");
#print "Received input is : ", inputName
# and see http://www.tutorialspoint.com/python/string_maketrans.htm
from string import maketrans
intab = "abcdefghijklmnopqrstuvwxyz"
outtab = "nopqrstuvwxyzabcdefghijklm"
trantab = maketrans(intab, outtab)
inputName = sys.argv[1]
 
print inputName,inputName.translate(trantab);

So you see Python has this great maketrans built-in function that we’re able to use to implement the ROT-13 cipher. Of course veterans will probably know an even simpler way to accomplish this, perhaps with the pack/unpack functions which I also considered using.

You call the script like this:

$ npr-test.py john

john wbua

I compiled a list of common four-letter names which I won’t fully divulge. They are in a file called names, one name per line. But how to quickly put it though this program? My old, bad lazy Unix habit was to do this:

$ cat names|while read line; do
npr-test.py $line >> /tmp/results
done

I’ve got it memorized so I lose no time except typing the characters. But I also know the modern way is xargs.

xargs is the really hard part
I keep thinking that xargs is a good habit and one I should get into. But it’s not so easy. It took me awhile to find the appropriate example. And then you run across the debate that holds that Gnu parallel is still the better tool. Anyhow, here’s the xargs way…

$ cat names|xargs -I {} npr-test.py {}|more

amit nzvg
amos nzbf
arno neab
axel nkry
...

Conclusion
This little python program speaks volumes about the versatility of this language. It does have some really interesting properties and at first blush is worth getting to know better. No wonder others have embraced it. It also has helped us solve the weekend puzzle!

The stated answer? Glen and Tyra. You’ll see you can feed either one of these names into the program and come out with the other. I found it amusing that Will Shortz described the puzzle as “hard.” I didn’t think so – not with this program – but I was not the randomly selected winner, either, so I didn’t get a chance to explain how I did it.

The guy who did win explained that he simply wrote the ROT-13 version of the alphabet below the alphabet so he had a convenient look-up table. Clever.

Categories
DNS

“FBI’s” DNS Severs – Trying to Clear Some of the Hysteria

Intro
This Fox News article about infected computers was brought to my attention:
Hundreds of thousands may lose Internet in July

Hey, their URLs look like my URLs. Wonder if they’re using WordPress.

But I digress. I was asked if this is a hoax. I really had to puzzle over it for awhile. It smells like a hoax. I didn’t find any independent confirmation on the isc.org web site, where I get my BIND source files from. I didn’t see a blog post from Paul Vixie mentioning this incident.

And the comments. Oh my word the comments. Every conspiracy theorist is eating this up, gorging on it. The misinformation is appalling. There is actually no single comment as of this writing (100+ comments so far) that lends any technical insight into the saga.

Let’s do a reset and start from a reasonable person basis, not political ideology.

To recount, apparently hundreds of thousands of computers got hacked in a novel way. They had their DNS servers replaced with DNS servers owned by hackers. in Windows you can see your DNS servers with

> ipconfig /all

in a CMD window.

Look for the line that reads

DNS Servers…

These DNS servers could redirect users to web sites that encouraged users to fill out surveys which generated profit for the hackers.

It’s a lot easier to control a few servers than it is to fix hundreds of thousands of desktops. So the FBI got permission through the courts to get control of the IP addresses of the DNS servers involved and decided to run clean DNS servers. This would keep the infected users working, and they would no longer be prompted to fill out surveys. In fact they would probably feel that everything was great again. But this solution costs money to maintain. Is the FBI running these DNS servers? I highly, highly doubt it. I’ll bet they worked with ISC (isc.org) who are the real subject matter experts and quietly outsourced that delicate task to them.

So in July, apparently, (I can’t find independent confirmation of the date) they are going to pull the plug on their FBI DNS servers.

On that day some 80,000 users in the US alone will not be able to browse Internet sites as a result of this action, and hundreds of thousands outside of the US.

If the FBI works with some DNS experts – and Paul Vixie is the best out there and they are apparently already working with him – they could be helpful and redirect all DNS requests to a web site that explains the problem and suggests methods for fixing it. It’s not clear at this point whether or not they will do this.

That’s it. No conspiracy. The FBI was trying to do the right thing – ease the users off the troubled DNS services to somewhat minimize service disruption. I would do the same if I were working for the FBI.

Unfortunately, feeding fodder to the the conspiracists is the fact that the mentioned web site, dcwg.org, is not currently available. The provenance of that web site is also hard to scrutinize as it’s registered to an individual, which looks fishy. But upon digging deeper I have to say that probably the site is just overwhelmed right now. It was first registered in November – around the time this hack came to light. It stands for DNS Changer Working Group.

Comparitech provides more technical details in this very well-written article: DNS changer malware.

Another link on their site that discusses this topic: check-to-see-if-your-computer-is-using-rogue-DNS

Specific DNS Servers
I want to be a good Netizen and not scan all the address ranges mentioned in the FBI document. So I took an educated guess as to where the DNS servers might actually reside. I found three right off the bat:

64.28.176.1
213.109.64.1
77.67.83.1

Knowing actual IPs of actual DNS servers makes this whole thing a lot more real from a technical point-of-view. Because they are indeed unusual in their behaviour. They are fully resolving DNS servers, which is a rarity on the Internet but would be called for by the problem at hand. Traceroute to them all shows the same path, which appears to be somewhere in NYC. A DNS query takes about 17 msec from Amazon’s northeast data center which is in Virginia. So the similarity of the path seems to suggest that they got hold of the routing for these subnets and are directing them to the same set of clean DNS servers.

Let’s see what else we can figure out.

Look up 64.28.176.1 on arin.net Whois. More confirmation that this is real and subject to a court order. In the comments section it says:

In accordance with a Court Order in connection with a criminal investigation, this registration record has been locked and during the period from November 8, 2011 through March 22, 2012, any changes to this registration record are subject to that Court Order.

I thought we could gain even more insight looking up 213.109.64.1 which is normally address range outside of North America, but I can’t figure out too much about it. You can look it up in the RIPE database (ripe.net). You will in fact see it is registered to Prolite Ltd in Russia. No mention of a court order. So we can speculate that unlike arin.net, RIPE did not bother to update their registration record after the FBI got control. So Prolite Ltd may either have been an active player in the hack, or merely had some of their servers hacked and used for the DNSchanger DNS server service.

Of course we don’t know what the original route looked like, but I bet it didn’t end in new York City, although that can’t be ruled out. But now it does. I wonder how the FBI got control of that subnet and if it involved international cooperation.

7/9 update
The news reports re-piqued my interest in this story. I doubled back and re-checked those three public DNS servers I had identified above. 64.28.176.1, etc. Sure enough, they no longer respond to DNS queries! The query comes back like this:

$ dig ns com @77.67.83.1

; <<>> DiG 9.7.3-P3-RedHat-9.7.3-8.P3.el6_2.2 <<>> ns com @77.67.83.1
;; global options: +cmd
;; connection timed out; no servers could be reached

Conclusion
As a DNS and Internet expert I have some greater insight into this particular news item than the average commentator. Interested in DNS? Here’s another article I wrote about DNS: Google’s DNS Servers Rock!

References and related
Here’s a much more thorough write-up of the DNS Changer situation: http://comparitech.net/dnschangerprotection

Categories
Admin

Highlights from the AWS Summit in NYC April 19 2012

Intro
It’s interesting to attend popular IT events. I haven’t been to one for awhile. As informative as the presentations are what you can learn by observing the other participants. Here are my observations from the Amazon AWS Summit yesterday in NYC.

Amazon has some really knoweldgeable managers addressing the audience. These guys had clearly mastered the details.

Learned by Observing Others
I’m a follower not a leader when it comes to consumer gadgets. At this event basically everyone had a Smartphone. Blackberries were well-represented though not predominant. iPhones and other Android phones seemed be in great use (unfortunately I cannot tell one from the other). iPads were pretty common, too. As were small notebooks and the occasional standard laptop. Probably 15% of the audience had one of those devices. The conference gave out WiFi access, which required a username/password to be entered in a web page. That was pretty important as I could not get signal otherwise.

I learned that people now take pictures of slides they want to remember for later. Guess I have to get better at using my Smartphone camera! It’s a great idea, really.

It used to be that these events would be very male-dominated. That’s still the case, but not quite as much. Our lunch tables help 10 people each and there was on average one woman per table, so about 10%.

Now about AWS Itself
Lots and lots of companies are doing cool things with Amazon cloud services. Pinterest, PBS, Exfm, Vox even old-line companies like Shell. I don’t think any serious start-up would buy their own infrastructure today when Amazon has made it so easy and economical to put things in the cloud.

Amazon started the service in 2006 and thus are celebrating six years of cloud service. They have lowered prices 19 times. For me the most significant of those price drops was in early March when it became competitive, no, more than competitive to be considered against the pure hosting companies.

Amazon cloud services not counts about 60 service offerings. I don’t claim to understand even most of them, but the ones I do seem really well thought-out. They really deserve to be that Magic Quadrant leader that Gartner paints them out to be, leading in both vision and ability to execute.

S3 is for storing objects. You don’t care how the object is stored. Your reference to the object will be a URL. One example actual usage: storing images. Each image stored as an object in S3. This makes it easy to begin to use a content distribution network (CDN).

I use EC2 so I have some familiarity with it. Users are supposed to have some familiarity with the author of their image. Oops. I do not. News to me: they have a facility to import VM images.

In virtual private cloud (VPC) the user gets to extend his own private address space into Amazon Cloud. Cool. There is a Direct Connect option to avoid traditional VPN tunnels. Security groups offer a decent way to enforce security but I don’t totally get it.

I’m interested in EtherIP. From this summit I learned of a company that has a product for that. The company is Astaro and the appliance is RED (Remote Ethernet Device). Cool.

One molecular discovery company chained together 30000 Amazon cpu cores, ran it for three hours, for a total of 100,000 hours of compute time, and only paid $5,000/hour. All that horsepower together makes for the 50th fastest supercomputer in the world. The equipment value was about $22,000,000.

I guess attendance was at least 2,000 people. There were lots of younger people there as well as an ecosystem of supporting third-party companies.

The Python language was mentioned three or four times. Its star is on the rise.

I came as a private user of AWS. This is my story. I was disappointed that in all their use cases and examples, there was not one mention of individual users such as myself. It was all geared towards organizations. You get worried when you don’t see anyone like yourself out there. I even asked one of the speakers about that but he said that they take users like me as a given. The hard work was getting the enterprises comfortable with it. I’m not entirely convinced.

Conclusion
Amazon has an exciting cloud offering and they get it. Their customers include probably the most interesting companies on the planet.

Categories
Admin DNS Proxy

The IT Detective Agency: Browsing Stopped Working on Internet-Connected Enterprise Laptops

Intro
Recall our elegant solution to DNS clobbering and use of a PAC file, which we documented here: The IT Detective Agency: How We Neutralized Nasty DNS Clobbering Before it Could Bite Us. Things were going smoothly with that kludge in place for months. Then suddenly it wasn’t. What went wrong and how to fix?? Read on.

The Details
We began hearing reports of increasing numbers of users not being able to access Internet sites on their enterprise laptops when directly connected to Internet. Internet Explorer gave that cryptic error to the effect Internet Explorer cannot display the web page. This was a real problem, and not a mere inconvenience, for those cases where a user was using a hotel Internet system that required a web page sign-on – that web page itself could not be displayed, giving that same error message. For simple use at home this error may not have been fatal.

But we had this working. What the…?

I struggled as Rop demoed the problem for me with a user’s laptop. I couldn’t deny it because I saw it with my own eyes and saw that the configuration was correct. Correct as in how we wanted it to be, not correct as in working. Basically all web pages were timing out after 15 seconds or so and displaying cannot display the web page. The chief setting is the PAC file, which is used by this enterprise on their Intranet.

On the Internet (see above-mentioned link) the PAC file was more of an annoyance and we had aliased the DNS value to 127.0.0.1 to get around DNS clobbering by ISPs.

While I was talking about the problem to a colleague I thought to look for a web server on the affected laptop. Yup, there it was:

C:\Users\drj>netstat -an|more

Active Connections

  Proto  Local Address          Foreign Address        State
  TCP    0.0.0.0:80             0.0.0.0:0              LISTENING
  TCP    0.0.0.0:135            0.0.0.0:0              LISTENING
  ...

What the…? Why is there a web server running on port 80? How will it respond to a PAC file request? I quickly got some hints by hitting my own laptop with curl:

$ curl -i 192.168.3.4/proxy.pac

HTTP/1.1 404 Not Found
Content-Type: text/html; charset=us-ascii
Server: Microsoft-HTTPAPI/2.0
Date: Tue, 17 Apr 2012 18:39:30 GMT
Connection: close
Content-Length: 315


Not Found

Not Found


HTTP Error 404. The requested resource is not found.

So the server is Microsoft-HTTPAPI, which upon invetsigation seems to be a Microsoft Web Deployment Agent Service (MsDepSvc).

The main point is that I don’t remember that being there in the past. I felt it’s presence, probably a new “feature” explained the current problem. What to do about it however??

Since this is an enterprise, not a small shop with a couple PCs, turning off MsDepSvc is not a realistic option. It’s probably used for peer-to-peer software distribution.

Hmm. Let’s review why we think our original solution worked in the first place. It didn’t work so well when the DNS was clobbered by my ISP. Why? I think because the ISP put up a web server when it encountered a NXDOMAIN DNS response and that web server gave a 404 not found error when the browser searched for the PAC file. Turning the DNS entry to the loopback interface, 127.0.0.1, gave the browser a valid IP, one that it would connect to and quickly receive a TCP RST (reset). Then it would happily conclude there was no way to reach the PAC file, not use it, and try DIRECT connections to Internet sites. That’s my theory and I’m sticking to it!

In light of all this information the following option emerged as most likely to succeed: set the Internet value of the DNS of the PAC file to a valid IP, and specifically, one that would send a TCP RST (essentially meaning that TCP port 80 is reachable but there is no listener on it).

We tried it and it seemed to work for me and my colleague. We no loner had the problem of not being able to connect to Internet sites with the PAC file configured and directly connected to Internet.

I noticed however that once I got on the enterprise network via VPN I wasn’t able to connect to Internet sites right away. After about five minutes I could.

My theory about that is that I had a too-long TTL on my PAC file DNS entry. The TTL was 30 minutes. I shortened it to five minutes. Because when you think about it, that DNS value should get cached by the PC and retained even after it transitions to Intranet-connected.

I haven’t retested, but I think that adjustment will help.

Conclusion
I also haven’t gotten a lot of feedback from this latest fix, but I’m feeling pretty good about it.

Case: again mostly solved.

Hey, don’t blame me about the ambiguity. I never hear back from the users when things are working 🙂

References
A closely related case involving Verizon “clobbering” TCP RST packets also bit us.

Categories
Admin Security

User Add/Delete Jython Script for RSA Authentication Manager

Intro
I thought I might save someone else the trouble of re-creating a respectable program using the Authentication Manager SDK for version 7.1 which can simply add and delete users to keep the local database in sync with an external database.

History Lesson
We had this problem licked under Authentication Manager v 6.1.2. In that version the sdk was TCL-based and for whatever reason, it seemed a whole lot simpler to understand the model and get working code. When we began to look at v 7.1 we saw we were confronted with a whole different animal that required new understanding and new skills to master.

The Details
Jython is Python plus Java. I really don’t know either language so I used a technique you might call programming by extrapolation. Here is the code. Not really understanding python I preserved as much as possible for fear of breaking something. I nevertheless had to be a little innovative and create a new function.

'''
 * Jython class demonstrating the Administration API
 * usage from a Jython script.
 *
 * Run this script in the utils directory of the Authentication Manager installation.
 *
 * Execute the command "rsautil jython AdminAPIDemos.py create <admin user name> <password>"
 * Execute the command "rsautil jython AdminAPIDemos.py assign <admin user name> <password>"
 * Execute the command "rsautil jython AdminAPIDemos.py update <admin user name> <password>"
 * Execute the command "rsautil jython AdminAPIDemos.py delete <admin user name> <password>"
 *
 * If you are executing this script in an environment other than the predefined
 * rsautil scripting tool you must make the CommandClientAppContext.xml file
 * available in the end of the classpath for this script. You must also configure
 * the necessary connection parameters in a properties file located in the process
 * working directory. See the provided samples for more information.
'''
 
# imports
from jarray import array
import sys
# DrJ required import
# Not Workign! from org.python.modules import re
from java.util.regex import *
from java.lang import *
 
 
from java.util import Calendar,Date
from java.lang import String
 
from org.springframework.beans import BeanUtils
 
from com.rsa.admin import AddGroupCommand
from com.rsa.admin import AddPrincipalsCommand
from com.rsa.admin import DeleteGroupCommand
from com.rsa.admin import DeletePrincipalsCommand
from com.rsa.admin import LinkGroupPrincipalsCommand
from com.rsa.admin import LinkAdminRolesPrincipalsCommand
from com.rsa.admin import SearchAdminRolesCommand
from com.rsa.admin import SearchGroupsCommand
from com.rsa.admin import SearchPrincipalsCommand
from com.rsa.admin import SearchRealmsCommand
from com.rsa.admin import SearchSecurityDomainCommand
from com.rsa.admin import UpdateGroupCommand
from com.rsa.admin import UpdatePrincipalCommand
from com.rsa.admin.data import AdminRoleDTOBase
from com.rsa.admin.data import GroupDTO
from com.rsa.admin.data import IdentitySourceDTO
from com.rsa.admin.data import ModificationDTO
from com.rsa.admin.data import PrincipalDTO
from com.rsa.admin.data import RealmDTO
from com.rsa.admin.data import SecurityDomainDTO
from com.rsa.admin.data import UpdateGroupDTO
from com.rsa.admin.data import UpdatePrincipalDTO
from com.rsa.authmgr.admin.agentmgt import AddAgentCommand
from com.rsa.authmgr.admin.agentmgt import DeleteAgentsCommand
from com.rsa.authmgr.admin.agentmgt import LinkAgentsToGroupsCommand
from com.rsa.authmgr.admin.agentmgt import SearchAgentsCommand
from com.rsa.authmgr.admin.agentmgt import UpdateAgentCommand
from com.rsa.authmgr.admin.agentmgt.data import AgentConstants
from com.rsa.authmgr.admin.agentmgt.data import AgentDTO, ListAgentDTO
from com.rsa.authmgr.admin.hostmgt.data import HostDTO
from com.rsa.authmgr.admin.principalmgt import AddAMPrincipalCommand
from com.rsa.authmgr.admin.principalmgt.data import AMPrincipalDTO
from com.rsa.authmgr.admin.tokenmgt import GetNextAvailableTokenCommand
from com.rsa.authmgr.admin.tokenmgt import LinkTokensWithPrincipalCommand
from com.rsa.authn import SearchPasswordPoliciesCommand
from com.rsa.authn import UpdatePasswordPolicyCommand
from com.rsa.authn.data import PasswordPolicyDTO
from com.rsa.command import ClientSession
from com.rsa.command import CommandException
from com.rsa.command import CommandTargetPolicy, ConnectionFactory
from com.rsa.command.exception import DataNotFoundException, DuplicateDataException
from com.rsa.common.search import Filter
 
'''
 * This class demonstrates the usage patterns of the
 * Authentication Manager 7.1 API.
 *
 * <p>
 * The first set of operations performed if the first
 * command line argument is equal to "create".
 * The sample creates a restricted agent, a group, and a user.
 * Links the user to the group and the group to the agent.
 * </p>
 * <p>
 * The second set of operations performed if the first
 * command line argument is equal to "delete".
 * Lookup the user, group and agent created above.
 * Delete the user, group and agent.
 * </p>
 * <p>
 * A third set of operations is performed if the first
 * command line argument is equal to "assign".
 * Lookup the user and assign the next available
 * SecurID token to the user.
 * Lookup the SuperAdminRole and assign it to the user.
 * </p>
 * <p>
 * A fourth set of operations performed if the first
 * command line argument is equal to "update".
 * Update the Agent, Group, and User objects.
 * </p>
 * <p>
 * A fifth set of operations performed if the first
 * command line argument is equal to "disable".
 * Lookup a password policy with a name that starts
 * with "Initial" and then disable the password history
 * for that policy. Use this to allow the sample to
 * perform multiple updates of the user password using
 * the same password for each update.
 * </p>
 * <p>
 * The APIs demonstrated include the use of the Filter
 * class to generate search expressions for use with
 * all search commands.
 * </p>
'''
class AdminAPIDemos:
 
    '''
     * We need to know these fairly static values throughout this sample.
     * Set the references to top level security domain (realm) and system
     * identity source to use later.
     *
     * @throws CommandException if something goes wrong
    '''
    def __init__(self):
        searchRealmCmd = SearchRealmsCommand()
        searchRealmCmd.setFilter( Filter.equal( RealmDTO.NAME_ATTRIBUTE, "SystemDomain"))
        searchRealmCmd.execute()
        realms = searchRealmCmd.getRealms()
        if( len(realms) == 0 ):
            print "ERROR: Could not find realm SystemDomain"
            sys.exit( 2 )
 
        self.domain = realms[0].getTopLevelSecurityDomain()
        self.idSource = realms[0].getIdentitySources()[0]
 
 
    '''
     * Create an agent and set it to be restricted.
     *
     * @param: name the name of the agent to create
     * @param: addr the IP address for the agent
     * @param: alt array of alternate IP addresses
     * @return: the GUID of the agent just created
     * 
     * @throws CommandException if something goes wrong
    '''
    def createAgent(self, name, addr, alt):
        # need a HostDTO to be set
        host = HostDTO()
        host.setName(name)
        host.setPrimaryIpAddress(addr)
        host.setSecurityDomainGuid(self.domain.getGuid())
        host.setNotes("Created by AM Demo code")
 
        # the agent to be created
        agent = AgentDTO()
        agent.setName(name)
        agent.setHost(host)
        agent.setPrimaryAddress(addr)
        agent.setAlternateAddresses(alt)
        agent.setSecurityDomainId(self.domain.getGuid())
        agent.setAgentType(AgentConstants.STANDARD_AGENT)
        agent.setRestriction(1) # only allow activated groups
        agent.setEnabled(1)
        agent.setOfflineAuthDataRefreshRequired(0)
        agent.setNotes("Created by AM Demo code")
 
        cmd = AddAgentCommand(agent)
 
	try:        
	    cmd.execute()
        except DuplicateDataException:
            print "ERROR: Agent " + name + " already exists."
	    sys.exit(2)
 
        # return the created agents GUID for further linking
        return cmd.getAgentGuid()
 
 
    '''
     * Lookup an agent by name.
     *
     * @param: name the agent name to lookup
     * @return: the GUID of the agent
     * 
     * @throws CommandException if something goes wrong
    '''
    def lookupAgent(self, name):
        cmd = SearchAgentsCommand()
        cmd.setFilter(Filter.equal(AgentConstants.FILTER_HOSTNAME, name))
        cmd.setLimit(1)
        cmd.setSearchBase(self.domain.getGuid())
        # the scope flags are part of the SecurityDomainDTO
        cmd.setSearchScope(SecurityDomainDTO.SEARCH_SCOPE_ONE_LEVEL)
 
        cmd.execute()
 
	if (len(cmd.getAgents()) < 1):
            print "ERROR: Unable to find agent " + name + "."  
	    sys.exit(2)
 
        return cmd.getAgents()[0]
 
 
    '''
     * Update an agent, assumes a previous lookup done by lookupAgent.
     *
     * @param agent the result of a previous lookup
     *
     * @throws CommandException if something goes wrong
    '''
    def updateAgent(self, agent):
        cmd = UpdateAgentCommand()
 
        agentUpdate = AgentDTO()
        # copy the rowVersion to satisfy optimistic locking requirements
        BeanUtils.copyProperties(agent, agentUpdate)
 
        # ListAgentDTO does not include the SecurityDomainId
        # use the GUID of the security domain where agent was created
        agentUpdate.setSecurityDomainId(self.domain.getGuid())
 
        # clear the node secret flag and modify some others
        agentUpdate.setSentNodeSecret(0)
        agentUpdate.setOfflineAuthDataRefreshRequired(1)
        agentUpdate.setIpProtected(1)
        agentUpdate.setEnabled(1)
        agentUpdate.setNotes("Modified by AM Demo code")
 
        # set the requested updates in the command
        cmd.setAgentDTO(agentUpdate)
 
        # perform the update
        cmd.execute()
 
 
    '''
     * Delete an agent.
     *
     * @param: agentGuid the GUID of the agent to delete
     * 
     * @throws CommandException if something goes wrong
    '''
    def deleteAgent(self, agentGuid):
        cmd = DeleteAgentsCommand( [agentGuid] )
        cmd.execute()
 
 
    '''
     * Create an IMS user, needs to exist before an AM user can be
     * created.
     *
     * @param: userId the user's login UID
     * @param: password the user's password
     * @param: first the user's first name
     * @param: last the user's last name
     * 
     * @return: the GUID of the user just created
     * 
     * @throws CommandException if something goes wrong
    '''
    def createUser(self, userId, password, first, last):
        cal = Calendar.getInstance()
 
        # the start date
        now = cal.getTime()
# DrJ: add 50 years from now!    
        cal.add(Calendar.YEAR, 50)
 
        # the account end date
        expire = cal.getTime()
 
        principal = PrincipalDTO()
        principal.setUserID( userId )
        principal.setFirstName( first )
        principal.setLastName( last )
        #     principal.setPassword( password )
 
        principal.setEnabled(1)
        principal.setLockoutStatus(0)
        principal.setAccountStartDate(now)
        #principal.setAccountExpireDate(expire)
        #principal.setAccountExpireDate(0)
        principal.setAdminRole(0)
        principal.setCanBeImpersonated(0)
        principal.setTrustToImpersonate(0)
 
        principal.setSecurityDomainGuid( self.domain.getGuid() )
        principal.setIdentitySourceGuid( self.idSource.getGuid() )
        principal.setDescription("Created by DrJ utilities")
 
        cmd = AddPrincipalsCommand()
        cmd.setPrincipals( [principal] )
 
        try:
            cmd.execute()
	except DuplicateDataException:
            print "ERROR: User " + userId + " already exists."
	    sys.exit(2)
 
        # only one user was created, there should be one GUID result
        return cmd.getGuids()[0]
 
 
    '''
     * Lookup a user by login UID.
     * 
     * @param: userId the user login UID
     *
     * @return: the GUID of the user record.
    '''
    def lookupUser(self, userId):
        cmd = SearchPrincipalsCommand()
        cmd.setFilter(Filter.equal(PrincipalDTO.LOGINUID, userId))
        cmd.setSystemFilter(Filter.empty())
        cmd.setLimit(1)
        cmd.setIdentitySourceGuid(self.idSource.getGuid())
        cmd.setSecurityDomainGuid(self.domain.getGuid())
        cmd.setGroupGuid(None)
        cmd.setOnlyRegistered(1)
        cmd.setSearchSubDomains(0)
 
        cmd.execute()
 
	if (len(cmd.getPrincipals()) < 1):
            print "ERROR: Unable to find user " + userId + "."
	    sys.exit(2)
 
        return cmd.getPrincipals()[0]
 
 
    '''
     * Update the user definition.
     *
     * @param user the principal object from a previous lookup
    '''
    def updateUser(self, user):
        cmd = UpdatePrincipalCommand()
        cmd.setIdentitySourceGuid(user.getIdentitySourceGuid())
 
        updateDTO = UpdatePrincipalDTO()
        updateDTO.setGuid(user.getGuid())
        # copy the rowVersion to satisfy optimistic locking requirements
        updateDTO.setRowVersion(user.getRowVersion())
 
        # collect all modifications here
        mods = []
 
        # first change the email
        mod = ModificationDTO()
        mod.setOperation(ModificationDTO.REPLACE_ATTRIBUTE)
        mod.setName(PrincipalDTO.EMAIL)
        mod.setValues([ user.getUserID() + "@mycompany.com" ])
        mods.append(mod) # add it to the list
 
        # also change the password
        mod = ModificationDTO()
        mod.setOperation(ModificationDTO.REPLACE_ATTRIBUTE)
        mod.setName(PrincipalDTO.PASSWORD)
        mod.setValues([ "MyNewPAssW0rD1!" ])
        mods.append(mod) # add it to the list
 
        # change the middle name
        mod = ModificationDTO()
        mod.setOperation(ModificationDTO.REPLACE_ATTRIBUTE)
        mod.setName(PrincipalDTO.MIDDLE_NAME)
        mod.setValues([ "The Big Cahuna" ])
        mods.append(mod) # add it to the list
 
        # make a note of this update in the description
        mod = ModificationDTO()
        mod.setOperation(ModificationDTO.REPLACE_ATTRIBUTE)
        mod.setName(PrincipalDTO.DESCRIPTION)
        mod.setValues([ "Modified by AM Demo code" ])
        mods.append(mod) # add it to the list
 
        # set the requested updates into the UpdatePrincipalDTO
        updateDTO.setModifications(mods)
        cmd.setPrincipalModification(updateDTO)
 
        # perform the update
        cmd.execute()
 
 
    '''
     * Delete a user.
     *
     * @param: userGuid the GUID of the user to delete
     * 
     * @throws CommandException if something goes wrong
    '''
    def deleteUser(self, userGuid):
        cmd = DeletePrincipalsCommand()
        cmd.setGuids( array( [userGuid], String ) )
        cmd.setIdentitySourceGuid( self.idSource.getGuid() )
        cmd.execute()
 
 
    '''
     * Create an Authentication Manager user linked to the IMS user.
     * The user will have a limit of 3 bad passcodes, default shell
     * will be "/bin/sh", the static password will be "12345678" and
     * the Windows Password for offline authentication will be "Password123!".
     *
     * @param: guid the GUID of the IMS user
     * 
     * @throws CommandException if something goes wrong
    '''
    def createAMUser(self, guid):
        principal = AMPrincipalDTO()
        principal.setGuid(guid)
        principal.setBadPasscodes(3)
        principal.setDefaultShell("/bin/sh")
        principal.setDefaultUserIdShellAllowed(1)
        # these next three innocent-looking lines cost you a license! do not use them!! - DrJ 
        #principal.setStaticPassword("12345678")
        #principal.setStaticPasswordSet(1)
        #principal.setWindowsPassword("Password123!")
 
        cmd = AddAMPrincipalCommand(principal)
        cmd.execute()
 
 
    '''
     * Create a group to assign a user to.
     *
     * @param: name the name of the group to create
     * @return: the GUID of the group just created
     * 
     * @throws CommandException if something goes wrong
    '''
    def createGroup(self, name):
        group = GroupDTO()
        group.setName(name)
        group.setDescription("Created by AM Demo code")
        group.setSecurityDomainGuid(self.domain.getGuid())
        group.setIdentitySourceGuid(self.idSource.getGuid())
 
        cmd = AddGroupCommand()
        cmd.setGroup(group)
 
	try:
            cmd.execute()
	except DuplicateDataException:
            print "ERROR: Group " + name + " already exists."
	    sys.exit(2)
 
        return cmd.getGuid()
 
    '''
     * Lookup a group by name.
     *
     * @param: name the name of the group to lookup
     * @return: the GUID of the group
     * 
     * @throws CommandException if something goes wrong
    '''
    def lookupGroup(self, name):
        cmd = SearchGroupsCommand()
        cmd.setFilter(Filter.equal(GroupDTO.NAME, name))
        cmd.setSystemFilter(Filter.empty())
        cmd.setLimit(1)
        cmd.setIdentitySourceGuid(self.idSource.getGuid())
        cmd.setSecurityDomainGuid(self.domain.getGuid())
        cmd.setSearchSubDomains(0)
        cmd.setGroupGuid(None)
 
        cmd.execute()
 
	if (len(cmd.getGroups()) < 1):
            print "ERROR: Unable to find group " + name + "."
	    sys.exit(2)
 
        return cmd.getGroups()[0]
 
 
    '''
     * Update a group definition.
     *
     * @param group the current group object
    '''
    def updateGroup(self, group):
        cmd = UpdateGroupCommand()
        cmd.setIdentitySourceGuid(group.getIdentitySourceGuid())
 
        groupMod = UpdateGroupDTO()
        groupMod.setGuid(group.getGuid())
        # copy the rowVersion to satisfy optimistic locking requirements
        groupMod.setRowVersion(group.getRowVersion())
 
        # collect all modifications here
        mods = []
 
        mod = ModificationDTO()
        mod.setOperation(ModificationDTO.REPLACE_ATTRIBUTE)
        mod.setName(GroupDTO.DESCRIPTION)
        mod.setValues([ "Modified by AM Demo code" ])
        mods.append(mod)
 
        # set the requested updates into the UpdateGroupDTO
        groupMod.setModifications(mods)
        cmd.setGroupModification(groupMod)
 
        # perform the update
        cmd.execute()
 
 
    '''
     * Delete a group.
     *
     * @param: groupGuid the GUID of the group to delete
     * 
     * @throws CommandException if something goes wrong
    '''
    def deleteGroup(self, groupGuid):
        cmd = DeleteGroupCommand()
        cmd.setGuids( [groupGuid] )
        cmd.setIdentitySourceGuid( self.idSource.getGuid() )
        cmd.execute()
 
 
    '''
     * Assign the user to the specified group.
     *
     * @param: userGuid the GUID for the user to assign
     * @param: groupGuid the GUID for the group
     * 
     * @throws CommandException if something goes wrong
    '''
    def linkUserToGroup(self, userGuid, groupGuid):
        cmd = LinkGroupPrincipalsCommand()
        cmd.setGroupGuids( [groupGuid] )
        cmd.setPrincipalGuids( [userGuid] )
        cmd.setIdentitySourceGuid(self.idSource.getGuid())
 
        cmd.execute()
 
    '''
     * Assign the group to the restricted agent so users can authenticate.
     *
     * @param: agentGuid the GUID for the restricted agent
     * @param: groupGuid the GUID for the group to assign
     * 
     * @throws CommandException if something goes wrong
    '''
    def assignGroupToAgent(self, agentGuid, groupGuid):
        cmd = LinkAgentsToGroupsCommand()
        cmd.setGroupGuids( [groupGuid] )
        cmd.setAgentGuids( [agentGuid] )
        cmd.setIdentitySourceGuid(self.idSource.getGuid())
 
        cmd.execute()
 
    '''
     * Assign next available token to this user.
     *
     * @param: userGuid the GUID of the user to assign the token to
     * 
     * @throws CommandException if something goes wrong
    '''
    def assignNextAvailableTokenToUser(self, userGuid):
        cmd = GetNextAvailableTokenCommand()
        try:
            cmd.execute()
        except DataNotFoundException:
            print "ERROR: No tokens available"
        else:
            tokens = [cmd.getToken().getId()]
            cmd2 = LinkTokensWithPrincipalCommand(tokens, userGuid)
            cmd2.execute()
            print ("Assigned next available SecurID token to user jdoe")
 
    '''
     * Lookup an admin role and return the GUID.
     *
     * @param name the name of the role to lookup
     * @return the GUID for the required role
     *
     * @throws CommandException if something goes wrong
     '''
    def lookupAdminRole(self, name):
        cmd = SearchAdminRolesCommand()
 
        # set search filter to match the name
        cmd.setFilter(Filter.equal(AdminRoleDTOBase.NAME_ATTRIBUTE, name))
        # we only expect one anyway
        cmd.setLimit(1)
        # set the domain GUID
        cmd.setSecurityDomainGuid(self.domain.getGuid())
 
        cmd.execute()
	if (len(cmd.getAdminRoles()) < 1):
            print "ERROR: Unable to find admin role " + name + "."
	    sys.exit(2)
 
        return cmd.getAdminRoles()[0].getGuid()
 
    '''
     * Assign the given admin role to the principal provided.
     *
     * @param adminGuid the GUID for the administrator
     * @param roleGuid the GUID for the role to assign
     *
     * @throws CommandException if something goes wrong
     '''
    def assignAdminRole(self, adminGuid, roleGuid):
        cmd = LinkAdminRolesPrincipalsCommand()
        cmd.setIgnoreDuplicateLink(1)
        cmd.setPrincipalGuids( [ adminGuid ] )
        cmd.setAdminRoleGuids( [ roleGuid ] )
        cmd.execute()
        print ("Assigned SuperAdminRole to user jdoe")
 
    '''
     * Lookup a password policy by name and return the object.
     *
     * @param name the policy name
     * @return the object
     *
     * @throws CommandException if something goes wrong
     '''
    def lookupPasswordPolicy(self, name):
        cmd = SearchPasswordPoliciesCommand()
        cmd.setRealmGuid(self.domain.getGuid())
 
        # match the policy name
        cmd.setFilter(Filter.startsWith(PasswordPolicyDTO.NAME, name))
 
        cmd.execute()
 
	if (len(cmd.getPolicies()) < 1):
            print ("ERROR: Unable to find password policy with name starting with " + name + ".")
	    sys.exit(2)
 
        # we only expect one anyway
        return cmd.getPolicies()[0]
 
    '''
     * Update the given password policy, currently it just disables
     * password history.
     *
     * @param policy the policy to update
     *
     * @throws CommandException if something goes wrong
     '''
    def updatePasswordPolicy(self, policy):
        cmd = UpdatePasswordPolicyCommand()
 
        # disable password history
        policy.setHistorySize(0)
        cmd.setPasswordPolicy(policy)
 
        cmd.execute()
 
    '''
     * Create a collection of related entities, user, agent, group, token.
     *
     * @param admin the administrator user name
     * @param password the administrator password
     * 
     * @throws Exception if something goes wrong
    '''
    def doCreate(self):
 
        # Create a hypothetical agent with four alternate addresses
        addr = "1.2.3.4"
        alt = [ "2.2.2.2",  "3.3.3.3", "4.4.4.4", "5.5.5.5" ]
 
        # create a restricted agent
        agentGuid = self.createAgent("Demo Agent", addr, alt)
        print ("Created Demo Agent")
 
        # create a user group
        groupGuid = self.createGroup("Demo Agent Group")
        print ("Created Demo Agent Group")
 
        # assign the group to the restricted agent
        self.assignGroupToAgent(agentGuid, groupGuid)
        print ("Assigned Demo Agent Group to Demo Agent")
 
        # create a user and the AMPrincipal user record
        userGuid = self.createUser("jdoe", "Password123!", "John", "Doe")
        self.createAMUser(userGuid)
        print ("Created user jdoe")
 
        # link the user to the group
        self.linkUserToGroup(userGuid, groupGuid)
        print ("Added user jdoe to Demo Agent Group")
 
    '''
     * add user by DrJ
     *
     * @param admin the administrator user name
     * @param password the administrator password
     * 
     * @throws Exception if something goes wrong
    '''
    def doAdd(self):
        # create a user and the AMPrincipal user record
        # loop over all users listed in addusers.txt
        f = open('addusers.txt','r')
        str = f.readline()
        while str:
            strs = str.rstrip()
            cols = strs.split(",")
            userid = cols[0]
            fname = cols[1]
            lname = cols[2]
            print userid
# if user already exists we want to go continue with the list
	    try:
                userGuid = self.createUser(userid, "*LK*", fname, lname)
                self.createAMUser(userGuid)
                print "Created user userid,fname,lname: ", userid,",",lname,",",fname,"\n"
            except:
                print "exception for user ",userid,"\n"
            str = f.readline()
 
        f.close()
 
 
    '''
     * Assign the next available token to the user.
     *
     * @param admin the administrator user name
     * @param password the administrator password
     * 
     * @throws Exception if something goes wrong
    '''
    def doAssignNextToken(self):
 
        # lookup and then ...
        userGuid = self.lookupUser("jdoe").getGuid()
 
        # assign the next available token to this user
        self.assignNextAvailableTokenToUser(userGuid)
 
        # now that he has a token make him an admin
        roleGuid = self.lookupAdminRole("SuperAdminRole")
        self.assignAdminRole(userGuid, roleGuid)
 
    '''
     * Delete the entities created by the doCreate method.
     *
     * @param admin the administrator user name
     * @param password the administrator password
     * 
     * @throws Exception if something goes wrong
    '''
    def doDelete(self):
 
        # lookup and then ...
        # loop over all users listed in delusers.txt
        f = open('delusers.txt','r')
        str = f.readline()
        while str:
            # format: userid,fname,lname  . We just want the userid
            cols = str.split(",")
            userid = cols[0]
            print userid
# if user doesn't exist we want to go continue with the list
	    try:
                userGuid = self.lookupUser(userid).getGuid()
                # ... cleanup
                self.deleteUser(userGuid)
                print "Deleted user ",userid
            except:
                print "exception for user ",userid,"\n"
            str = f.readline()
 
        f.close()
 
    '''
     * Update the various entities created by the doCreate method.
     *
     * @throws Exception if something goes wrong
     '''
    def doUpdate(self):
        # lookup and then ...
        agent = self.lookupAgent("Demo Agent")
        group = self.lookupGroup("Demo Agent Group")
        user = self.lookupUser("jdoe")
 
        # ... update
        self.updateAgent(agent)
        print ("Updated Demo Agent")
        self.updateGroup(group)
        print ("Updated Demo Agent Group")
        self.updateUser(user)
        print ("Updated user jdoe")
 
    '''
     * Disable password history limit on default password policy so
     * we can issue multiple updates for the user password.
     *
     * @throws Exception if something goes wrong
     '''
    def doDisablePasswordHistory(self):
        # lookup and then ...
        policy = self.lookupPasswordPolicy("Initial")
 
        # ... update
        self.updatePasswordPolicy(policy)
        print ("Disabled password history")
 
# Globals here
'''
 * Show usage message and exit.
 * 
 * @param msg the error causing the exit
'''
def usage(msg):
    print ("ERROR: " + msg)
    print ("Usage: APIDemos <create|delete> <admin username> <admin password>")
    sys.exit(1)
 
'''
 * Use from command line with three arguments.
 * 
 * <p>
 * First argument:
 * create - to create the required entities
 * assign - to assign the next available token to the user
 * delete - to delete all created entities
 * </p>
 * <p>
 * Second argument is the administrator user name.
 * Third argument is the administrator password.
 * </p>
 * 
 * @param args the command line arguments
'''
 
if len(sys.argv) != 4:
    usage("Missing arguments")
 
# skip script name
args = sys.argv[1:]
 
# establish a connected session with given credentials
conn = ConnectionFactory.getConnection()
session = conn.connect(args[1], args[2])
 
# make all commands execute using this target automatically
CommandTargetPolicy.setDefaultCommandTarget(session)
 
 
try:
    # create instance
    api = AdminAPIDemos()
    # call delusers before addusers
    print "Deleting users...\n"
    api.doDelete()
    print "Adding users...\n"
    api.doAdd()
 
finally:
    # logout when done
    session.logout()

I of course worked from their demo file, AdminAPIDemos.py, and kept the name for simplicity. I added a a doAdd routine and modified their doDelete function.

These modified functions expect external files to exist, addusers.txt and delusers.txt. The syntax of addusers.txt is:

loginname1,first_name,last_name
loginname2,first_name,last_name
...

Delusers.txt has the same syntax.

The idea is that if you can create these files once per day with the new users/removed users from your corporate directory by some other means, then you have a way to use them as a basis for keeping your AM internal database in sync with your external enterprise directory, whatever it might be.

Other Notes
Initially I saw my users were set to expire after a year or so. The original code I borrwed from had lines like this:

        cal = Calendar.getInstance()
 
        # the start date
        now = cal.getTime()
 
        cal.add(Calendar.YEAR, 1)
 
        # the account end date
        expire = cal.getTime()

which caused this. I eventually found how to set a flag to create the account with unlimited validity.

I also introduced a very simple regex handling to break up the input lines. This caused the need for importing additional classes:

from java.util.regex import *
from java.lang import *

I could not get python regexes to work.

I also found these three innocent-looking lines were costing me a license unit for each added user:

        principal.setStaticPassword("12345678")
        principal.setStaticPasswordSet(1)
        principal.setWindowsPassword("Password123!")

So I commented them out as I did not need them.

That’s it!

Getting the SDK running cost me a few days but at least I’ve documented that as well in pretty good detail: Problems with Jython API for RSA Authentication Manager.

Conclusion
We’ve shared with the community an actual, working jython API for adding/removing users from an RSA Authentication Manager v 7.1 database.

Categories
Admin Security

Problems with Jython API for RSA Authentication Manager

Intro
This session is not for the faint-of-heart. I describe some of the many issues I had in trying to run a slightly modified jython program which I use to keep the local directory in sync with an external directory source. Since I am a non-specialist in all these technologies, I am describing this from a non-specialist’ point-of-view.

The Details
RSA provides an authentication manager sdk, AM7.1_sdk.zip, which is required to use the API.

I had it all working on our old appliance. I thought I could copy files onto the new appliance and it would all work again. It’s not nearly so simple.

You log on and su to rsaadmin for all this work.

Let’s call RSAHOME = /usr/local/RSASecurity/RSAAuthenticationManager.

Initially the crux of the problem is to get
$RSAHOME/appserver/jdk/samples/admin/src/AdminAPIDemos.py to run.

But how do you even run it if you know nothing about java, python, jython, and, IMS and weblogic? Ha. It isn’t so easy.

As you see from the above path I unpacked the sdk below appserver. I did most of my work in the $RSAHOME/appserver/jdk/samples/admin directory. First thing is to very carefully follow the instructions in the sdk documentation about copying files and about initializing a trust.jks keystore. You basically grab .jar files from several places and copy them to $RSAHOME/appserver/jdk/lib/java. Failure to do so will be disastrous.

Ant is apparently a souped-up version of make. You need it to compile the jython example. They don’t tell you where it is. I found it here:

$RSAHOME/appserver/modules/org.apache.ant_1.6.5/bin/ant

I created my own run script I call jython-build:

#!/bin/bash
Ant=/usr/local/RSASecurity/RSAAuthenticationManager/appserver/modules/org.apache.ant_1.6.5/bin/ant
# compiles the examples
#$Ant compile
#$Ant verify-setup
# this worked!
#$Ant run-create-jython
$Ant run-add-jython

This didn’t work at first. One error I got:

$Ant verify-setup
Error: JAVA_HOME is not defined correctly.
  We cannot execute java

OK. Even non-java experts know how to define the JAVA_HOME environment variable. I did this:

$ export JAVA_HOME=/usr/local/RSASecurity/RSAAuthenticationManager/appserver/jdk

I did a

$ $Ant verify-setup

and got missing com.bea.core.process.5.3.0.0.jar. I eventually found it in …utils/jars/thirdparty. And so I started copying in all the missing files. This was before I discovered the documentation! Copying all the jar files can get you so far, but you will never succeed in copying in wlfullclient.jar because it doesn’t exist! Turns out there is a step described in the sdk documentation which you need to do that creates this jar file. Similarly, trust.jks, your private keystore, does not exist. You have to follow the steps in the sdk documentation to create it with keytool, etc. You’ll need to augment your path, of course:

$ export PATH=$PATH:/usr/local/RSASecurity/RSAAuthenticationManager/appserver/jdk/bin

Some errors are experienced in and around this time:

org/apache/log4j/Appender not found
org/apache/commons/logging not found

This was when I was copying jar files in one-by-one and not following the sdk instructions.

I also got

weblogic.security.SSL.TrustManager class not found. This was more vexing at the time because it didn’t exist in any of my jar files! This is when I discovered that wlfullclient.jar has to be created by hand. It contains that class. Here’s how I check the jar contents:

$ jar tvf wlfullclient.jar|grep weblogic/security/SSL/Trust

   481 Wed May 09 18:12:02 EDT 2007 weblogic/security/SSL/TrustManager.class

For the record, my …lib/java directory, which has a few more files than it actually needs, looks like this:

activation-1.1.jar                commons-pool-1.2.jar                    opensaml-1.0.jar
am-client.jar                     commons-validator-1.3.0.jar             oscache-2.3.2rsa-1.jar
am-server-o.jar                   console-integration-api.jar             replication-api.jar
ant-1.6.5.jar                     dbunit-2.0.jar                          rsaweb-security-3.0.jar
antlr-2.7.6.jar                   dom4j-1.6.1.jar                         rsawebui-3.0.jar
asm-1.5.3.jar                     EccpressoAsn1.jar                       serializer-2.7.0.jar
axis-1.3.jar                      EccpressoCore.jar                       spring-2.0.7.jar
axis-saaj-1.3.jar                 EccpressoJcae.jar                       spring-mock-2.0.7.jar
c3p0-0.9.1.jar                    framework-common.jar                    store-command.jar
certj-2.1.1.jar                   groovy-all-1.0-jsr-05.jar               struts-core-1.3.5.jar
cglib-2.1_3.jar                   hibernate-3.2.2.jar                     struts-extras-1.3.5.jar
classpath.jar                     hibernate-annotations-3.2.1.jar         struts-taglib-1.3.5.jar
classworlds-1.1.jar               hibernate-ejb-persistence-3.2.2.jar     struts-tiles-1.3.5.jar
clu-common.jar                    hibernate-entitymanager-3.2.1.jar       systemfields-o.jar
com.bea.core.process_5.3.0.0.jar  ims-server-o.jar                        trust.jks
commons-beanutils-1.7.0.jar       install-utils.jar                       ucm-clu-common.jar
commons-chain-1.1.jar             iScreen-1-1-0rsa-2.jar                  ucm-server-o.jar
commons-cli-1.0.jar               iScreen-ognl-1-1-0rsa-2.jar             update-instance-node-ext.jar
commons-codec-1.3.jar             jargs-1.0.jar                           wlcipher.jar
commons-collections-3.0.jar       javassist-3.9.0.GA.jar                  wlfullclient.jar
commons-dbcp-1.2.jar              jboss-archive-browsing-5.0.0.Alpha.jar  wrapper-3.2.1rsa1.jar
commons-digester-1.6.jar          jdom-1.0.jar                            wsdl4j-1.5.1.jar
commons-discovery-0.2.jar         jline-0.9.91rsa-1.jar                   xalan-2.7.0.jar
commons-fileupload-1.2.jar        jsafe-3.6.jar                           xercesImpl-2.7.1.jar
commons-httpclient-3.0.1.jar      jsafeJCE-3.6.jar                        xml-apis-1.3.02.jar
commons-io-1.2.jar                jython-2.1.jar                          xmlsec-1.2.97.jar
commons-lang-2.2.jar              license.bea                             xmlspy-schema-2006-sp2.jar
commons-logging-1.0.4.jar         log4j-1.2.11rsa-3.jar
commons-net-2.0.jar               ognl-2.6.7.jar

I don’t know if these steps are necessary, but they should work at this stage:

$Ant compile
$Ant verify-setup
$Ant run-create-jython

But are we done? No way. Don’t forget to edit your config.properties:

# JNDI factory class.
java.naming.factory.initial = weblogic.jndi.WLInitialContextFactory
 
# Server URL(s).  May be a comma separated list of URLs if running against a cluster
# NOTE: Replace authmgr-test.drj.com with the hostname of the managed server
java.naming.provider.url = t3s://authmgr-test.drj.com:7002
 
# User ID for process-level authentication.
# run rsautil manage-secrets --action list to learn these
#
com.rsa.cmdclient.user = CmdClient_blahblahblah
 
# Password for process-level authentication
com.rsa.cmdclient.user.password = blahblahblah
 
# Password for Two-Way SSL client identity keystore
com.rsa.ssl.client.id.store.password = password
 
# Password for Two-Way SSL client identity private key
com.rsa.ssl.client.id.key.password = password
 
# Provider URL for Two-Way SSL client authentication
ims.ssl.client.provider.url = t3s://authmgr-test.drj.com:7022
 
# Identity keystore for Two-Way SSL client authentication
ims.ssl.client.identity.keystore.filename = client-identity.jks
 
# Identity keystore private key alias for Two-Way SSL client authentication
ims.ssl.client.identity.key.alias = client-identity
 
# Identity keystore trusted root CA certificate alias
ims.ssl.client.root.ca.alias = root-ca
 
# SOAPCommandTargetBasicAuth provider URL
ims.soap.client.provider.url = https://authmgr-test.drj.com:7002/ims-ws/services/CommandServer

As it says you need to run

$ rsautil manage-secrets –action list

or

$ rsautil manage-secrets –action listkeys

to get the correct username/password. The URLs also need appropriate tweaking of course. Failure to do these things will produce some fairly obvious errors, however. Strangely, the keystore-related values which look like placeholders since they say “password” really don’t have to be modified.

Try to run jython-build now and you may get, like I did,

Cannot import name AddGroupCommand

which is a clear reference to this line in the python file:

from com.rsa.admin import AddGroupCommand

Rooting around, I find this class in ims-server-o.jar which I already have in my …lib/java. So I decided to make a ~/.api file which contains not only the JAVA_HOME, but a CLASSPATH:

export JAVA_HOME=/usr/local/RSASecurity/RSAAuthenticationManager/appserver/jdk
export CLASSPATH=/usr/local/RSASecurity/RSAAuthenticationManager/appserver/jdk/lib/java/ims-server-o.jar

Actually my ~/.api file contains more. I just borrowed $RSAHOME/utils/rsaenv and added those definitions, but I’m not sure any of the other stuff is needed.

Are we there yet? Not in my case. Now we get a simple “Access Denied.” I was stymied by this for awhile. I went to reporting and found this failure logged under Authentication Monitor:

(Description) User admin attempted authentication using authenticator RSA_password.

(Reason) Authentication method failed.

What this was about is that I had forgot to update my build.xml file with the correct username and password:

    <property name="admin.name" value="admin"/>
    <property name="admin.pw" value="mypassword"/>

After correcting that, it began to work.


Update after several months

Well, after some months of inattention, now once again it doesn’t work! I checked my adduser log file and saw this nugget in the traceback:

[java] File "src/AdminAPIDemos.py", line 846, in ?
[java] com.rsa.authn.AuthenticationCommandException: Access Denied

The only other clue I have is that another administrator changed the Super Admin password a couple weeks ago as it was expired. I haven’t resolved this one yet, it’s a work in progress! Ah. Simple. I needed to update my build.xml with the latest admin password. Hmm. That could be kind of a pain if it’s expiring every 90 days.

Conclusion
I was battered by this exercise, mostly by my own failure to read the manual. But some things are simply not documented. Why all the fuss just to get demo code working? Because we can customize it. I felt that once I had the demo running, the sky was the limit and creating customization to do automated add/delete would not be that difficult.

To see the jython script I created to do the user add/deletes click on this article: Add/Delete Jython Script for RSA Authentication Manager