Tag: ftp

Categories
Admin Linux Network Technologies

The IT Detetecive Agency: the case of the unreliable FTP

Intro
So one of my power users complains that his FTPs to a particular site fail frequently, but not always. I rolled up my sleeves and set to work. The thing I do best is find the essence of a problem – what is the bare minimum sequence of events that reproduces it. I’m still getting my head around this one and I haven’t cracked the case yet, but I’ve learned about a few obscure packet generation tools.

The details
I may flesh this out later. The essence of the thing is that a packet trace (using tcpdump) shows that randomly no SYN-ACK packet is returned for our SYN packet to the FTP server on port 21. The FTP server resides on the Amazon cloud, but on the West Coast. We are on the East Coast. Not that that matters.

So I learned to reproduce the problem myself with the built-in ftp client. But I wanted even more control.

Packet generation tools
My trajectory went kind of like this:

ftp -> ping -> nmap -> hping3 -> mausezahn (-> scapy)

I had to compile mausezahn but I did manage to get it to work. I guess the developer has passed away. It doesn’t offer complete control over tcp packet generation, but nearly so.

I just discovered scapy. It appears to offer complete control over packet generation, including the tcp options such as mss, but the proof is in the pudding and I haven’t had time to check it out.

See the references fo links to further information about where to find these packages.

Preliminary findings
I began to see that with nmap and hping3 I was getting SYN-ACKs back consistently. What’s the difference between their SYN packets and ftp’s? They don’t use any options whereas my ftp client does.
And that is the essence of the problem. A tcp SYN packet which sets options like SACK, wscale and MSS is not being responded to around 30% of the time. No options set? SYN-ACKs come back 100% of the time. Pings are answered 99 – 100% of the time. mausezahn (mz) allows to set the window size. The window size is irrelevant.

Is it one particular tcp option that is the culprit, or just the fact of using any of them? Unfortunately that’s where you reach the limits of mz. mz only allows you to turn on or off all options. scapy promises to be more granular. So at least with mz by itself I can turn of/off the problem at will. That is getting to the essence of the problem.

Another wrinkle? Only certain source IPs have the problem! I have an identical system using a different ISP and it works all day long.

Conclusion
A lot of work and only modest progress to show for it. I need cooperation of the ftp administrator to do a simultaneous trace. Either the packet never gets to him, or his infrastructure discards it, or he responds but I never see the response. A two-sided trace will narrow down which of these three things is happening.

But I did learn that fine-control packet generation is a bit difficult to come by, which comes as a surprise in this day and age. You have to do some work to get full control over your packets.

I have nos stomach for writing my own C++ code to have total control.

It’s still an open case.

References
nmap.org talks about nmap. nmap is a pretty standard package available for major distributions. But it is not sufficiently configurable.
I’ve written about hping3 before, showing how to compile it.
I used this site for mausezahn source code and documentation.
scapy is well-documented here.

Categories
Admin Network Technologies

Extended Passive Mode FTP through Checkpoint Firewall

Intro
The vast majority of time there is no problem doing an FTP to a server behind a firewall protected by Checkpoint’s Firewall-1. But occasionally there is.

The details
The problem I am about to document I think will only occur on a server that has multiple interfaces. I have seen it occur on multiple operating systems, so that doesn’t seem to matter. On the other hand, I have also not seen it on other similar systems, a point which I don’t fully yet understand.

Nevertheless, a work-around is always appreciated, so I provide what I found here, to complete my extensive documentation of problems I’ve encountered and resolved.

Here is a snippet from the FTP session showing the problem:

ftp> cd uploadDirectory
250 CWD command successful.
ftp> put smallfile.txt
local: smallfile.txt remote: smallfile.txt
229 Entering Extended Passive Mode (|||36945|)
200 EPRT command successful.
421 Service not available, remote server timed out. Connection closed

And here is the solution:

Enter epsv4 after logon and before any other commands are issued. Problem fixed!

Conclusion
We have shown a way to fix a firewall-related problem that manifests itself during extended passive mode FTPs. Some more research should be done to understand under what circumstances this problem should be expected, but it seems to occur with a Checkpoint Firewall-1 firewall and an FTP server with multiple interfaces.

Categories
Web Site Technologies

Security Considerations for WordPress Plugins and Upgrades

The following comments apply to WordPress v 3.1.3 and may not apply to earlier versions, with which I have no familiarity.

WordPress has an interesting idea for doing upgrades and downloading plugins. It took some getting used to until I learned to embrace it. I needed to understand the security considerations. Now I have a much better handle on it and feel comfortable with it.

First thing after installing WordPress, Murphy’s law you know, I was presented with an important security upgrade the very next day. I did the upgrade the hard way, doing all the file manipulation by hand. Copying files here and there, etc. I run the web server with a different user than the owner of the HTML documents to make things more secure. So I naively figured there was no way WordPress’s offer of automatically updating my installation would be possible in my case. After all all it could do was to run with the permissions of the web server, which as I say doesn’t have permissions to write to the relevant parts of the filesystem, right?

Then I learned that my colleagues on the Newton Robotics Team were managing to do it under the same conditions, so it piqued my curiosity. The next plugin I wished to install, WP-Syntax, offered me the same possibility of automatically installing it from the WordPress Admin GUI. It suggested that all I needed was to enter FTP credentials or use FTP/SSL. It did not explain how those credentials were going to be used, and I feared that they would be shared with another site.  Let’s think about this (this is how an IT person thinks).  There are two main possibilties. 1) The FTP client is initiated from an external site, probably where the repository where the plugin is housed, e.g., wordpress.org.  It was my gut feeling that was the case.  2) that the FTP client is on my local server where I run WordPress.  But, huh, what’s the point of that?

Turns out that 2) is what’s happening.  But then what is the point and how does it work?  By reverse engineering and reasoning, it must work as follows.  WordPress must download the plugin from the distribution site, perhaps through HTTP or FTP.  Perhaps it uses the FTP proxy feature where an intermediate can have an FTP connection to twp FTP servers and transfer files between them.  To expand it and put it into the local WordPress plugins directory, where the web server doesn’t have permissions to write, it definitely has to use FTP, but you gave it the credentials of the account that does have permissions to write to the plugins directory!  Clever, huh?  Of course this presupposes something.  Maybe if I read the WordPress requirements I would see that running an FTP server is strongly recommended. But I didn’t so this is another lesson learned through the school of hard knocks!  You see,  Ubuntu server and I think most linux distributions do not even bother to give you an FTP server.  Without a local FTP server WordPress cannot pull off its trick.  I’m not sure why they cannot use sftp, which is pretty universal these days.  In Ubuntu, you have the FTP client, but not the server.

I tried to run ftpd on my server to see what I would get.  It was missing and several packages which provide it were mentioned.  I chose inetutils-ftpd:  sudo apt-get install inetutils-ftpd.  I quickly learn that it relies on inetd, which I see I am not even running.  But it also has the option to run as a daemon: ftpd -D, which I chose to do (it won’t start after reboot without more jiggering, but I can start it by hand as I don’t need it often).

But how do I test my new FTP server?  Will it really work when WordPress tries to use it?  

Feb 2012 Update
I am now comfortable with directing WordPress to do my upgrade. I got tired of it bugging me about the 3.3.1 release so I relented and upgraded to it. I learned how to backup my database first, which is when I saw it was dominated by all the spam and scams I have been receiving. So I went back to the dashboard, got rid of 600 spam comments and re-ran the database mysqldump. The database dump file reduced in size from 10 MB to 3 MB! So it was 70% spam. Great people out there, huh? But I digress. I temporarily enabled my FTP daemon as described above and all went fine.

Then I enabled simple captcha challenge for POSTers. For now simple math seems to be flummoxing the auto-scam submitters! Next day my instance died. No idea why…