Intro
In my years at cybersecurity I’ve been sufficiently removed from the action that I’ve rarely been involved in an actual case. Until last night. A friend, whom I’ll call Jute, got a formal complaint about one of his hosted Windows servers.
We have detected multiple hacking attempts from your ip address 47.5.105.236 (Hilfer Online) to access our systems.
>
> Log of attempts:
> – Hack attempt failed at 2019-01-17T14:22:41.6539784Z. Attempted user name: Not specified (typical for port scanners or denial of service attacks), system accessed: RDP, ip address accessed: 158.69.241.92
> – Hack attempt failed at 2019-01-17T14:22:26.2213808Z. Attempted user name: Not specified (typical for port scanners or denial of service attacks), system accessed: RDP, ip address accessed: 158.69.241.92
> – Hack attempt failed at 2019-01-17T14:22:10.6304194Z. Attempted user name: Not specified (typical for port scanners or denial of service attacks), system accessed: RDP, ip address accessed: 158.69.241.92
>
>
> Please investigate this problem.
>
> Sent using IP Ban Pro
> http://ipban.com
Hack, cont.
I’ve changed the IPs to protect the guilty! But I’ve conveyed the specificity of the error reporting. Nice and detailed.
Jute has a Windows Server 2012 at that IP. He is not running a web server, so that conveniently and dramatically narrows the hackable footprint of his server. I ran a port scan and found ports 135, 139 and 3389 open. His provider (which is not AWS) offers a simple firewall which I suggested we use to block ports 135 and 139 which are for Microsoft stuff. He was running it as a local sever so I don’t tihnk he needed it.
Bright idea: use good ole netstat
The breakthrough came when I showed him the poor man’s packet trace:
netstat -an
from a CMD prompt. He ran that and I saw not one but two RDP connections. One we easily identified as his, but the other? It was coming form another IP belonging to the same provider! RDP is easily identified by just looking for the port 3389 connections. Clearly we had caught first-hand an unauthorized user.
I suggested a firewall rule to allow only his Verizon range to connect to server on port 3389. But, I am an enterprise guy, used to stateful firewalls. When we set it up we cut off his RDP session to his own server! Why? I quickly concluded this was amateur hour and a primitive, ip-chains-like stateless firewall. So we have to think about rules for each packet, not for each tcp connection.
Once we put in a rule to block access to ports 135 and 139, we also blocked Jute’s own RP session. So the instructions said once you use the firewall, an implicit DENY ALL is added to the bottom of the rules.
So we needed to add a rule like:
SRC: ANY DST: 47.5.105.236 SRC_PORT: ANY DST_PORT: 3389 ACTION: allow
But his server needs to access web sites. That’s a touch difficult with a stateless firewall. You have to enter the “backwards” rule (outbound traffic is not restricted by firewall):
SRC: ANY DST: 47.5.105.236 SRC_PORT: 443 DST_PORT: ANY ACTION: allow
But he also needs to send smtp email, and look up DNS! This is getting messy, but we can do it:
SRC: ANY DST: 47.5.105.236 SRC_PORT: 25 DST_PORT: ANY ACTION: allow
SRC: ANY DST: 47.5.105.236 SRC_PORT: 53 DST_PORT: ANY ACTION: allow
We looked up the users and saw Administrator and another user Update. We did not recognize Update so he deleted it! And changed the password to Administrator.
Finally we decided we had to bump this hacker.
So we made two rules to allow him but deny the zombie computer:
SRC: 158.69.240.92 DST: 47.5.105.236 SRC_PORT: ANY DST_PORT: 3389 ACTION: reject
SRC: ANY DST: 47.5.105.236 SRC_PORT: ANY DST_PORT: 3389 ACTION: allow
Pyrrhic victory
Success. We bumped that user right out while permitting Jute’s access to continue. The bad news? A new interloper replaced it! 95.216.86.217.
OK. So with another rule we can bump that one too.
Yup. Another success. another interloper jumps on in its place. 124.153.74.29. So we bump that one. But I begin to suspect we are bailing the Titanic with a thimble. It’s amazing. Within seconds a blocked IP is being replaced with a new one.
We need a more sweeping restriction. So we reasoned that Jute will RP from his provider where his IP does not really change.
So we replace
SRC: ANY DST: 47.5.105.236 SRC_PORT: ANY DST_PORT: 3389 ACTION: allow
with
SRC: 175.198.0.0/16 DST: 47.5.105.236 SRC_PORT: ANY DST_PORT: 3389 ACTION: allow
and we also delete the specific reject rules.
But now at this point for some reason the implicit DENY ALL rule stops working. From my server I could do an nc -v 47.5.105.236 3389 and see that that port was open, though it should ont have been. So we have to add a cleanup rule at the bottom:
SRC: ANY DST: 47.5.105.236 SRC_PORT: ANY DST_PORT: ANY ACTION: reject
That did the trick. Port no longer opened.
There still appears in netstat -an listing the last interloper, but I think it just hasn’t been timed out yet. netstat -an also clearly shows (to me anyway) what they were doing: scanning large swaths of the Internet for other vulnerable servers! The tables were filled with SYN-SENT to port 3389 of consecutive IPs! Amazing.
So I think Jute’s server was turned into a zombie which was tasked with recruiting new zombies.
We had finally frozen them out.
Later that night
Late that same night he calls me in a panic. He uses a bunch of downstream servers and that wasn’t working! The downstream servers run on a range of ports 14800 – 15200.
Now bear in mind the provider only permits us 10 firewall rules, so it’s getting kind of tight. But we manage to squeeze in another rule:
SRC: ANY DST: 47.5.105.236 SRC_PORT: 14800-15200 DST_PORT: ANY ACTION: allow
He breathes a sigh of relief because this works! But I want him we are opening a slight hole now. Short term there’s nothing we can do. It’s a small exposure: 400 open ports out of 65000 possible. It should hold him for awile with any luck.
He also tried to apply all updates at my suggestion. I’m still not sure what vulnerability was (is) exploited.
Case: tentaively closed
Our first attempt to use the Windows firewall itself was not initially successful. We may return to it.
Conclusion
We catch a zombie computer totally exploiting RDP on a Windows 2012 server. We knocked it off and it was immediately repaced with another zombie doing the same thing. Their task was to find more zombies to join to the botnet. Inbound firewall rules defined on a stateless firewall were identified which stopped this exploitation while permitting desired traffic. Not so easy when you are limited to 10 firewall rules!
This is a case where IPBAN did us a favor. The system worked as it was supposed to. We got the alert, and acted on it immediately.
I’m not 100% sure which RP vulnerability was exploited. It may have been an RCE – remote code execution not even requiring a valid logon.
References and related
The rest of the security world finally caught up with this, with Microsoft releasing a critical patch in May. I believe I was one of the first to publicly document this exploit. https://nakedsecurity.sophos.com/2019/06/10/the-goldbrute-botnet-is-trying-to-crack-open-1-5-million-rdp-servers/?utm_source=Naked+Security+-+Sophos+List&utm_campaign=0fd82f7fce-Naked+Security+-+June+test+-+groups+1+and+3&utm_medium=email&utm_term=0_31623bb782-0fd82f7fce-418487137