Tag: Checkpoint

Categories
Firewall

Checkpoint Gaia admin tips

Intro

Suppose, hypothetically, that you had super admin access to a CMA in SmartConsole v 80.40, but lacked ssh or GUI access to firewalls within that CMA? What could you do? Can you run commands in a pinch? Yes. You can. Here are some concrete examples.

Caveats

In the servers section of the domain you can right-click and choose “Run one-time script.” That’s great, but I think there are limits. It will time out a script that takes too long. IDK, maybe 10 seconds or so is the maximum time allowed. The returned text gets truncated if it’s too long. 15 lines of text is OK. 200 is not. Somewhere inbetween those two is the limit.

Running clish commands

clish commands can indeed be run this way. I was interested in examining a few routes on a firewall with many static routes. I ran:

netstat -rn|grep 198.23|head -15

Set a static route

clish -sc “set static-route 197.6.75.0/24 nexthop gateway address 10.23.42.10 on”

Redistribute this route via BGP

clish -sc “set route-redistribution to bgp-as 38002.48928 from static-route 197.6.75.0/24 on”

Run a PING (best to restrict the number of ping packets)

ping -c3 1.1.1.1

Show a part of configuration, e.g., BGP stuff

clish -c “show configuration”|grep bgp|head -15

Show cluster IPs

cphaprob -vs all -a -m if|grep 10.182.136

Learn the name of the switch and switch port an interface is connected to (Cisco switches only)

This is a really awesome trick. And it works. Maybe it relies on something clled CDP. Not sure. But you run it and it will tell you the hostname of the switch and the port, e.g., eth1/5.

tcpdump -vnni eth1-08 -s 1500 -c 1 ‘(ether[12:2]=0x88cc or ether[20:2]=0x2000) and not tcp and not udp and not icmp’

The interface name eth1-08 above is just an example.

This command is general-putpose and works with any device with any OS, assuming you can run a packet trace with tcpdump or equivalent. Very cool.

Conclusion

Real firewall admins I know fail to realize that even when they lack shell access to a firewall they can pretty issue any command they need if they use the one-time script option in SmartConsole. It just helps to follow along the lines of the examples above – limiting output, etc. Even clish config changes can be made! A common reason to be in this situation is to learn someone changed a password or cleaned up old accounts.

As a bonus I show how to identify the name of the switch a firewall is connected to as well as the switch port. The general-purpose command works on any OS.

Categories
Admin Firewall Proxy

Checkpoint SYN Defender: what you don’t know can hurt you

Intro

Our EDI group hails me last Friday and says they can’t reach their VANs, or at best intermittently. What to do, what to do… I go on the offensive and say they have to stop using FTP (and that’s literal FTP, not sftp, not FTPs, just plain old FTP), it’s been out of date for at least 15 years.

But that wasn’t really helping the situation, so I had to dig a lot deeper. And frankly, I was coincidentally having intermittent issues with my scripted speedtests. Could the two be related?

The details

We have a bunch of synthetic monitors we run though that same firewall. They were failing every few minutes, and then became good.

And these FTPs were like that as well. Some would work and then minutes later not work.

The firewall person on call looked at the firewall, saw some of the described traffic passing through, and declared firewall is fine.

So I got a more cooperative firewall colleague on this. And he got a really expert Checkpoint support person on the call. That guy led us to look at SYN DEFENDER which is part of IPS and enabled via fw accel. If it sees too many out of state packets in a given time it will shut down the interface where the problem was observed!

The practical effect is that even if you’re taking traces on the Checkpoint, checking the logs, etc, you won’t see the traffic! So that really throws most firewall admins is this situation is so unusual and they are not trained to look for it.

In this case it was an internal firewall and ir was comfortable to disable SYN DEFENDER on it. All problems went away after that.

Four months later…

Then four months later, after the firewall was upgraded to v 81.10, they must have set SYN DEFENDER (AKA synatk) up all over again. And of course no one was thinking about it or expecting what happened next, which is, these exact same problems started all over again. But there were different firewall colleagues involved, none with any first-hand experience of the issue. Then I got involved and just sort of tackled my way through it in a trouble-shooting session. No one was placing any judgments (my-stuff-is- fine,-yours-must-be-broken kind of thinking). Then I eventually recalled the old problem, and looked up this post to help name it – SYN DEFENDER – so that that would be meaningful to the firewall colleague. Yup, he took it from there. And we were good. I admonished the on-call guy who totally missed it, and he humbly admitted to not being familiar with this feature and how does it work. So I will explain it to him.

Results of running fwaccel synatk config:

enabled 0
enforce 0
global_high_threshold 10000
periodic_updates 1
cookie_resolution_shift 6
min_frag_sz 80
high_threshold 5000
low_threshold 1000
score_alpha 100
monitor_log_interval (msec) 60000
grace_timeout (msec) 30000
min_time_in_active (msec) 60000

These are probably the defaults as we haven’t messed with them. Right now you see it’s disabled. It spontaneously re-enabeld itself after only a few days, and the problems started all over again.

References and related

VAN: Value Added Network

Categories
Network Technologies

How we got a little extra oomph from our firewall cluster, and why this trick no longer works for us

Intro
I was a running some Checkpoint firewalls in a cluster. In fact it’s been that way for years and years. At some point you get comfortable and forget to challenge and understand how it was set up. In this case re-examining the setup rewarded us with temporary survival as we were able to offload the primary member. Read on for the details…

The details
This firewall cluster included an active/standby pairing – a Nokia cluster with no state sync. The active firewall, an older model, was often hitting 99% or even 100% cpu usage on a daily basis. Dropped packets were correlated with these cpu spikes, and time-sensitive protocols, especially SIP used by IP phones, suffered mightily. Call quality often degraded, or the call was altogether dropped.

Some other relevant facts in this case: these firewalls were not doing NAT, they were acting more like routers with a firewall function. There are a handful of key servers behind them, like a VPN concentrator, a proxy, a Juniper ISG VPN concentrator, etc. On the external side was an Internet router, also under our control.

So the breakthrough was in revisiting what makes them active/passive in the first place. We weren’t relying on Checkpoint clustering. We used VRRP, defined through a Voyager setup. Then we set up our routing on all protected devices to use these VRRP IPs for their default routes. It all worked great until more and more usage crept in and then complaints started rolling in.

Upgrading costs $$ and the procurement cycle takes some time. What to do immediately, if anything?

The loudest complaints were from users of the Juniper ISG SSL VPN concentrators, who ran VOIP over those connections. What I realized (which of course is obvious in hindsight), is that this device could have all its traffic routed to the standby firewall where there was no cpu load whatsoever, and leave everything else on the active firewall.

How we did it
This was accomplished by adjusting the default route of the ISG to use the physical IP of the standby firewall, as opposed to the VRRP IP. Then, to avoid asymmetric routing, a host route was defined on the Internet router for this ISG, using as gateway the physical external IP of the standby firewall (again as opposed to the external VRRP IP.)

How it worked
It worked like a charm. We were well below our Internet link capacity, after all. So the master firewall was really the chokepoint for this voice traffic. Once we got it onto this unused firewall all the complaints stopped.

This is of course just a stop-gap measure because of course now we have no redundancy if we lose one of the firewalls. But meanwhile we’ve bought some time and kept the work-from-home users running smoothly. The master firewall still hits 99% cpu, but not quite as frequently. It’s difficult to find a true root cause, but an upgrade is definitely in order. Acceleration is already in place.

Why it won’t work for you – Checkpoint Cluster
Fast-forward five years and I tried this same trick which has served me well over the years. No worky. Why? Well these days we’ve switched to use of a Checkpoint Cluster with SYNC. In a Checkpoint cluster the secondary firewall will not forward traffic. In fact a firewall guy was the first to inform me of that. I didn’t believe him so I went ahead and configured it anyway. Sure enough, it simply didn’t work.

So for us, this trick has played itself out. But we used it multiple times during the five years it was available to us.

Conclusion
By re-visiting some old design principles were we able to get a little more mileage out of our firewalls and buy ourselves some time until we can do a planned upgrade.