While my laptop was being shipped to me I wanted to be as productive as possible using my Samsung Galaxy A35. I was vaguely aware of the availability of Microsoft 365 apps such as Outlook. How far could I take this…?
The recipe
To cut to the chase, I was maybe 60 – 70 % effective. I used equipment found in the typical IT person’s home plus one inexpensive purchase from Walmart.
Here is what I used:
HDMI monitor
old Amazon firestick
cheap bluetooth keyboard purchased from Walmart
phone stand
And here’s what I really wished I had but did not:
bluetooth mouse
Which apps worked well:
Outlook
Teams, especiallt chat, less so the meetings function
One Note
Edge
VPN client
I must say the bluetooth keyboard worked really well for doing some serious typing up of emails.
How the external monitor worked
So I “came up” (in quotes because I’m sure many others figured out this same thin) with the idea of casting my phone screen onto an external monitor by way of the screen mirroring capacibility available on even the oldest amazon Firestick. On the phone you simply go to Smart View Mirror Screen.
So that prevented me from having to hold the phone at least while I was drafting emails.
But, and it’s a big one, is that the external monitor was not a TV and the sound from meetings was killed by this setup. And I did not see a way to keep audio local to the phone while only casting the screen.
A smaller problem is that the refresh lag is quite noticeable under conditions of rapid screen refresh. So it may take a second or two to show what the phone’s screen shows.
Still, it’s pretty cool.
I would have bought a bluetooth mouse but it simply wasn’t available at my local Walmart. I was pretty inconvenienced without it having to constantly touch the phone screen for various things.
And the external keyboard
Pretty well. Even some shortcuts worked. Alt-TAB, which I use a lot to switch between apps has some kind of vaguely similar effect on the phone, but not to the point where I could rely on it usefully. The unlock shortcut button sort of woke the up the phone screen at least.
TAB helped me to pop from one field in the form to the next the way I would use it on a PC.
Overall responseiveness was satisfactory.
The small form factor was not a detriment, and maybe even an advantage since it’s so light and portable.
What if you have an HP G5 docking station lying around?
Well I do. It has a USB-C cord which you normally plug into your HP laptop. But I didn’t have the power supply for it so I couldn’t use it when I would have needed it. Well, it basically works with a Samsung phone – at least the keyboard and mouse worked. In my 10 second testing the attached HDMI display did not automatically show anything. Maybe there are some phone settings which would need to be changed. I didn’t mess with it at all.
But it’s cool seeing a mouse working. It suddenly paints a mouse pointer on your phone screen which you can move around and click to launch an app.
Apps are often baby implmentations
At first I struggeled with the Outlook app, trying to use it as though it were my full-blown Outlook client on my PC. It only had one week’s worth of messages, which was pretty limiting since I was out for more than a week. Then I had a lightbulb moment and remembered that the Web version of Outlook worked on my phone. So I switched to using Outlook through the Edge browser – much better for me. That’s https://outlook.office.com/ . I could get full history and therefore do more reliable searching through messages.
Responsive Design work-around
Sometimes the mobile app version of a web site just doesn’t have the featuires, but looks nice. Edge has a feature you can choose called View Desktop Site which gives you the “real” web site. Now it may look tiny, forcing you to expand and shrink with two fingers. But at least it will generally work.
Where is Notepad or Notepad++
I didn’t look for an app. I suppose there is one. Somtimes you just want to inspect your clipboard. I settled on pasting into a new draft Outlook email to do my visual inspection of my clipboard.
References and related
I prepared the above solution with one day’s notice. If you had a couple days you might check out the Samsung Dex. I guess it would work for modern Samsung Galaxy phones though I haven’t tried it myself.
The web version of business Outlook, which is a pretty good implementtion of the full-blown client is https://outlook.office.com/
It’s convenient to name drop different types of cyber attacks at a party. I often struggle to name more than a few. I will try to maintain a running list of them.
But I find you cannot speak about cybersecurity unless you also have a basic understanding of information technology so I am including some of those terms as well.
As I write this I am painfully aware that you could simply ask ChatGPT to generate a list of all relevant terms in cybersecurity along with their definitions – at least I think you could – and come up with a much better and more complete list. But I refuse to go that route. These are terms I have personally come across so they have special significance for me personally. In other words, this list has been organically grown. For instance I plowed through a report by a major vendor specializing in reviewing other vendor’s offerings and it’s just incredible just how dense with jargon and acronyms each paragragh is: a motherlode of state-of-the-art tech jargon.
AiTM (Adversary in the Middle)
Baitortion
I guess an attack which has a bait such as a plum job offer combined with some kind of extortion? The usage was not 100% clear.
BYOVD (Bring Your Own Vulnerable Driver)
Clickfix infection chain
Upon visiting compromised websites, victims are redirected to domains hosting fake popup windows that instruct them to paste a script into a PowerShell terminal to fix an issue.
Collision attack
I.e., against the MD5 hash algorithm as done in the Blast RADIUS exploit.
Credential Stuffing Attack
I.e., password re-use. Takes advantage of users re-using passwords for different applications. Nearly three of four consumers re-use password this way. Source: F5. Date: 3/2024
Data Wiper
Authentication Bypass
See for instance CVE-2024-0012
Email bombing
A threat actor might flood a victom with spam then offer “assistance” to fix it.
Evasion
Malicious software built to avoid detection by standard security tools.
Password spraying
A type of attack in which the threat actor tries the same password with multiple accounts, until one combination works.
Port Scan
Host Sweep
Supply Chain attack
Social Engineering
Hacking
Hacktivist
I suppose that would be an activitst who uses hacking to further their agenda.
Living off the land
Data Breach
Keylogger
Darknet
Captcha
Click farms
Jackpotting
This is one of my favorite terms. Imagine crooks implanted malware into an ATM and were able to convince it to dispense all its available cash to them on the spot! something like this actually happened. Scary.
Overlay Attack
Example: When you open a banking app on your phone, malware loads an HTML phishing page that’s designed to look just like that particular app and the malware’s page is overlaid on top.
Payment fraud attack
In a recent example, the victim experienced “multiple fraudulently induced outbound wire transfers to accounts controlled by unknown third parties.”
Skimmer
XSS (Cross site Scripting)
bot
Anti-bot, bot defense
Mitigation
SOC
Selenium (Se) or headless browser
Obfuscation
PII, Personally Identifiable Information
api service
Reverse proxy
Inline
endpoint, e.g., login, checkout
scraping
Layer 7
DDOS
Carpet bombing DDOS attack
Many sources hitting many targets within the same subnet. See:
A social engineering attack where scammers target grandparents by pretending to be a grandchild in a bind.
GUI
(JavaScript) Injection
Command Injection
Hotfix
SDK
URL
GET|POST Request
Method
RegEx
Virtual Server
TLS
Clear text
RCA
SD-WAN
PoV
PoC
X-Forwarded-For
Client/server
Threat Intelligence
Carding attack
Source code
CEO Fraud
Phishing
Vishing
(Voice Phishing) A form of cyber-attack where scammers use phone calls to trick individuals into revealing sensitive information or performing certain actions.
Business email compromise (BEC)
Deepfake
Threat Intelligence
Social engineering
Cybercriminal
SIM box
Command and control (C2)
Typo squatting
Voice squatting
A technique similar to typo squatting, where Alexa and Google Home devices can be tricked into opening attacker-owned apps instead of legitimate ones.
North-South
East-West
Exfiltrate
Malware
Infostealer
Obfuscation
Antivirus
Payload
Sandbox
Control flow obfuscation
Buffer overflow
Use after free
Indicators of Compromise
AMSI (Windows Antimalware Scan Interface)
Polymorphic behavior
WebDAV
Protocol handler
Firewall
Security Service Edge (SSE)
Secure Access Service Edge (SASE)
Zero Trust
Zero Trust is a security model that assumes that all users, devices, and applications are inherently untrustworthy and must be verified before being granted access to any resources or data.
Zero Trust Network Access (ZTNA)
ZTA (Zero Trust Architecture)
Zero Trust Edge (ZTE)
Secure Web Gateway (SWG)
Cloud Access Security Broker (CASB)
Remote Browser Isolation (RBI)
Content Disarm and Reconstruction (CDR)
Firewall as a service
Egress address
Data residency
Data Loss Prevention (DLP)
Magic Quadrant
Managed Service Provider (MSP)
0-day or Zero day
User Experience (UX)
Watermark
DevOps
Multitenant
MSSP
Remote Access Trojan (RAT)
SOGU
2024. A remote access trojan.
IoC (Indicators of Compromise)
Object Linking and Embedding
(Powershell) dropper
Backdoor
Data Bouncing
A technique for data exfiltration that uses external, trusted web hosts to carry out DNS resolution for you
TTP (Tactics, Techniques and Procedures)
Infostealer
Shoulder surfing
Ransomware
Pig butchering
This is particularly disturbing to me because there is a human element, a foreign component, crypto currency, probably a type of slave trade, etc. See the Bloomberg Businessweek story about this.
A text-based interfaces that allow for remote server control.
Crypto Miner
RCE (Remote Code Execution)
Threat Actor
APT (Advanced Persistent Threat)
Compromise
Vulnerability
Bug
Worm
Remote Access VPN (RAVPN)
XDR (Extended Detection and Response)
SIEM (Security Information and Event Management)
User Entity Behavior Analytics (UEBA)
Path traversal vulnerability
An attacker can leverage path traversal sequences like “../” within a request to a vulnerable endpoint which ultimately allows access to sensitive files like /etc/shadow.
Tombstoning
Post-exploit persistence technique
Volumetric DDoS
MFA bomb
Bombard a user with notifications until they finally accept one.
Use-after-free (UAF)
A use-after-freevulnerability occurs when programmers do not manage dynamic memory allocation and deallocation properly in their programs.
Cold boot attack
A cold boot attack focuses on RAM and the fact that it is readable for a short while after a power cycle.
One of those annoying terms borrowed from the military that only marketing people like to throw around. It means what you think it might mean.
Blue Team – see Red Team
BSI (The German Federal Office for Information Security)
DLS (Data Leak Sites)
Sites where you can see who has had their data stolen.
Red Team
In a red team/blue team exercise, the red team is made up of offensive security experts who try to attack an organization’s cybersecurity defenses.
SNORT (Probably stands for something)
An open source rule-matching engine to scan network traffic and serve as an IDS.
IT terminology
2FA (2 Factor Authentication)
802.1x
ACL (Access Control List)
AD (Active Directory)
ADO (Azue DevOps)
AFK (Away From Keyboard)
AGI (Artificial General Intelligence)
AGI is the theory and development of computer systems that can act rationally.
AIOps
Applying AI to IT operations.
ANN (Artificial Neural Network)
Ansible
I would call it an open source orchestrator.
anti-aliasing
When you smooth out color in neighboring pixels.
anycast
Anydesk
A popular remote management software.
apache
A formerly popular open source web server which became bloated with features.
APM (Application Performance Management)
ARIN
ARM
A processor architecture from ARM Corporation, as opposed to, e.g., x86. Raspberry Pis use ARM. I think Androids do as well.
ARP (Address Resolution Protocol)
ASCII
An early attempt at representing alpha-numeric characters in binary. Was very english-focussed.
ASN (Autonomous System Number)
Each AS is assigned an autonomous system number, for use in Border Gateway Protocol routing
ASN.1 (Abstract Syntax Notation One)
A standard interface description language (IDL) for defining data structures that can be serialized and deserialized in a cross-platform way.
ASPA (Autonomous System Provider Authorization)
An add-on to RPKI that allows an ASN to create a record that lists which ASNs can be providers for that ASN. The concepts are “customer” (an ASN) and “providers” (a list of ASNs). This is used to do hop by hop checking of AS paths.
ASR (Aggregation Services Router)
A high-end Interent router offered by Cisco for business customers.
AV (anti-virus)
AWS (Amazon Web Services)
Azure AD
BGP (Border Gateway Protocol)
BIND (Berkeley Internet Name Daemon)
An open source implementation of DNS, found on many flavors of linux.
BOM (Bill of Material)
Boot start
A flag for a driver in Windows that tells it to always start on boot.
bootp
A predecessor protocol to DHCP.
broadcast
Browser
BYOD (Bring Your Own Device)
I.e., when employees are permitted to use their personal smartphone to conduct company business.
BYOL (Bring Your Own License)
F5 permits this approach to licensing one of their cloud appliances.
CA (Certificate Authority)
Callback
A routine designed to be called when someone else’s code is executing. At least that’s how I understand it.
CDR (Call Detail Record)
Metadata for a phone call.
CDN (Content Distribution Network)
CDP (Cisco Discovery Protocol)
This protocol allows devices connected to switch ports to learn what switch and which switch port they are connected to. It is a layer 2 protocol.
CDSS (Cloud Delivered Security Services)
Only used in Palo Alto Networks land.
CGN (Carrier Grade NAT)
The address space 100.64.0.0/10 is handled specially by ISPs for CGN. RFC 6598
CHAP
Chatbot
A computer program that simulates human conversation with and end user.
CI (Configuration Item)
CI/CD
An ITIL term referring to the object upon which changes are made.
CISA (Cybersecurity and Infrastructure Security Agency)
CISO (Chief Information Security Officer)
Cleartext
Format where no encryption has been applied.
CMDB (Configuration Management Database)
CMO (Current Mode of Operations)
CNN (Congruential Neural Network)
Computer Vision
A field of AI that leverages machine learning and neutral networks to enable machines to identify and understand visual information such as images and videos.
Copilot
Microsoft’s AI built into their productivity software. Sorry, no more Clippy.
Courrier
A well-known fixed-width font.
CPE (Customer Premise Equipment)
CSR (Certificate Signing Request)
CUPS (Common Unix Printing Systems)
Customer Edge (CE)
CVE
CVEs, or Common Vulnerabilities and Exposures, are a maintained list of vulnerabilities and exploits in computer systems. These exploits can affect anything, from phones to PCs to servers or software. Once a vulnerability is made public, it’s given a name in the format CVE–. There are also scoring systems for CVEs, like the CVSS (Common Vulnerability Scoring System), which assigns a score based on a series of categories, such as how easy the vulnerability is to exploit, whether any prior access or authentication is required, as well as the impact the exploit could have.
CVSS (Common Vulnerability Scoring System)
Part of CVE lingo.
DAST (Dynamic Application Security Testing)
Data at rest
Data in motion
Data Plane
A physical security appliance separates data traffic from its management traffic, which transits the managemenbt plane.
Data Remanence
The residual representation of data that remains even after attempting to erase or initialize RAM.
DDI (DNS, DHCP and IP address management)
Debian Linux
A nice distro which I prefer. It is free and open source. Its packages are relatively uptodate.
Deep Learning
A subset of machine learningthat focus on using deep neural networks with multiple layers to model complex patterns in data.
Deepfake
A manipulated video or other digital representation produced by sophisticated machine-learning techniquies that yield seemingly realistic, but fabricated images and sounds.
DHCP (Dynamic Host Control Protocol)
Distributed Cloud
A Gartner term for a SaaS service which runs over multiple cloud environments.
DLL
DLP (Data Loss Prevention)
DNAT (Destination NAT)
DNS (Domain Name System)
DNSSEC (Domain Name System Security Extensions)
DOA (Dead on Arrival)
Usage: That equipment arrived DOA!
Docker
DoH (DNS over HTTPS)
Domain
DRM (Digital Rights Management)
DVI (DeVice Independent file)
See LaTEX entry.
EAP
East-West
Data movement with a data center, I believe, as oppose to North-South.
EBITDA (Earnings Before Interest, Taxes, Depreciation and Amortization)
Hey, an IT person needs to know some business terminology!
Eduroam
Enhanced Factory Reset (EFR)
Entra
From Microsoft. The new name for Azure AD
EU AI Act
EULA (End User Licnese Agreement)
Exact Data Matching (EDM)
FAQ (Frequently Asked Questions)
Fedora Linux
Free and open source linux. New features are introduced here before migrating into Redhat Linux
FEX (Fabric Extender)
FIFO (First in, First Out)
FIPS (Federal Information Processing Standard)
Government security practices. Best to avoid if possible.
FMO (Future Mode of Operation)
As opposed to CMO.
FN (False Negative)
Forensics
Fortran
An ancient procedural programming language popular in the scientific and engineering communities from decades ago.
FOSS (Free and Open Source Software)
FP (False Positive)
freeBSD
A Unix variant which still exists today.
Fritz!Box
A popular home router in Germany.
GA (General Availability)
Gartner Group
A well-regarded research firm which reviews software and SaaS products. They decide which vendors are in the Magic Quadrant.
GBIC
A type of fiber optic transceiver that converts electric signals to optical signals.
GCP (Google Cloud Provider)
GDPR (General Data Protection Regulation)
An EU directive to achieve data privacy.
Generative AI
AI which can create new human-quality content, including text, images, audio or video.
GMP (Good Manufacturing Practice)
FDA lingo that implies their rules are being followed.
GMT – see UTC
GRE
GSLB (global Server Load Balancing)
GUI (Graphical User Interface)
HA (High Availability)
Hallucination
When an LLM perceives patterns that are non-existent creating nonsensical or inaccurate outputs.
Hands and Eyes
When you don’t have physical access to a server, you need someone who does to be this for you.
At least in the world of F5 this means IP Intelligence, i.e., the reputation of a given IP address.
IPS (Intrusion Prevention System)
IPSEC
IPv6 (Internet Protocol version 6)
iRule
F5 specific lingo for programmable control over load-balancing and routing decisions. Uses the TCL language.
ISC (Internet Software Consortium)
A body which maintains an open source reference implementation for DNS (BIND) and DHCP.
ISO 9001
ISP (Internet Service Provider)
ITIL (IT Infrastructure Library)
JSON (JavaScript Object Notation)
Pronounced JAY-son. A popluar format for data exchange. Sort of human-friendly. Example: {“hi”:”there”,”subnets_ignore”:[“10/8″,”192.168/16”]}
Kanban
Agile way of tracking progress on tasks and brief meetings.
Kernel mode
Kerning
Adjusting the spacing between letters in a proportional font.
KEV (Known Exploited vulnerabilities)
CISA maintains this catalog.
K8s (Kubernetes)
Open source system for automating deployment, scaling, and management of containerized applications
KVM (Kernel Virtual Module)
L2TP (Layer 2 Tunneling Protocol)
L3, L4, L7 (Layer 3, Layer 4, Layer 7)
Refers to ISO 7-layer traffic model.
LACP (Link Access Control Protocol)
LAMP (Linux Apache MySQL and PHP)
An application stack which gives a server needed software to do “interesting things.”
LaTEX
A markup language based on TEX I used to use to write a scientific paper. I think it gets transformed into a DVI, and then into a postscript file.
LEC (Local Exchange Carrier)
Link
Linux
LLD (Low Level Design)
LLD (Low Level Discovery)
A command-line browser for unix systems.
LLDP (Link layer Discovery Protocol)
See also CDP
An open source OS similar to Unix.
LLM (Large Langiuage Model)
lynx
A command-line browser for linux systems.
MAC (Media Access Control) Address
Layer 2 address of a device, e.g., fa-2f-36-b4-8c-f5
Machine Learning
A subfield of AI that deals with creating systems that can learn from data and improve their performance without explicit programming.
Magic Quadrant
Gartner’s term for vendors who exceed in both vision and ability to execute.
Management Plane
See Data Plane.
Mandiant
MD5 (Message Digest 5)
MDM (Mobile Device Management)
Management software used to administer smartphones and tablets.
MFA (Multi Factor Authentication)
MITRE ATT&CK
Modbus protocol
MS-CHAPv2
MSS (Maximum Segment Size)
Set by a TCP option in the beginning of the communcation.
MTTI (Mean Time To Identification)
Probably only Cisco uses this acronym e.g., in their ThousandEyes product.
MTTR (Mean Time To Resolution)
MTU (Maximum transmission unit)
Often 1500 bytes.
multicast
NAESAD (North American Energy Software Assurance Database)
Named pipes
I read it’s a Windows thing. huh. Hardly. It’s been on unix systems long before it was a twinkle in the eye of Bill gates. It acts like a pipe (|) except you give it a name in the filesystem and so it is a special file type. It’s used for inter-process communication.
NAT (Network Address Translation)
NDA (Non-Disclosure Agreement)
.NET
Netflow
Think of it like a call detail record for IP communications. Metadata for a communications stream.
NGFW (Next Generation FireWall)
Palo Alto Networks describes their firewalls this way.
NGINX
A web server that is superioir to apache for most applications.
NLP (Natural Language Processing)
A branch of AI that uses machine learning to enable computers to understand, interpret, and respond to human language.
NOC (Network Operations Center)
North-South
Data movement from/to the data center. Also see East-West.
NSA (National Security Agency)
NTLM
Relies on a three-way handshake between the client and server to authenticate a user.
OAuth bearer token
A security token with the property that any party in possession of the token (a “bearer“) can use the token in any way that any other party in possession of it can.
An online community that produces freely available articles, methodologies, documentation, tools, and technologies in the fields of IoT, system software and web application security.
PAP
Patch
PaaS (Platform as a Service)
PBR (Policy Based Routing)
PCI (Payment Card International?)
A standard which seeks to define security practices around the handling of credit cards.
PDF (Portable Document File)
PDU (Protocol Data Unit)
PE (Provider Edge)
Telecom lingo so cisco uses this term a lot.
PEM (Privacy Enhanced Mail)
The format certificates are normally stored in.
PHP (Probably stands for something)
A scripting language often used to program back-end web servers.
PII (Personally Identifiable Information)
PKCS (Public Key Cryptography Standard)
PKI (Public Key Infrastructure)
Plain Text
A human-readable format, i.e., no encyrption and not a binary file.
PLC (programmable logic controller)
PM (Product Manager)
Could also be Project Manager but for me it usually means Product Manager.
PO (Purchase Order)
POC (Point of Contact)
POC (Proof of Concept)
Port Channel
Portable Executable (PE)
POTS (Plain Old Telephone Service)
Voice-grade telephone service employing analog signal transmission over copper
POV (Proof of Value)
Private Cloud
Prompt Engineering
The practice of crafting effective prompts that elicit high-quality answers from generative AI tools.
PS (PostScript)
A file type I used to use. It is a vector-oriented language, stack-based, which tells the printer how to move its ink pens around the page. Before there was PDF, there was postscript.
PS (PowerShell)
A versatile scripting language developed by Microsoft and available on all Windows computers.
PTO (Paid Time Off)
Purple Team
Purple teams combine red team and blue team functions. See Red Team.
PyPi (Python Package Index)
Python
A popular programming language, not the snake.
QSFP (Quad Small Form factor Pluggable)
A newer kind of SFP.
Rack Unit
RADIUS
RAG (Retrieval Augmented Generation)
A method to train LLMs.
Ray
An open-source unified compute framework used by the likes of OpenAI, Uber, and Amazon which simplifies the scaling of AI and Python workloads, including everything from reinforcement learning and deep learning to tuning and model serving.
RBAC (Role-Based Access Control)
RDP (Remote Desktop Protocol)
Recursive
A function which calls itself.
Redhat Linux
A commercialized version of Fedora whose packages are always dated, usually by years.
Redirect
Remediation
Addressing a security flaw.
Remote Desktop Licensing (RDL) services
Often deployed on Windows severs with Remote Desktop Services deployed.
Retrieval-Augmented Generation (RAG)
Reverse Engineer
To figure out the basic building blocks or code by first observing behavior of a system.
Reverse Proxy
A TCP gateway which terminates a tcp connection and maintains a separate tcp connection to a back-end server.
RFC (Request for Comment)
RFI (Requst for Information)
RFO (Reason for Outage)
RFP (Request for Proposal)
RFQ (Request for Quote)
RIPE
RIR (Regional Internet Registry)
RMA (Return Merchandise Authorization)
You hear this a lot when It guys need to get a replacement for failed equipment.
RMM (ReMote Management)
ROA (Route Origin Authorization)
ROCE (Return on Capital Employed)
Hey, an IT person has to know a few business terms!
Route 53
In AWS-land, an intellugent DNS service, i.e., geoDNS +.
RPC (Remote Procedure Call)
RPKI (Resource Public key Infrastructure)
Provides a way to connect Internet number resource information to a trust anchor.
RPi (Raspberry Pi)
A popular small, inexpensive server aimed at the educational crowd.
RPZ (Response Policy Zone)
A concept in DNS for either a DNS firewall or way to overwrite DNS responses.
RR (Resource Record)
RSA
Asymmetric encryption standard named after its creators, Ron Rivest, Adi Shamir and Leonard Adleman.
RTFM (Read The “flippin'” Manual)
SaaS (Software as a Service)
SAML
SANS
Private outfit in the US which specializes in information security and cybersecurity training.
Sans-Serif
A font type which does not have the fancy rounded blobs at the tips of the letter, such as Helvetica.
SASE (Secure Access Service Edge)
SCADA
Scale sets
In cloud, a service which automates the build-up or tear-down of VMs behind a load balancer.
SDWAN (Software defined WAN)
SEO (Search Engine Optimization)
SFP (Small Form factor Pluggable)
A type of optic transceiver that converts electric signals to optical signals.
SGML (Standard Generalized Markup Language)
If you ask the French they proudly point to this as the predeccesor to the more widely known HTML.
SFTP (Secure file Transfer Protocol)
SHA (Secure Hash Algorithm)
SIEM (Security Information and Event Management)
SRE (Site Reliability Engineer)
SMP (Symmetric Multi Processing)
SMTP (Secure Mail Transfer Protocol)
SNAT (Source NAT)
SNI (Server Name Indication or similar, I think)
When multiple HTTP[S web sites whare a single IP this technology can be used to identify which certificate to send to a requester.
SNMP (Simple Network Management Protocol)
All security appliances support this protocol which permits system monitoring.
Spoofing
When a source IP address is faked.
SSH
SSL (Secure Socket Layer)
SOC (System on a Chip)
I believe the RPi is described to be this.
SOC (Security Operations Center)
Solaris
A Unix variant possibly still available. Offered by Oracle and formerly Sun Microsystems Corporation.
SSL (Secure Sockets Layer)
SSL labs
A Qualys (so you know it has to be good quality) service where you can test a web site’s SSL certificate.
TPM, a Microsoft security feature required by Windows 11, is a dedicated chip designed to provide “hardware-level security services for your device,” keeping your private information and credentials safe from unauthorized users.
TSF (Tech Support File)
Palo Alto Networks-specific lingo for a dump file they require for a firewall support case.
Ubuntu Linux
A commercialized implementation of Debian Linux from Canonical.
UC (Unified Communications)
Cisco likes this term.
udev rules
udev rules in Linux are used to manage device nodes in the /dev directory. Those nodes are created and removed every time a user connects or disconnects a device.
UI5
SAP’s UI for HTML 5.
Ultrix
A Unix variant which ran on DEC workstations.
Underlay
SD Wan terminology for the underlying network. As opposed to overlay.
Unit testing
UPS (Uninterruptible Power Supply)
URL
Use case
UTC (Universal Time Coordinated)
What used to be called GMT.
UTF-8
Common representation of common language characters. I think of it as a successor to ASCII.
Validated
In FDA parlance, an adjective used to describe a system which follows FDA controls.
VDI
A virtual desktop offered by Citrix.
VLAN
VM (Virtual Machine)
VMWare
Will Broadcom destroy this company the way they did to Bluecoat/Symantec?
VNC (Virtual Networking Computer)
VNC is a software used to remotely control a computer.
VPC (Virtual Private Cloud)
vPC (Virtual Port Channel)
A virtual port channel (vPC) allows links that are physically connected to two different Cisco FEXes to appear as a single port channel by a third device.
VPG (Virtual Port Group)
A Cisco-ism.
VPN – Virtual Private Network
VRF
A logically separated network when using MPLS.
WAF (Web Application Firewall)
Webhook
Website
Wiki
A less formal and usually more collaborative approach to documentation, the prime example being Wikipedia.
Windows PE or Win PE
A small OS for repairing or restoring Windows systems.
WWW (World Wide Web)
x86
A type of processor architecture. Found in most Windows PCs.
XHR (XMLHttpRequest)
I.e., ajax.
XML (eXtensible Markup Language)
Common file format for data exchange, but not too human-friendly.
As my devoted followers will be aware, I nearly kiled myself converting my Centos 6 VM to CentOS 8. For instance see Upgrading WordPress brings a thicket of problems. That is an experience I only want to go through every 10 years, and fortunately, CentOS was just the right platform as its support was supposed to last 10 years I started this blog in either 2011 I believe. I went to CentOS 8 in 2020.
But instead of eight more good years, I’ve learned that CentOS is basically a dead product. EOL in industry parlance. IBM killed it. The last upgrades to CentOS 8 came at the end of 2021. There is a sort of CentOS, now called CentOS Stream, but it should be basically thought of a another Fedora. Probably IBM was losing too much money with people choosing CentOS (free) over RedHat paid subscription.
But anyway, I’ve come to resent how out-of-date the packages are on CentOS and I am much more favorable to plain old Debian linux, largely due to my work on the Raspberry Pi. There the packages like python are much more uptodate. I guess the support is for five years.
The other VM I would consider for my next iteration is Amazon Linux. It has a lot of what I need already installed, so less fuss. But I think they’re only supported for three years.
Rocky Linux, the CentOS replacement
After three years I finally hard about the best CentOS replacement. Rocky Linux. I guess it’s still a bit obscure, but you can find it as an AMI on the marketplace. It has no cost and its stated aim is to be bug-for-bug compatible with Redhat! See the Wikipedia article.
References and related
There is a snarky commentary about this topic which inspired this article. I don’t have the link right now but it is enlightening. I will post it if I ever find it.
Many of us were quite enthusiastically awaiting the availability of SpaceX’s Starlink Internet service. On paper it sounded promising. the first results came in and the reality was far less impressive, but the update I got yesterday (July 2023) is that the service got better and better.
July 2023 Update
I guess they continued to add more satellites making the coverage better and better. When there is an outage it is only for a second – short enough for even real-time media to easily recover.
The original post, written when the service was newer, is below.
The details
I do not have this service but spoke with someone who does. He lives in Puerto Rico where the broadband option are limited. There’s the local cable company, then maybe some boutique services where you use microwave dishes, and this year, finally, Starlink. He had just a couple users on it. I think the net results are that it basically works, but with a big caveat. It sucks for real-time communication. And that’s precisely what he needed it for.
So you know when you’re streaming a movie, that downloads the movie in six(?) second chunks, so it’s a bit robust in the face of brief outages. But when you’re doing web conferencing an outage is very noticeable. And that’s what they experienced, time and again. Brief outages that interrupted their real-time applications. Perhaps lasting for a few seconds, but enough to spoil the broadcast.
Then one night, knowing their cable provider, Liberty, was out, they tested it again. It seemed fine at night. But during the day next day it failed in the same way – brief, disruptive outages.
Maybe some of it is due to holes in the satellite coverage and will get better as the fleet fills out. We’re not sure at this point.
And, yes, the dish was placed in a place where the app showed something like 98% visibility to the satellites in the sky.
Some interesting screenshots of what a Starlink IP looks like in Puerto Rico
ipinfo.ioipchicken.comspeedtest results
That speedtest looks quite good to me!
Results of Starlink app for this actual user in Puerto RicoRunning PING to google.com shows a single dropped packet
A few words from the actual user
Liberty Cable is not working even after a change of the cable modem. So this past two weeks his household has been exclusively using Starlink. In his own words:
“The single most important thing to consider when using Starlink is how obstructed your northern facing view of the sky is. I am attaching a screenshot from my Starlink app. The red shows the obstructed area. My placement is 2.5% obstructed but I still get an interruption every 4 minutes the app says. In reality it might be every 20 minutes for a few seconds.
“While my Liberty service has been out Starlink has been a life saver for us. A second user can still do her video calls but it will freeze during those 20-ish minute intervals for a few seconds. It’s not the end of the world for her but not totally idea.
“For me, my VPN will disconnect for those same few seconds and then will reconnect. If I’m entering a trade that can be a crucial few seconds while my vpn and software reconnects, but it’s workable.
“The Starlink app is free and available for everyone to use. I would suggest that anyone who is interested in the service to first download the app and scan the sky where they think they have the freest point of view north. They will only get purely uninterrupted service if the app registers 100% obstacle-free view. The properties that are most suited for Starlink are the ones at the top of a hill, with a field, or a roof taller than the surrounding trees, especially the trees to the north. An obstructed view like mine is perfectly suitable for streaming movies as they tend to buffer a few minutes in advance, downloading files, and surfing the web. Without a completely obstructed view of the sky, video calls, VPN connection, Remote desktop connection, and online gaming will be interrupted in a frustrating manner.
“I am also attaching a text file of my results from running a ping -t to google.com from my Starlink connection. This test ran for about 25 minutes. The request time outs are the times when the Starlink satellite was not able to connect due to my obstructions. However, also notice that during these times it only lost 1 ping and was immediately able to reconnect. Again, somewhat frustrating but it’s a usable product.
And during Hurricane Fiona?
Starlink performed like a champ during the hurricane. I assumed that coverage would be spotty during the drenching downpour but the user said no he was streaming Netflix. It was just a little more spotty than usual. Now that the island is without power as I write this, his Internet service is as good as usual and the day after the hurricane was a normal (remote) work day like any other.
Conclusion
Don’t throw away your cable modem*. In general as of this writing in June 2022, Starlink is a good solution for those working from home, but be prepared to be bumped every 20 minutes or so from your video conferencing or other real time uses. And of course it’s good for surfing the web or on-demand streaming.
I don’t cnosider this the final word however. There’s still hope. I’ll update this post if the quality of service ever improves.
*Unless you’re one of the many whose cable modem service isn’t all that great to begin with.
If you’ve ever had the misfortune to access a web site in China in your Edge browser, you may find that from that point onwards all your new tab pages display in Chinese despite of your best efforts to eradicate it.
The details
I was in that same boar until today. There are many bad leads out there on the Internet. In fact I never did find the solution on the Internet. I got it from a colleague.
You click on the three dots, go to Settings and search for reset.
Do the Reset. It is a little disruptive, as i have found. It does not delete everything, but it certainly resets some things. As soon as that’s done you will no longer have new tab pages be in Chinese.
It’s been awhile since I’ve had the opportunity to relatean IT mystery. After awhile they are repates of what’s already happened in the past, or it’s too complex to relate, or I was only peripherally involved. But today I came across a good one. It falls into the never been seen before category.
The details
A web server behind my web application firewall became unreachable. In the browser they get a message This site can’t be reached. The app owners came to me looking for input. I checked the WAF and it was fine. The virtual server was looking healthy. So I took a packet trace, something to this effect:
$ tcpdump -nni 0.0 host 192.168.2.124
14:00:45.180349 IP 192.68.1.13.42045 > 192.68.2.124.443: Flags [S], seq 1106553901, win 23360, options [mss 1460,sackOK,TS val 3715803515 ecr 0], length 0 out slot1/tmm3 lis=/Common/was90extqa.drjohn.com.app/was90extqa.drjohn.com_vs port=0.53 trunk=
14:00:45.181081 IP 192.68.2.124 > 192.68.1.13: ICMP host 192.68.2.124 unreachable - admin prohibited filter, length 64 in slot1/tmm2 lis= port=0.47 trunk=
14:00:45.181239 IP 192.68.1.13.42045 > 192.68.2.124.443: Flags [R.], seq 1106553902, ack 0, win 0, length 0 out slot1/tmm3 lis=/Common/was90extqa.drjohn.com.app/was9
0extqa.drjohn.com port=0.53 trunk=
I’ve never seen that before, ICMP host 192.68.2.124 unreachable – admin prohibited filter. But I know ICMP can be used to relay out-of-band routing information on occasion, though I do not see it often. I suspect it is a BAD THING and forces the connection to be shut down. Question is, where was it coming from?
The communication is via a firewall so I check the firewall. I see a little more traffic so I narrow the filter down:
$ tcpdump -nni 0.0 host 192.168.2.124host 443
And then I only see the initial SYN packet followed by the RST – from the same source IP! So since I didn’t see the bad ICMP packet on the firewall, but I do see it on the WAF, I preliminarily conclude the problem exists on the WAF.
Rookie mistake! Did you fall for it? So very, very often, in the heat of debugging, we invent some unit test which we’ve never done before, and we have to be satisified with the uncertainty in the testing method and hope to find a control test somehow, somewhere to validate our new unit test.
Although I very commonly do compound filters, in this case it makes no sense, as I realized a few minutes later. My port 443 filter would of course exclude logging the bad ICMP packets because ICMP does not use tcp port 443! So I took that out and re-run it. Yup. bad ICMP packet still present on the firewall, even on the interface of the firewall directly connected to the server.
So at this point I have proven to my satisfaction that this packet, which is ruining the communication, really comes frmo the server.
What the server guys say
Server support is outsourced. The vendor replies
As far as the patching activities go , there is nothing changed to the server except distro upgrading from 15.2 to 15.3. no other configs were changed. This is a regular procedure executed on almost all 15.2 servers in your environment. No other complains received so far…
So, the usual It’s not us, look somewhere else. So the app owner asks me for further guidance. I find it’s helpful to create a test that will convince the other party of the error with their service. And what is one test I would have liked to have seen but didn’t cnoduct? A packet trace on the server itself. So I write
I would suggest they (or you) do a packet trace on the server itself to prove to themselves that this server is not behaving ini an acceptable way, network-wise, if they see that same ICMP packet which I see.
The resolution
This kind of thing can often come to a stand-off, or many days can be wasted as an issue gets escalated to sufficiently competent technicians. In this case it wasn’t so bad. A few hours later the app owners write and mention that the home-grown local firewall seemed suspect to them. They dsabled it and this traffic began to work.
They are reaching out to the vendor to understand what may have happened.
Case: closed!
Conclusion
An IT mystery was resolved today – something we’ve never seen but were able to diagnose and overcome. We learned it’s sometimes a good thing to throw a wider net when seeing unexpected reset packets because maybe just maybe there is an ICMP host unreachable packet somewhere in the mix.
Most firewalls would just drop packets and you wait for a timeout. But this was a homegrown firewall running on SLES 15. So it abides by its own ways of working, I guess. So because of the RST, your connection closes quickly, not timing out as with a normal network firewall.
As always, one has to maintain an open mind as to the true source of an issue. What was working yesterday does not today. No one admits to changing anything. Finding clever ad hoc unit tests is the way forward, and don’t forget to validate the ad hoc test. We use curl a lot for these kinds of tests. A browser is a complex beast and too much of a black box.
Here I’ve combined work I’ve done previously into one single useful application: I can initiate the live streaming of our band practice on YouTube with the click of a single button on a remote control, and stop it with another click.
Equipment
Raspberry Pi 3 or 4 with Raspberry Pi OS, e.g., Raspbian Lite is just fine
Logitech webcam or USB microphone
USB extender (my setup needed this, others may not)
Universal USB-based remote control – see references for a known good one
Method 1
In this method I rapidly blink the onboard red power (PWR) LED of the RPi while streaming is active. Outside of those times it is a solid red. This is my preferred mode – it’s a very visible sign that things are working. I am very excited about this approach.
#!/bin/sh
# DrJ 8/30/2021
# https://www.jeffgeerling.com/blogs/jeff-geerling/controlling-pwr-act-leds-raspberry-pi
# put LED into GPIO mode
echo gpio | sudo tee /sys/class/leds/led1/trigger > /dev/null
# flash the bright RED PWR (power) LED quickly to signal whatever
while /bin/true; do
echo 0|sudo tee /sys/class/leds/led1/brightness > /dev/null
sleep 0.5
echo 1|sudo tee /sys/class/leds/led1/brightness > /dev/null
sleep 0.5
done
#!/bin/sh
# DrJ 8/30/2021
# https://www.jeffgeerling.com/blogs/jeff-geerling/controlling-pwr-act-leds-raspberry-pi
# put LED into GPIO mode
echo gpio | sudo tee /sys/class/leds/led1/trigger > /dev/null
# turn on the bright RED PWR (power) LED
echo 1|sudo tee /sys/class/leds/led1/brightness > /dev/null
#!/bin/bash
# DrJ 8/2021
# Control the livestream of audio to youtube
# works in conjunction with an attached keyboard
# I use bash interpreter to give me access to RegEx matching
HOME=/home/pi
log=$HOME/audiocontrol.log
program=continuousaudio.sh
##program=tst.sh # testing
PGM=$HOME/$program
# de-press ENTER button produces this:
match="1, 28, 0"
epochsOld=0
cutoff=3 # seconds
DEBUG=1
ledtime=10
#
echo "$0 starting monitoring at "$(date)
# Note the use of script -q -c to avoid line buffering of the evread output
script -q -c $HOME/evread.py /dev/null|while read line; do
[[ $DEBUG -eq 1 ]] && echo line is $line
# seconds since the epoch
epochs=$(date +%s)
elapsed=$((epochs-$epochsOld))
if [[ $elapsed -gt $cutoff ]]; then
if [[ "$line" =~ $match ]]; then
echo "#################"
echo We caught this inpupt: $line at $(date)
# see if we are already running continuousaudio or not
pgrep -f $program>/dev/null
# 0 means it's been found
if [ $? -eq 0 ]; then
# kill it
echo KILLING $program
pkill -9 -f $program; pkill -9 ffmpeg
pkill -9 -f blinkLED
echo Shine the PWR LED
$HOME/shineLED.sh
else
# start it
echo Blinking PWR LED
$HOME/blinkLED.sh &
echo STARTING $PGM
$PGM > $PGM.log.$(date +%m-%d-%y:%H:%M) 2>&1 &
fi
epochsOld=$epochs
fi
[[ $DEBUG -eq 1 ]] && echo No action taken. Continue to listen
fi
done
The crontab entry and the referenced files are the same as in Method 2.
Method 2
In method 2 I flash the built-in LED on the webcam for a few seconds before starting the audio, and again when the streaming has terminated – as visible signal that the button press registered.
#!/bin/bash
# DrJ 8/2021
# Control the livestream of audio to youtube
# works in conjunction with an attached keyboard
# I use bash interpreter to give me access to RegEx matching
HOME=/home/pi
log=$HOME/audiocontrol.log
program=continuousaudio.sh
##program=tst.sh # testing
PGM=$HOME/$program
# de-press ENTER button produces this:
match="1, 28, 0"
epochsOld=0
cutoff=3 # seconds
DEBUG=1
ledtime=10
#
echo "$0 starting monitoring at "$(date)
# Note the use of script -q -c to avoid line buffering of the evread output
script -q -c $HOME/evread.py /dev/null|while read line; do
[[ $DEBUG -eq 1 ]] && echo line is $line
# seconds since the epoch
epochs=$(date +%s)
elapsed=$((epochs-$epochsOld))
if [[ $elapsed -gt $cutoff ]]; then
if [[ "$line" =~ $match ]]; then
echo "#################"
echo We caught this inpupt: $line at $(date)
# see if we are already running continuousaudio or not
pgrep -f $program>/dev/null
# 0 means it's been found
if [ $? -eq 0 ]; then
# kill it
echo KILLING $program
pkill -9 -f $program; pkill -9 ffmpeg
sleep 1
echo turn on led for a few seconds
$HOME/videotst.sh &
sleep $ledtime
pkill -9 ffmpeg
else
# start it
echo turn on led for a few seconds
$HOME/videotst.sh &
sleep $ledtime
pkill -9 ffmpeg
sleep 1
echo STARTING $PGM
$PGM &
fi
epochsOld=$epochs
fi
[[ $DEBUG -eq 1 ]] && echo No action taken. Continue to listen
fi
done
I press the Enter button once on the remote to begin the livestream to YouTube. I press it a second time to stop.
By extension this could also control other programs as well (like the photo frame). And other keys could be mapped to other functions. Record-only, don’t livestream, anyone?
I want to do these things because it’s a little tight in the room where I want to livestream – hard to get around. So this keeps me from having to squeeze past other people to access the RPi to for instance power cycle it. In my previous treatment, I had livestreaming start up as soon as the RPi booted up, which means it would only stop when it was similarly powered off, which I found somewhat limiting.
The purpose of videotst.sh in Method 2
videotst.sh serves almost no purpose whatsoever! It can simply be commented out. It’s somewhat specific to my webcam.
You see, I wanted to get some feedback that when I pressed the ENTER button the remote control the RPi had read that and was trying to start the livestream. I thought of flashing one of the built-in LEDs on the RPi. I still need to look into that.
With the robotics team we had soldered on an external LED onto one of the GPIO pins, but that’s way too much trouble.
So what videotst.sh does for me is to engage the webcam, specifically its video component, throwing away the actual video but with the net result that the webcam’s built-in green LED illuminates for a few seconds! That lets me know, “Yeah, your button press was registered and we’re beginning to start the livestream.” You see, because when you run ffmpegwireless6.sh with this webcam, it’s all about the audio. It only uses the audio of the webcam and thus the green “in use” LED never illuminates, unfortunately, while it is livestreaming the pure audio stream. So, similarly, when you press ENTER a second time to stop the stream I illuminate the webcam’s LED for a few seconds by using videotst.sh once again.
Techniques developed for this project
evread.py does some nasty buffering of its output, meaning, although it dos read the key presses on the remote, it holds the results “close to its chest,” and then spits them out, all at once, when the buffer is full. Well, that totally defeats the purpose needed here where I want to know if there’s been a single click. After some insightful Internet searches (note that I did not use Google as a verb, a practice I carry into my personal communication) I discovered the program script, which, when armed with the arguments -q -c, allows you to unbuffer the output of a program! And, it actually works. Cool.
And I made the command decision to “eat” the input. You see the timer of 3 seconds in broadcastswitch.sh? After you’ve done any button press it throws away any further button presses for the next three seconds. I just think that’ll reduce the misfires. In fact, I might take up the practice of double-clicking the ENTER button just to be sure I actually pressed it.
I’m using the double bracket notation more in my bash scripts. It permits use of a RegEx comparison operator. =~. I love regular expressions. More the perl style, PCRE, while this uses extended regular expressions, ERE. But I suppose those are good as well.
Getting control over the power LED was a nice coup. I’m only disappointed that you cannot control its brightness. In the dark it throws off quite a bit of light. But you cannot.
The green LED does not seem nearly as bright so I chose not to play with it. What I don’t want is to have to strain to see whether the thing is livestreaming or not.
Of course getting the whole remote control thing to work at all is another great advancement.
Techniques still to be developed
I still might investigate using voice-driven commands in place of a remote. Obviously, that’s a big nut to crack. Even if I managed to turn it on, turning it off while ffmpeg has commandeered the audio channel is even harder. I wonder if ffmpeg can split the audio stream so another process can be run alongside it to listen for voice commands? Or if an upstream process in front of ffmpeg could be used for that purpose? Or simply run with two microphones (seems wasteful of material)?? Needs research.
Suppose you want to take this on the road? Internet service can be unreliable after all. It’s well known you can power the RPi 3 for many hours with a small portable battery. So how about mapping a second button on the remote to a record-only mode (using the arecord utility, for instance)?? Then you can upload the audio at a time of you convenience. That’s something I can definitely program if I find I need it.
Lingering Problems with this approach
Despite all the care I’ve taken with the continuousaudio.sh script, still, there are times when YouTube does not show that a livestream is going on. I have no idea why at this point. If I knew the cause, I’d have fixed it!
As the livestream aspect of this is actual immaterial to me, I will probably switch to a pure recording mode where I upload in a later step – perhaps all done by the remote control for pure convenience.
Since this blog post has become popular, I may keep it preserved as is and start a new one for this recording approach as some people may genuinely be interested in the livestream aspect.
A very rough estimate of the failure rate is maybe as high as 50% but probably no lower than 25%. So, not great odds if you’re relying on success.
There’s another issue which I consider more minor. The beginning of the stream always sounds like a tape played on fast forward for a few seconds. The end also cut off a few seconds early I think.
Conclusion
We have presented a novel approach to livestreaming on a Raspberry Pi 3 using a remote control for added convenience. All the techniques were home-developed at drjohnstechtalk.com. The materials don’t cost much and it really does work.
I have always been somewhat agog at the idea of limiting bandwidth on my linux servers. Users complain about slow web sites and you want to try it for yourself, slowing your connection down to meet the parameters of their slower connection. More recently I happened on librespeed, an alternative to speedtest.net, where you can run both server and client. But in order to avoid transferring too much data and monopolizing the whole line, I wanted to actually put in some bandwidth throttling. I began an exploration of available methods to achieve this and found some satisfactory approaches that are readily available on Redhat-type linuxes.
bandwidth throttling, bandwidth rate limiting, bandwidth classes – these are all synonyms for what is most commonly called traffic shaping.
What doesn’t work so well
I think it’s important to start with the walls that I hit.
Cgroup
I stumbled on cgroups first. The man page starts in a promising way
cgroup - control group based traffic control filter
Then after you research it you see that support was enabled for cgroups in linux kernels already long ago. And there is version 1 and 2. And only version 1 supports bandwidth limits. But if you’re just a mid-level linux person such as myself, it is confusing and unclear how to take advantage of cgroup. My current conclusion is that it is more a subsystem designed for use by systemctl. In fact if you’ve ever looked at a status, for instance of crond, you see a mention of a cgroup:
sudo systemctl status crond
? crond.service - Command Scheduler
Loaded: loaded (/usr/lib/systemd/system/crond.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2021-08-09 15:44:24 EDT; 5 days ago
Main PID: 1193 (crond)
Tasks: 1 (limit: 11278)
Memory: 2.1M
CGroup: /system.slice/crond.service
mq1193 /usr/sbin/crond -n
I don’t claim to know what it all means, but there it is. Some nice abilities to schedule and allocate finite resources, at a very high level.
So I get the impression that no one really uses cgroups to do traffic shaping.
apache web server to the rescue – not
Since I was mostly interested in my librespeed server and controlling its bandwidth during testing, I wondered if the apache web server has this capability built-in. Essentially, it does! There is the module mod_ratelimit. So, quest over, and let the implementation begin! Except not so fast. In fact I did enable that module. And I set it up on my librespeed server. It kind of works, but mostly, not really, and nothing like its documented design.
That’s their example section. I have no interest in such low limits and tried various values from 4000 to 12000. I only got two different actual rates from librespeed out of all those various configurations. I could either get 83 Mbps or around 162 Mbps. And that’s it. Merely having any statement whatsoever starts limiting to one of these strange values. With the statement commented out I was getting around 300 Mbps. So I got rate-limiting, but not what I was seeking and with almost no control.
So the apache config approach was a bust for me.
Trickle
There are some linux programs that are perhaps promoted too heavily? Within a minute of posting my first draft of this someone comes along and suggests trickle. Well, on CentOS yum search trickle gives no results. My other OS was SLES v 15 and I similarly got no results. So I’m not enamored with trickle.
tc – now that looks promising
Then I discovered tc – traffic control. That sounds like just the thing. I had to search around a bit on one of my OSes to find the appropriate package, but I found it. On CentOS/Redhat/Fedora the package is iproute-tc. On SLES v15 it was iproute2. On FreeBSD I haven’t figured it out yet.
But it looks unwieldy to use, frankly. Not, as they say, user-friendly.
tcconfig + tc – perfect together
Then I stumbled onto tcconfig, a python wrapper for tc that provides convenient utilities and examples. It’s available, assuming you’ve already installed python, through pip or pip3, depending on how you’ve installed python. Something like
$ sudo pip3 install tcconfig
I love the available settings for tcset – just the kinds of things I would have dreamed up on my own. I wanted to limit download speeds, and only on the web server running on port 443, and noly from a specific subnet. You can do all that! My tcset command went something like this:
More importantly – does it work? Yes, it works beautifully. I run a librespeed cli with three concurrent streams against my AWS server thusly configured and I get around 149 Mbps. Every time.
Note that things are opposite of what you first think of. When I want to restrict download speeds from a server but am imposing traffic shaping on the server (as opposed to on the client machine), from its perspective that is upload traffic! And port 443 is the source port, not the destination port!
Raspberry Pi example
I’m going to try regular librespeed tests on my home RPi which is cabled to my router to do the Internet monitoring. So I’m trying
Despite the strange delay-distro appearing in the tcshow output, the results are perfect. Here are my librespeed results, running against my own private AWS server:
Time is Sat 21 Aug 16:17:23 EDT 2021
Ping: 20 ms Jitter: 1 ms
Download rate: 100.01 Mbps
Upload rate: 9.48 Mbps
!
Problems creep in on RPi
I swear I had it all working. This blog post is the proof. Now I’ve rebooted my RPi and that tcset command above gives the result Illegal instruction. Still trying to figure that one out!
March, 2022 update. My RPi had other issues. I’ve re-imaged the micro SD card and all is good once again. I set traffic shaping policies as shown in this post.
Conclusion about tcconfig
It’s clear tcset is just giving you a nice interface to tc, but sometimes that’s all you need to not sweat the details and start getting productive.
Possible issue – missing kernel module
On one of my servers (the CentOS 8 one), I had to do a
$ sudo yum install kernel-modules-extra
$ sudo modprobe sch_netem
before I could get tcconfig to really work.
To do list
Make the tc settings permanent.
Verify tc + tcconfig work on a Raspberry Pi. (tc is definitely available for RPi.)
Conclusion
We have found a pretty nice and effective way to do traffic shaping on linux systems. The best tool is tc and the best wrapper for it is tcconfig.
References and related
Librespeed is a great speedtest.net alternative for hard-code linux types who love command line and being in full control of both ends of a speed test. I describe it here.
Power cycling one’s cable modem automatically via an attached RPi. I refer to this blog post specifically because I intend to expand that RPi to also do periodic, automated speedtesting of my home braodband connection, with traffic shaping in place if all goes well (as it seems to thus far).
Bandwidth management and “queueing discipline” in all its gory detail is explained in this post, including example raw tc commands. I haven’t digested it yet but it may represent a way for me to get my RPi working again without a re-image: http://www.fifi.org/doc/HOWTO/en-html/Adv-Routing-HOWTO-9.html
I’m imagining a scenario where yuo are in a world where a private pki reigns. In that case you want to just make sure lftp knows where to find the private root CA and possibly the intermediate CA.
A publicity-adverse colleague of mine wrote this amazing program. I wanted to publish it not so much for what it specifically does, but as well for the programming techniques it uses. I personally find i relatively hard to look up concepts when using TCL for an F5 iRule.
# RULE_INIT is executed once every time the iRule is saved or on reboot. So it is ideal for persistent data that is shared accross all sessions.
# In our case it is used to define a template with some variables that are later substituted
when RULE_INIT {
# "static" variables in iRules are global and read only. Unlike regular TCL global variables they are CMP-friendly, that means they don't break the F5 clustered multi-processing mechanism. They exist in memory once per CMP instance. Unlike regular variables that exist once per session / iRule execution. Read more about it here: https://devcentral.f5.com/s/articles/getting-started-with-irules-variables-20403
#
# One thing to be careful about is not to define the same static variable twice in multiple iRules. As they are global, the last iRule saved overwrites any previous values.
# Originally the idea was to load an iFile here. That's also the main reason to even use RULE_INIT and static variables. The reasoning was (and I don't even know if this is true), that loading the iFile into memory once would have to be more efficient than to do it every time the iRule is executed. However, it is entirely possible that F5 already optimized iFiles in a way that loads them into memory automatically at opportune times, so this might be completely unnecessary.
# Either way, as you can tell, in the end I didn't even use iFiles. The reason for that is simply visibility. iFiles can't be easily viewed from the web UI, so it would be quite inconvenient to work with.
# The template idea and the RULE_INIT event stayed, even though it doesn't really serve a purpose, except maybe visually separating the templates from the rest of the code.
#
# As for the actual content of the variable: First thing to note is the use of {} to escape the entire string. Works perfectly, even though the string itself contains braces. TCL magic.
# The rest is just the actual PAC file, with strategically placed TCL variables in the form of $name (this becomes important later)
set static::pacfiletemplate {function FindProxyForURL(url, host)
{
var globalbypass = "$globalbypass";
var localbypass = "$localbypass";
var ceglobalbypass = "$ceglobalbypass";
var zpaglobalbypass = "$zpaglobalbypass";
var zscalerbypassexception = "$zscalerbypassexception";
var bypass = globalbypass.split(";").concat(localbypass.split(";"));
var cebypass = ceglobalbypass.split(";");
var zscalerbypass = zpaglobalbypass.split(";");
var zpaexception = zscalerbypassexception.split(";");
if(isPlainHostName(host)) {
return "DIRECT";
}
for (var i = 0; i < zpaexception.length; ++i){
if (shExpMatch(host, zpaexception[i])) {
return "PROXY $clientproxy";
}
}
for (var i = 0; i < zscalerbypass.length; ++i){
if (shExpMatch(host, zscalerbypass[i])) {
return "DIRECT";
}
}
for (var i = 0; i < bypass.length; ++i){
if (shExpMatch(host, bypass[i])) {
return "DIRECT";
}
}
for (var i = 0; i < cebypass.length; ++i) {
if (shExpMatch(host, cebypass[i])) {
return "PROXY $ceproxy";
}
}
return "PROXY $clientproxy";
}
}
set static::forwardingpactemplate {function FindProxyForURL(url, host)
{
var forwardinglist = "$forwardinglist";
var forwarding = forwardinglist.split(";");
for (var i = 0; i < forwarding.length; ++i){
if (shExpMatch(host, forwarding[i])) {
return "PROXY $clientproxy";
}
}
return "DIRECT";
}
}
}
# Now for the actual code (executed every time a user accesses the vserver)
when HTTP_REQUEST {
# The request URI can of course be used to differentiate between multiple PAC files or to restrict access.
# So can basically any other request attribute. Client IP, host, etc.
if {[HTTP::uri] eq "/proxy.pac"} {
# Here we set variables with the exact same name as used in the template above.
# In our case the values come from a data group, but of course they could also be defined
# directly in this iRule. Using data groups makes the code a bit more compact and it
# limits the amount of times anyone needs to edit the iRule (potentially making a mistake)
# for simple changes like adding a host to the bypass list
# These variables are all set unconditionally. Of course it is possible to set them based
# on for example client IP (to give different bypass lists or proxy entries to different groups of users)
set globalbypass [ class lookup globalbypass ProxyBypassLists ]
set localbypass [ class lookup localbypassEU ProxyBypassLists ]
set ceglobalbypass [ class lookup ceglobalbypass ProxyBypassLists ]
set zpaglobalbypass [ class lookup zpaglobalbypass ProxyBypassLists ]
set zscalerbypassexception [ class lookup zscalerbypassexception ProxyBypassLists ]
set ceproxy [ class lookup ceproxyEU ProxyHosts ]
# Here's a bit of conditionals, setting the proxy variable based on which virtual server the
# iRule is currently executed from (makes sense only if the same iRule is attached to multiple
# vservers of course)
if {[virtual name] eq "/Common/proxy_pac_http_90_vserver"} {
set clientproxy [ class lookup formauthproxyEU ProxyHosts ]
} elseif {[virtual name] eq "/Common/testproxy_pac_http_81_vserver"} {
set clientproxy [ class lookup testproxyEU ProxyHosts]
} elseif {[virtual name] eq "/Common/proxy_pac_http_O365_vserver"} {
set clientproxy [ class lookup ceproxyEU ProxyHosts]
} else {
set clientproxy [ class lookup clientproxyEU ProxyHosts ]
}
# Now this is the actual magic. As noted above we have now set TCL variables named for example
# $globalbypass and our template includes the string "$globalbypass"
# What we want to do next is substitute the variable name in the template with the variable values
# from the code.
# "subst" does exactly that. It performs one level of TCL execution. Think of "eval" in basically
# any language. It takes a string and executes it as code.
# Except for "subst" there are two in this context very useful parameters: -nocommands and -nobackslashes.
# Those prevent it from executing commands (like if there was a ping or rm or ssh or find or anything
# in the string being subst'd it wouldn't actually try to execute those commands) and from normalizing
# backslashes (we don't have any in our PAC file, but if we did, it would still work).
# So what is left that it DOES do? Substituting variables! Exactly what we want and nothing else.
# Now since the static variable is read only, we can't do this substitution on the template itself.
# And if we could it wouldn't be a good idea, because it is shared accross all sessions. So assuming
# there are multiple versions of the PAC file with different proxies or bypass lists, we would
# constantly overwrite them with each other.
# The solution is simply to save the output of the subst in a new local variable that exists in
# session context only.
# So from a memory point of view the static/global template doesn't really gain us anything.
# In the end we have the template in memory once per CMP and then a substituted copy of the template
# once per session. So as noted earlier, could've probably just removed the entire RULE_INIT block,
# set the template in session context (HTTP_REQUEST event) and get the same result,
# maybe even slightly more efficient.
set pacfile [subst -nocommands -nobackslashes $static::pacfiletemplate]
# All that's left to do is actually respond to the client. Simple stuff.
HTTP::respond 200 content $pacfile "Content-Type" "application/x-ns-proxy-autoconfig" "Cache-Control" "private,no-cache,no-store,max-age=0"
# In this example we have two different PAC files with different templates on different URLs
# Other iRules we use have more differentiation based on client IP. In theory we could have one big iRule
# with all the PAC files in the world and it would still scale very well (just a few more if/else or switch cases)
} elseif { [HTTP::uri] eq "/forwarding.pac" } {
set clientproxy [ class lookup clientproxyEU ProxyHosts]
set forwardinglist [ class lookup forwardinglist ProxyBypassLists ]
set forwardingpac [subst -nocommands -nobackslashes $static::forwardingpactemplate]
HTTP::respond 200 content $forwardingpac "Content-Type" "application/x-ns-proxy-autoconfig" "Cache-Control" "private,no-cache,no-store,max-age=0"
} else {
# If someone tries to access a different path, give them a 404 and the right URL
HTTP::respond 404 content "Please try http://webproxy.drjohns.com/proxy.pac" "Content-Type" "text/plain" "Cache-Control" "private,no-cache,no-store,max-age=0"
}
}
