Catching anomalies on OT networks with fuzzy hashing

Our engineers published this work at the 2024 IEEE Cyber Awareness and Research Symposium, with the experiments run at a university SCADA testbed against public industrial capture datasets. We are describing it here because the constraint that shaped it, that the tooling built for corporate networks does not fit the networks that move water and power, is a live problem for every utility and manufacturer we work with.

The systems at risk are not abstract. Public-facing industrial control systems run water treatment, energy distribution and manufacturing lines, and the reported attack volume against them rises every year. Enterprise intrusion detection does not translate cleanly onto them, because it is tuned for traffic that looks nothing like theirs.

Why the difference helps¶

Enterprise networks are noisy. People browse, applications update, and normal is a wide distribution. Industrial networks are the opposite: the same controllers exchange the same function codes against the same addresses on the same intervals, day after day. That determinism is usually discussed as a liability, since it makes the network predictable to an attacker. It is also the thing that makes anomaly detection tractable, because a departure from pattern is genuinely unusual rather than a Tuesday.

So the goal was a rolling picture of what normal looks like on a given device conversation, and an alert on anything that drifts from it, computed cheaply enough to run in real time.

Why not an ordinary hash¶

A cryptographic hash is designed to have exactly the property you do not want here. Change one byte and the digest is unrecognizable. Two frames that are semantically identical but differ in a sequence number or a checksum produce completely unrelated digests, which makes the hash useless as a similarity measure. That is a feature for integrity checking and a dead end for baselining.

Similarity-preserving hashes, also called fuzzy or locality-sensitive hashes, invert that property: small input changes produce small digest changes, and the similarity between two digests can be computed cheaply. The technique is mature in malware clustering and digital forensics, which is where we took it from. We evaluated three and chose TLSH.

The problem nobody warns you about¶

The dominant protocol on these networks carries an address, a function code and a small payload. A whole frame runs from roughly forty-three to two hundred and fifty-five bytes. Similarity hashes are not designed for inputs that small, and they degrade sharply below their intended range. A per-packet pipeline returns near-zero similarity on packets that are legitimately near-identical, which means the system is not slightly wrong, it is inverted.

Before anything else could be decided, we needed a unit of input large enough for the hash function to behave. We worked through four candidates.

• Signature analysis, borrowed from antivirus: hash known-bad packets and compare against a blacklist. Failed. The frames are too short and too similar for a blacklist to discriminate.
• Single-packet inspection, borrowed from deep packet inspection: hash each packet on its own. Failed, for the size reason above, and drowned in false positives.
• Whole-capture hashing, borrowed from malware analysis: hash entire capture files against a reference set. Partially useful. It can tell you a capture came from the same network, but it cannot localize an attack inside it, so it is a forensics tool rather than a detector.
• Cluster hashing over a sliding window: group packets from one device conversation into clusters, hash each cluster, compare each new digest against the recent window. This is the one that worked.

There was also a representation question underneath the unit question. We tested four encodings of the same bytes and measured similarity separately for known-similar and known-different pairs. The string representation gave the widest gap between the two distributions, roughly ninety-one percent for similar pairs against under five percent for different ones, which is precisely what a discriminator needs. Sorting the bytes lexically scored highest of all on similar pairs and above ninety-five percent on different ones too, which is a useful reminder that a high score on the cases you want is worthless without a low score on the cases you do not.

The window is what keeps it honest¶

One digest per cluster is a measurement. The detector is what you build around it: score each new digest against the recent window of digests for the same device pair, and surface clusters whose best similarity to that window is unusually low. Because the window advances, the baseline follows normal operational drift, so a scheduled maintenance window does not permanently poison the model the way a static baseline would.

def score(cluster: list[Packet], window: Deque[str]) -> float | None:    """Score a conversation against the rolling baseline, not a packet    against a rule. Plant floors are periodic; the rule never fires."""    h = tlsh.hash(b"".join(p.payload for p in cluster))    if h is None:        return None          # under 50 bytes of entropy: no opinion offered    return min(tlsh.diff(h, prior) for prior in window) / DIFF_CEILING

Cluster size is the knob, and it is a real one. Too small and you are back to the single-packet failure. Too large and a thirty-second attack is diluted by the surrounding hour of ordinary traffic. We used a variable size tuned per device pair, sized to capture roughly one full repeating cycle of that conversation.

Evaluation ran against the CIC Modbus 2023 dataset and a public industrial capture corpus, both of which carry labelled attack windows. The cluster method consistently beat the other three approaches at localizing those windows.

The bench behind the work ran controllers from four vendors with real ladder logic rather than a single convenient platform, and the attack traffic used to test the baseline was executed against it rather than assumed from the literature: denial of service, man in the middle, packet injection and command manipulation. That matters more than it sounds. A detector tuned on one vendor’s timing characteristics is measuring that vendor, not the technique.

The honest limits¶

• The work targeted one protocol. Real industrial networks also speak DNP3, BACnet, EtherNet/IP and a long tail of vendor protocols. The clustering approach should generalize, but each protocol needs its own parser and its own cluster sizing, and that is engineering, not a footnote.
• Canonicalizing the cluster bytes before hashing, the way intrusion detection rules normalize content, is an obvious lever for cutting false positives and remains future work.
• Layering learned attribution on top of the similarity signal is the natural next step, and other groups have shown fuzzy hashing pairs well with machine learning for exactly that.

The reason we keep this work close is that it makes a general point about the boundary between corporate and operational technology. Most published detection research assumes enterprise traffic, and most vendors sell against that assumption. Industrial networks are their own regime, and the techniques that actually catch things there are different. Anyone proposing to secure a plant floor with a product designed for an office network should be asked to explain that gap before anything is signed.