Cavendish Effect

The Cavendish Effect is when a monoculture is vulnerable to debilitating disease that could lead to extinction. The Cavendish banana is the namesake of this idea since all Cavendish banana plants are clones, and so a disease that can affect one of them can affect them all. But this effect can also affect those who simply use a lot of off-the-shelf things. For most attackers, it isn't worth the effort to develop a custom exploit that has some small chance of working against someone prior to detection. Even the most advanced targeted software attacks aim for the supply-side where a single exploit compromises a large enough mass of targets to justify the effort.

The XZ Utils backdoor relied on a lot of people using the exact same software. Not a very similar implementation, but the exact same implementation. It relied on people taking the exact same binary, compiled from the exact same source. Now that software can be highly personalized, the greater risk is not that your specific software will be attacked but that you will be using human-built fallible software that is used by a lot of people.

Example of Linux

In the 2000s a big part of Linux-based application security was truly diversity. Very few people used Linux and each distribution had its own quirks, making mass exploitation harder. At the time, Linux groupies would argue that Linux was more secure by default. Time has proven that this is really not that true. Linux users, once they were no longer a cadre of self-selected enthusiasts, now enthusiastically download and install popular flatpaks and are frequently pwned by the popular packages they're developing.

Operating systems were never the point of failure. Applications were. And applications can see everything a user can. The pattern of "established safe package turning malicious" captures them, again only because they use vendor software. The widespread nature of Linux exploits proves that Linux systems are not immune to this entire thing. And of course Spectre (security vulnerability) and Meltdown (security vulnerability) show that operating systems are not going to provide magical immunity.

A Plausible Attack

Another advantage of personal software is that it is tailored for use. Consequently, it tends to have a much lower attack surface than most other software. In the last few years, the only exploit I have experienced the effect of is Plausible Analytics's dashboard including a library that exposes a shell. Almost all of my own custom software lives behind Cloudflare Access and the remainder simply does not include upstream libraries for the most part.

However, Single Minds Are Clonal

While people have frequently discussed the fact that SOTA models can break all sorts of environments, the truth is that the software they write is also above the median developer's security quality. The downside, of course, is that model code can be fairly uniform and introduces its own risks in that respect. How clonal the software is is a matter of steering through the developer's prompts, and the differing user feature surface.

Unlike commercial SaaS, personal or in-house software has a feature set that does not need to cater to a wide audience so the attack surface is much smaller. For instance, our family EMR has far fewer features than Epic. The chance of us being hacked is much lower both because we don't support that much and also because there is really not that much of value in a bunch of X-rays of me and my daughter.

The clonality of the SOTA model's output is an unsolved risk, and it exists naturally because it is akin to standard of care and the Anna Karenina principle applies: no one will want to have a less than standard piece of software solely for the sake of diversity. Also, the fact that attacker models evolve but defender models' output can often live longer without revisiting presents a risk of one-sided improvement. One path forward may be software that is auto-updated by improved models without developers being aware of this.

Conclusion

Monocultures are bad. Upstream vendors are bad. A Cambrian explosion of software will make only attacks on the most valuable targets worth it. That means software being custom won't help banks and hospitals, but for an individual? Custom software is probably now good enough that it's better than the vendor.

Like most things that describe functionality that is specifically time-targeted, I will likely be shown wrong in some novel way that I haven't considered. Still, I think this is a worthwhile model for the now. Caveat emptor, check the date of packaging.