Complexity Science in Cyber Security
[*]Computers and the Internet have become indispensable for homes and organisations alike. The dependence on them increases by the day, be it for household users, in mission basic space control, strength grid management, medical applications or for corporate finance systems. But also in similar are the challenges related to the continued and reliable delivery of service which is becoming a bigger concern for organisations. Cyber security is at the spotlight of all threats that the organizations confront, with a majority rating it higher than the threat of terrorism or a natural disaster.
[*]in spite of of all the focus Cyber security has had, it has been a challenging journey so far. The global use on IT Security is expected to hit $120 Billion by 2017 , and that is one area where the IT budget for most companies either stayed flat or slightly increased already in the recent financial crises . But that has not significantly reduced the number of vulnerabilities in software or attacks by criminal groups.
[*]The US Government has been preparing for a “Cyber Pearl Harbour”  style all-out attack that might paralyze basic sets, and already cause physical destruction of character and lives. It is expected to be orchestrated from the criminal underbelly of countries like China, Russia or North Korea.
[*]The economic impact of Cyber crime is $100B annual in the United states alone .
[*]There is a need to fundamentally rethink our approach to securing our IT systems. Our approach to security is siloed and focuses on point solutions so far for specific threats like anti viruses, spam filters, intrusion detections and firewalls . But we are at a stage where Cyber systems are much more than just tin-and-wire and software. They include systemic issues with a social, economic and political part. The interconnectedness of systems, intertwined with a people component makes IT systems un-isolable from the human component. complicate Cyber systems today almost have a life of their own; Cyber systems are complicate adaptive systems that we have tried to understand and tackle using more traditional theories.
[*]2. complicate Systems – an Introduction
[*]Before getting into the motivations of treating a Cyber system as a complicate system, here is a fleeting of what a complicate system is. observe that the term “system” could be any combination of people, course of action or technology that fulfils a certain purpose. The wrist watch you are wearing, the sub-oceanic reefs, or the economy of a country – are all examples of a “system”.
[*]In very simple terms, a complicate system is any system in which the parts of the system and their interactions together represent a specific behaviour, such that an examination of all its component parts cannot explain the behaviour. In such systems the cause and effect can not necessarily be related and the relationships are non-linear – a small change could have a disproportionate impact. In other words, as Aristotle said “the whole is greater than the sum of its parts”. One of the most popular examples used in this context is of an urban traffic system and emergence of traffic jams; examination of individual cars and car drivers cannot help explain the patterns and emergence of traffic jams.
[*]While a complicate Adaptive system (CAS) also has characteristics of self-learning, emergence and evolution among the participants of the complicate system. The participants or agents in a CAS show heterogeneous behaviour. Their behaviour and interactions with other agents continuously evolving. The meaningful characteristics for a system to be characterised as complicate Adaptive are:
- The behaviour or output cannot be expected simply by analysing the parts and inputs of the system
- The behaviour of the system is emergent and changes with time. The same input and environmental conditions do not always guarantee the same output.
- The participants or agents of a system (human agents in this case) are self-learning and change their behaviour based on the outcome of the past experience
[*]complicate processes are often confused with “complicated” processes. A complicate course of action is something that has an unpredictable output, however simple the steps might seem. A complicated course of action is something with lots of complicate steps and difficult to unprotected to pre-conditions but with a predictable outcome. An often used example is: making tea is complicate (at the minimum for me… I can never get a cup that tastes the same as the past one), building a car is Complicated. David Snowden’s Cynefin framework gives a more formal description of the terms .
[*]Complexity as a field of study isn’t new, its roots could be traced back to the work on Metaphysics by Aristotle . Complexity theory is largely inspired by biological systems and has been used in social science, epidemiology and natural science study for some time now. It has been used in the study of economic systems and free markets alike and gaining acceptance for financial risk examination in addition (Refer my paper on Complexity in Financial risk examination here ). It is not something that has been very popular in the Cyber security so far, but there is growing acceptance of complexity thinking in applied sciences and computing.
[*]3. Motivation for using Complexity in Cyber Security
[*]IT systems today are all designed and built by us (as in the human community of IT workers in an organisation plus suppliers) and we collectively have all the knowledge there is to have regarding these systems. Why then do we see new attacks on IT systems every day that we had never expected, attacking vulnerabilities that we never knew existed? One of the reasons is the fact that any IT system is designed by thousands of individuals across the whole technology stack from the business application down to the inner network elements and hardware it sits on. That introduces a strong human component in the design of Cyber systems and opportunities become ubiquitous for the introduction of flaws that could become vulnerabilities .
[*]Most organisations have multiple layers of defence for their basic systems (layers of firewalls, IDS, hardened O/S, strong authentication etc), but attacks nevertheless happen. More often than not, computer break-ins are a collision of circumstances instead of a standalone vulnerability being exploited for a cyber-attack to succeed. In other words, it’s the “whole” of the circumstances and actions of the attackers that cause the damage.
[*]3.1 Reductionism vs Holisim approach
[*]Reductionism and Holism are two contradictory philosophical approaches for the examination and design of any object or system. The Reductionists argue that any system can be reduced to its parts and analysed by “reducing” it to the component elements; while the Holists argue that the whole is greater than the sum so a system cannot be analysed merely by understanding its parts .
[*]Reductionists argue that all systems and machines can be understood by looking at its component parts. Most of the modern sciences and examination methods are based on the reductionist approach, and to be fair they have served us quite well so far. By understanding what each part does you really can analyse what a wrist watch would do, by designing each part separately you really can make a car behave the way you want to, or by analysing the position of the celestial objects we can precisely predict the next Solar eclipse. Reductionism has a strong focus on causality – there is a cause to an affect.
[*]But that is the extent to which the reductionist view point can help explain the behaviour of a system. When it comes to emergent systems like the human behaviour, Socio-economic systems, Biological systems or Socio-cyber systems, the reductionist approach has its limitations. Simple examples like the human body, the response of a mob to a political stimulus, the reaction of the financial market to the news of a merger, or already a traffic jam – cannot be expected already when studied in detail the behaviour of the component members of all these ‘systems’.
[*]We have traditionally looked at Cyber security with a Reductionist lens with specific point solutions for individual problems and tried to anticipate the attacks a cyber-criminal might do against known vulnerabilities. It’s time we start looking at Cyber security with an alternate Holism approach in addition.
[*]3.2 Computer Break-ins are like pathogen infections
[*]Computer break-ins are more like viral or bacterial infections than a home or car break-in . A burglar breaking into a house can’t really use that as a set afloat pad to break into the neighbours. Neither can the vulnerability in one lock system for a car be exploited for a million others across the globe simultaneously. They are more akin to microbial infections to the human body, they can propagate the infection as humans do; they are likely to impact large portions of the population of a species as long as they are “connected” to each other and in case of harsh infections the systems are generally ‘secluded’; as are people put in ‘quarantine’ to reduce further spread . already the lexicon of Cyber systems uses biological metaphors – Virus, Worms, infections etc. It has many parallels in epidemiology, but the design principles often employed in Cyber systems are not aligned to the natural selection principles. Cyber systems rely a lot on uniformity of processes and technology elements as against varied of genes in organisms of a species that make the species more resilient to epidemic attacks .
[*]The Flu pandemic of 1918 killed ~50M people, more than the Great War itself. Almost all of humanity was infected, but why did it impact the 20-40yr olds more than others? Perhaps a difference in the body structure, causing different reaction to an attack?
[*]Complexity theory has attained great traction and proven quite useful in epidemiology, understanding the patterns of spread of infections and ways of controlling them. Researchers are now turning towards using their learnings from natural sciences to Cyber systems.
[*]4. Approach to Mitigating security threats
[*]Traditionally there have been two different and complimentary approaches to mitigate security threats to Cyber systems that are in use today in most functional systems :
[*]4.1 Formal validation and testing
[*]This approach chiefly relies on the testing team of any IT system to discover any faults in the system that could expose a vulnerability and can be exploited by attackers. This could be functional testing to validate the system gives the correct answer as it is expected, penetration testing to validate its resilience to specific attacks, and availability/ resilience testing. The scope of this testing is generally the system itself, not the frontline defences that are deployed around it.
[*]This is a useful approach for fairly simple self-contained systems where the possible user journeys are fairly straightforward. For most other interconnected systems, formal validation alone is not sufficient as it’s never possible to ‘test it all’.
[*]Test automation is a popular approach to reduce the human dependency of the validation processes, but as Turing’s Halting problem of Undecideability[*] proves – it’s impossible to build a machine that tests another one in all situations. Testing is only anecdotal evidence that the system works in the scenarios it has been tested for, and automation helps get that anecdotal evidence quicker.
[*]4.2 Encapsulation and boundaries of defence
[*]For systems that cannot be fully validated by formal testing processes, we deploy additional layers of defences in the form of Firewalls or network segregation or encapsulate them into virtual machines with limited visibility of the rest of the network etc. Other shared techniques of additional defence mechanism are Intrusion Prevention systems, Anti-virus etc.
[*]This approach is ubiquitous in most organisations as a defence from the unknown attacks as it’s virtually impossible to formally ensure that a piece of software is free from any vulnerability and will keep so.
[*]Approaches using Complexity sciences could prove quite useful complementary to the more traditional ways. The versatility of computer systems make them unpredictable, or capable of emergent behaviour that cannot be expected without “running it” . Also running it in isolation in a test ecosystem is not the same as running a system in the real ecosystem that it is supposed to be in, as it’s the collision of multiple events that causes the apparent emergent behaviour (recalling holism!).
[*]4.3 varied over Uniformity
[*]Robustness to disturbances is a meaningful emergent behaviour in biological systems. Imagine a species with all organisms in it having the exact same genetic structure, same body configuration, similar antibodies and immune system – the sudden increase of a viral infection would have wiped out complete community. But that does not happen because we are all formed differently and all of us have different resistance to infections.
[*]Similarly some mission basic Cyber systems especially in the Aerospace and Medical industry implement “varied implementations” of the same functionality and centralised ‘voting’ function decides the response to the requester if the results from the different implementations do not match.
[*]It’s fairly shared to have redundant copies of mission basic systems in organisations, but they are homogenous implementations instead of different – making them equally prone to all the faults and vulnerabilities as the dominant ones. If the implementation of the redundant systems is made different from the dominant – a different O/S, different application container or database versions – the two variants would have different level of resilience to certain attacks. already a change in the ordern of memory stack access could vary the response to a buffer overflow attack on the variants  – highlighting the central ‘voting’ system that there is something wrong somewhere. As long as the input data and the business function of the implementation are the same, any deviations in the response of the implementations is a sign of possible attack. If a true service-based architecture is implemented, every ‘service’ could have multiple (but a small number of) heterogeneous implementations and the overall business function could randomly select which implementation of a service it uses for every new user request. A fairly large number of different execution paths could be achieved using this approach, increasing the resilience of the system .
[*]Multi variant Execution Environments (MVEE) have been developed, where applications with slight difference in implementation are executed in lockstep and their response to a request are observed . These have proven quite useful in intrusion detection trying to change the behaviour of the code, or already identifying existing flaws where the variants respond differently to a request.
[*]On similar lines, using the N-version programming concept ; an N-version antivirus was developed at the University of Michigan that had heterogeneous implementations looking at any new files for corresponding virus signatures. The consequence was a more resilient anti-virus system, less inclined to attacks on itself and 35% better detection coverage across the estate .
[*]4.4 Agent Based Modelling (ABM)
[*]One of the meaningful areas of study in Complexity science is Agent Based Modelling, a simulation modelling technique.
[*]Agent Based Modelling is a simulation modelling technique used to understand and analyse the behaviour of complicate systems, specifically complicate adaptive systems. The individuals or groups interacting with each other in the complicate system are represented by artificial ‘agents’ and act by predefined set of rules. The Agents could evolve their behaviour and adapt as per the circumstances. Contrary to Deductive reasoning[†] that has been most popularly used to explain the behaviour of social and economic systems, Simulation does not try to generalise the system and agents’ behaviour.
[*]ABMs have been quite popular to study things like crowd management behaviour in case of a fire evacuation, spread of epidemics, to explain market behaviour and recently financial risk examination. It is a bottom-up modelling technique wherein the behaviour of each agent is programmed separately, and can be different from all other agents. The evolutionary and self-learning behaviour of agents could be implemented using various techniques, Genetic Algorithm implementation being one of the popular ones .
[*]Cyber systems are interconnections between software modules, wiring of logical circuits, microchips, the Internet and a number of users (system users or end users). These interactions and actors can be implemented in a simulation form in order to do what-if examination, predict the impact of changing parameters and interactions between the actors of the form. Simulation models have been used for analysing the performance characteristics based on application characteristics and user behaviour for a long time now – some of the popular Capacity & performance management tools use the technique. Similar techniques can be applied to analyse the response of Cyber systems to threats, designing a fault-tolerant architecture and analysing the extent of emergent robustness due to varied of implementation.
[*]One of the meaningful areas of focus in Agent Based modelling is the “self-learning” course of action of agents. In the real world, the behaviour of an attacker would evolve with experience. This aspect of an agent’s behaviour is implemented by a learning course of action for agents, Genetic Algorithm’s being one of the most popular technique for that. Genetic Algorithms have been used for designing automobile and aeronautics engineering, optimising the performance of Formula one cars  and simulating the investor learning behaviour in simulated stock markets (implemented using Agent Based models).
[*]An interesting visualisation of Genetic Algorithm – or a self-learning course of action in action – is the demo of a simple 2D car design course of action that starts from scratch with a set of simple rules and end up with a workable car from a blob of different parts: http://rednuht.org/genetic_cars_2/
[*]The self-learning course of action of agents is based on “Mutations” and “Crossovers” – two basic operators in Genetic Algorithm implementation. They emulate the DNA crossover and mutations in biological evolution of life forms. by crossovers and mutations, agents learn from their own experiences and mistakes. These could be used to simulate the learning behaviour of possible attackers, without the need to manually imagine all the use situations and user journeys that an attacker might try to break a Cyber system with.
[*]Complexity in Cyber systems, especially the use of Agent Based modelling to estimate the emergent behaviour of systems is a comparatively new field of study with very little research done on it in addition. There is nevertheless some way to go before using Agent Based Modelling becomes a commercial proposition for organisations. But given the focus on Cyber security and inadequacies in our current stance, Complexity science is certainly an method that practitioners and academia are increasing their focus on.
[*]Commercially obtainable products or sets using Complexity based techniques will however take a while till they go into the mainstream commercial organisations.
[*] J. A. Lewis and S. Baker, “The Economic Impact of Cybercrime and Cyber Espionage,” 22 July 2013. [Online]
[*] L. Kugel, “Terrorism and the Global Economy,” E-Internatonal Relations Students, 31 Aug 2011. [Online].
[*] “Cybersecurity – Facts and Figures,” International Telecommunications Union, [Online].
[*] “Interesting Facts on Cybersecurity,” Florida Tech University Online, [Online].
[*] “Global security spending to hit $86B in 2016,” 14 Sep 2012. [Online].
[*] S. Forrest, S. Hofmeyr and B. Edwards, “The complicate Science of Cyber Defense,” 24 June 2013. [Online].
[*] “Cynefin Framework (David Snowden) – Wikipedia” [Online].
[*] “Metaphysics (Aristotle) – Wikipedia” [Online].
[*] R. Armstrong, “Motivation for the Study and Simulation of Cybersecurity as a complicate System,” 2008.
[*] S. A. McLeod, Reductionism and Holism, 2008.
[*] R. C. Armstrong, J. R. Mayo and F. Siebenlist, “Complexity Science Challenges in Cybersecurity,” March 2009.
[*] B. Salamat, T. Jackson, A. Gal and M. Franz, “Orchestra: Intrusion Detection Using similar Execution and Monitoring of Program Variants in User-Space,” Proceedings of the 4th ACM European conference on Computer systems, pp. 33-46, April 2009.
[*] R. C. Armstrong and J. R. Mayo, “Leveraging Complexity in Software for Cybersecurity (recondite),” Association of Computing Machinery, pp. 978-1-60558-518-5, 2009.
[*] C. Liming and A. Avizienis, “N-VERSION PROGRAMMINC: A FAULT-TOLERANCE APPROACH TO RELlABlLlTY OF SOFTWARE OPERATlON,” Fault-Tolerant Computing, p. 113, Jun1995.
[*] J. Oberheide, E. Cooke and F. Jahanian, “CloudAV: N-Version Antivirus in the Network Cloud,” University of Michigan, Ann Arbor, MI 48109, 2008.
[*] J. H. Holland, Adaptation in natural and artificial systems: An introductory examination with applications to biology, control, and artificial intelligence, Michigan: University of Michigan Press, 1975.
[*] K. &. B. P. J. Wloch, “Optimising the performance of a formula one car using a genetic algorithm,” similar Problem Solving from character-PPSN VIII, pp. 702-711, January 2004.
[*] P. E. (. o. D. Leon, “Press Transcript,” US Department of Defense, 11 Oct 2012. [Online].
[*] Gandhi, Gagan; “Financial Risk examination using Agent Based Modelling”, [Online]: http://www.researchgate.net/publication/262731281_Financial_Risk_Analysis_using_Agent_Based_Modelling
[*][*] Alan Turing – a mathematician who came to fame for his role in breaking the Enigma machines used to encrypt communication messages during the second world war – proved that a general algorithm whether or not a program would already terminate (or keep running forever) for all program-input pairs cannot exist.
[*][†] Deductive reasoning is a ‘top-down’ reasoning approach starting with a hypothesis and data points used to substantiate the claim. Inductive reasoning however is a ‘bottom-up’ approach that starts with specific observations which are then generalised to form a general theory.