Click here to view PowerPoint slides
Report on Combined Syndicates 2 and 5 Discussions
Peter Clark, Rashaad Jones, Mark Nixon, Martin Taylor
This report contains some ideas shared during our syndicate sessions. We tried to use these few hours of brainstorming to come up with guidelines about how to improve future visualisation techniques to address the challenges of analytic support to defensive information warfare (D/IW) operations. Our discussions focussed on a number of topics especially pertinent to visualisation in this domain:
2. COMMON OPERATIONAL PICTURE (COP) FOR D/IW
There are several different stakeholders (presentees, human perceivers) in networked information systems with different interests in visualising the conduct of D/IW:
The service provider, whose primary interest is in the costs, profits, schedule and performance achieved in operating networked information server hardware and software, will want an effective visualisation of the current and cumulative financial state of his/her enterprise.
The system subscriber, on the other hand is the ultimate end user of the service provided and will want an effective visualisation of the security and integrity of his/her information products (files, programs) stored in and maintained by the networked information service to which he/she subscribes.
Finally, the system administrator, whose primary purpose is continuity of operations, will want visualisations of current capacities, transactions (and their rates) and levels of resource utilisation. As the front-line engager in D/IW the system administrator has the greatest interest in his/her systems effectiveness at
2.1. Good COP, bad COP for D/IW
A Common Operating Picture (COP) for D/IW depends on the presentees (human perceivers) purpose, role and cognitive style. Different COPs for D/IW are distinguished by different:
Figure 2.1. Indexing Visualised Information by Labels, Descriptors and LocationsFor IW visualisation, we should also expect that differences in the granularity and dimensionality of information include continuous (analogue, scalar, vector) and discrete (categorial, symbolic, linguistic, tabular) as shown below in figure 2.2 for the lately mentioned air-traffic control challenge:
Figure 2.2. Indexing Visualised Information by Linguistic and Tabular Presentation
Finally, we note that different modes of information access (data, timing, etc.) may be imposed by the information (e.g., real-time, signal-based), chosen by the presentee explicitly (e.g., by variation in perspective, attribute, etc.) and may be static or, again, streamed (audio and/or video).
2.2. Fidelity Follows Purpose
In D/IW as in other problem domains, a key to effective visualisation will be the degree to which the information is presented with appropriate fidelity. Unless summaries or details are right-sized to the role of the presentee, effective (correct and timely) decision making on his/her part will not be enabled.
The objectness of a visualisation also serves its fidelity to the presentees purpose. This notion speaks to the interactiveness of the visualisation and the degree to which the presentee is able to manipulate it so as to draw out important problem domain features with relative ease. Objectness depends on presentee control and affords the constructed presentation its intuitiveness.
3. COLLABORATION AND TEAMWORK IN D/IW
Collaboration and teamwork for distributed D/IW operations are vital to their effectiveness through implementation of a divide-and-conquer strategy. Similarly, through collaboration and teamwork, countermeasures to IW attacks will be disseminated to vulnerable sites sooner and more readily.
Collaboration, however, raises challenges of its own in the D/IW domain if different team members are to divide their efforts effectively so as to maximise continuity of operations. Role-distinguished displays will be required in order to discourage interference of team members with each others work. At the same time, teamwork will also require that distributed operators visualise each others behaviour.
In order for the existence and progress of an attack to be identified most readily, a phased detection involving many pairs of eyes is called for while, at the same time, the security of the system, communications and individual team-member information stores are maintained.
4. NATURE OF D/IW AND ITS INTELLIGENCE INFORMATION
In this section, we discuss some of the more salient characteristics if IW and the intelligence information by means of which it is countered defensively.
4.1. The Pace and Tempo of IW
IW operations are significantly different from those of conventional warfare. Adversaries in IW are not restricted or paced by the rate at which physical objects of conventional warfare (aircraft, missiles, armoured vehicles, etc.) can be developed and manufactured they have only to be perceptive of system vulnerabilities and imaginative in ways of exploiting them. IW is rather like an accelerated arms race. The key to effectiveness in D/IW is in implementing and disseminating countermeasures to his attacks. This taxes the adversarys main resource, viz., his creativity with respect to new, unprecedented attacks. Here teamwork and collaboration are vital to winning by attrition at a numbers game with the IW adversary.
NB Countermeasures could well play into an adversarys hand by establishing new vulnerabilities.
Below we discuss the perceptual modes involved in D/IW. Underlying all of these is the desirability of understanding the plans and strategies of an IW adversary. To the extent that the IW adversarys intentions can be anticipated and his plans and strategies understood, there is the opportunity to engage in look ahead evaluation the potential of new vulnerabilities that can arise from any given D/IW countermeasure.
NB The IW adversarys intentions and strategies can reveal new vulnerabilities due to countermeasures.
The desirability of understanding the IW adversarys intentions, plans and strategies for mounting attacks calls for problem-solving capabilities in the domains of planning, analogical reasoning and decision/risk analysis in Artificial Intelligence (AI).
The common thread in all such AI problem-solving capabilities is the ability to recognize, declare and apply a rich variety of hypotheses regarding potential attacks against identified vulnerabilities. Indeed, such hypotheses may expose the perpetrator of an offensive IW (O/IW) attack to the risk of being caught in the act by contributing, ahead of time, to the formation and anticipation of a possible plan of attack by the D/IW defender before that plan is actually attempted by the O/IW attacker. In this way, higher-level problem-solving techniques contribute to more robust D/IW.
However, it must be emphasised that higher-level AI problem-solving techniques alone, absent effective visualisations for
4.2. The Identification of System Risk and Dimensions of Essential Information in D/IW
Identification of system risk is the raison dêtre for any of the data acquired in D/IW operations. For, it is only by monitoring the status of vulnerable system assets that D/IW operations can anticipate potential O/IW attacks and establish alerts for their occurrence. All other concerns, such as identifying potential attackers, are secondary.
There are, however, only a limited number of parameters in distributed information systems whereby an O/IW adversary can mount an attack and through which system assets can be exposed to risk. These parameters define dimensions of IW information used for determining anomalous vs. baseline behaviour of network message traffic which is, after all, the means of perpetrating O/IW attacks:
5. TAXONOMY OF D/IW PERCEPTUAL MODES, DATA TYPES AND DISPLAYS
In this section, we apply the concepts of NATO RTO Technical Report 30, Visualisation of Massive Military Datasets: Human Factors, Applications, and Technologies, (hereinafter, the HAT Report) to the challenges of visualisation in D/IW. Our aim is to draw out the fundamental perceptual modes required of D/IW, the HAT taxonomic classification of applicable data types for D/IW and the taxonomic classification of effective D/IW displays (visualisations).
We begin with the HAT Report notion of the dataspace inside the computer supporting D/IW visualisation. This dataspace, shown in figure 5.1, represents some aspect of the world the human seeks to understand and influence.
Figure 5.1. The IST-05 (VisTG) Reference Model
Display engines inside the computer enable human visualisation of the relevant aspect of the world by accessing and presenting dataspace content. In terms of the IST-05 (VisTG) Reference Model, Figure 5.1 exhibits a reciprocal relationship between the humans understanding and the dataspace in the computer on the one hand, and the humans visualisation and the engines in the computer that operate on the dataspace on the other.
Figure 5.2. The Emergence of Understanding from Effective Visualisation
As shown in Figure 5.2, the effectiveness of visualisation systems depends on how well they accommodate human cognitive and task performance limitations and on how well they engage and enable human cognitive, sensory and motor capacities.
5.1 Modes of Perception in D/IW
Human purposes in using D/IW visualisations are categorised by four modes of perception distinguished in the HAT Report:
These four modes of perception in D/IW are not of a piece: Alerting and exploring are the primary human purposes in these quarters. For, it is by exploring the vulnerabilities of the system that an O/IW adversary might seek to exploit that alerts may be established to signal critical events requiring immediate response. Interestingly, exploring vulnerabilities and hypotheses regarding an O/IW adversarys potential lines of attack should consume much of the D/IW operators planning efforts since this leads to the explicit declaration of events that can be detected by alerts.
5.2. The HAT Reports Six-Dimensional Taxonomy of Data Types in D/IW
The HAT Report prescribes six dimensions along which data types can be distinguished for any given visualisation challenge in support of the intelligence analyst:
5.3 The HAT Reports Four-Dimensional Taxonomy of Data Displays in D/IW
The HAT Report prescribes six dimensions along which displays (presentations) can be distinguished for any given visualisation challenge in support of the intelligence analyst:
5.3. Summary of Key HAT Report Findings
The perceptual modes in D/IW operations are alerting, exploring and searching. Alerting is the primary mode insofar as it reveals known (components of) earlier attacks and anomalous behaviour relative to standards on message protocols. It is technologically easy to prime for many such alerts NB a real attack may be a subtle one among many simultaneously alerted. Exploring is the key to averting future attacks. This involves understanding vulnerabilities and risks afforded by external accessibility and determining normal network user behaviour. Exploring supports priming for alerts and for planning alternative detection, baiting schemes. Finally, searching supports determination of the meaning, source and malevolence of a detected attack in progress. Monitoring/Controlling enter only indirectly into the surreptitious tracking of intruders and their resource accesses. The remedy here is a change of resource accesses and performance so as frustrate the attacker.
The taxonomic classification of D/IW data types is as follows: Data acquisition is streamed in real-time. Data sources are multi-source, from redundant router tables, etc. Data choice is chosen through selection of messaging attributes. Data identification is typically labelled except for time. Data values except for time are categorical. Data inter-relations exhibit strong constructional inter-relationships between message components (packets, messages, sessions).
The taxonomic classification of D/IW displays is as follows: Display timing is dynamic in support of real-time alerting, searching or monitoring/controlling but static in exploring potential future attacks and current vulnerabilities. Data selection for display is user-directed to account for the ways in which attacks can exploit many dimensions of message traffic. Data placement is located in 2D or 3D displays, labelled in tables. Data values are categorical in displays
6. EVALUATION AND ASSESSMENT ISSUES FOR D/IW VISUALISATION
Although much work in the design of effective visualisations for D/IW has already been accomplished, we believe that there is much room for exploring the effectiveness of new or existing D/IW visualisations and prospects for new and different ones.
Effectiveness estimates for new or existing D/IW visualisations require the definition of many scenarios for performance measures against known or standard attacks. For timing-sensitive scenarios, streamed IW data require time-to-detection tests. In any case, the variety and number of IW threat scenarios required for valid performance measures should be rich.
Correctness criteria for performance measures can be established for cases in which measures are not suitable as, e.g., when the question is simply whether an attack was detected at all. Here, risks are ranked qualitatively to some extent for detection purposes. Scoring weights on different risks are nonetheless required given their different costs to continuity of system operations.
For purposes of exploring new visualisations, we recommend the development of software-less demonstrations literally, PowerPoint-slide mock-ups of D/IW visualisations distinguished by purpose, role and cognitive style. Such software-less demos serve the purpose of rapid requirements capture for D/IW visualisation by encouraging broad analysis of the insights they afford. Throughout, a primary goal of the demos would be to characterise data types and displays for D/IW according to the HAT taxonomies.
7. FUTURE RESEARCH TOPICS
We mention but a few of many fruitful areas of further research that should be undertaken in D/IW. We note that DARPA has recently published a new Broad Agency Announcement (BAA) for research on effective visualisations in Information Assurance (a nearly identical field) as an indication of interest in this quarter of the NATO community.
Exploration vs. Search in D/IW visualisations This topic seeks to support the prevention of O/IW attacks by complementing visualisation with hypothesis formation and elaboration regarding system vulnerabilities and adversary capabilities for exploiting them. Here the combination of visualisation with hypothesis formation is the key to anticipating unprecedented attacks
Countermeasures to O/IW We mentioned monitoring/controlling as an activity useful in tracking and identifying perpetrators. Ideally, this would be but a first step in taking the fight to the IW adversary by, in turn, identifying his/her system vulnerabilities and exploiting them as a follow-on to the tracking and identification function afforded by the monitoring/controlling mode of D/IW perception.
Revisiting the theoretical foundations of distributed system security This topic follows from the observation that system vulnerabilities are not all of a general nature and that some systems are subject to attack by virtue of the user base that they support, the information that these users store in the system and its sensitivity. Here the locus of research addresses unauthorized access attacks (i.e., attacks other than denials-of-service). It involves not so much the system as it does the users (problem solvers). Vulnerability in this case is problem-centric vs. system-centric authorization of access. The promise is that, by assuming a problem-centric stance, techniques may be evolved that may ease the general difficulty of enforcing access control by forcing the adversary to follow problem solver (team) from system to system (2nd order user content security). The problem-solving users engage in a spread spectrum use of different systems as infrastructure for problem solving in ways that the adversary cannot readily follow.