Cost-sensitive models can only be constructed and evaluated when cost metrics are given. The issues involved in the measurement of cost factors have been studied by the computer risk analysis and security assessment communities. The literature suggests that attempts to fully quantify all factors involved in cost modeling usually generate misleading results because not all factors can be reduced to discrete dollars (or some other common unit of measurement) and probabilities [2,4,7,8,11]. It is recommended that qualitative analysis be used to measure the relative magnitudes of cost factors. It should also be noted that cost metrics are often site-specific because each organization has its own security policies, information assets, and risk factors .
|(by results)||(by techniques)|
|1. ROOT||illegal root access is obtained.||1.1 local||by first logging in as a legitimate user on a local system, e.g., buffer overflow on local system programs such as eject.||DCost=100 RCost=40|
|1.2 remote||from a remote host, e.g., buffer overflow of some daemon running suid root.||DCost=100 RCost=60|
|2. R2L||illegal user access is obtained from outside.||2.1 single||a single event, e.g., guessing passwords.||DCost=50 RCost=20|
|2.2 multiple||multiple events, hosts, or days, e.g., the multihop attack.||DCost=50 RCost=40|
|3. DOS||Denial-of-Service of target is accomplished.||3.1 crashing||using a single malicious event (or a few packets) to crash a system, e.g., the teardrop attack.||DCost=30 RCost=10|
|3.2 consumption||using a large number of events to exhaust network bandwidth or system resources, e.g., synflood.||DCost=30 RCost=15|
|4. PROBE||information about the target is gathered.||4.1 simple||many of probes within a short period of time, e.g., fast port scan.||DCost=2 RCost=5|
|4.2 stealth||probe events are distributed sparsely across a long time windows, e.g. slow port scan.||DCost=2 RCost=7|
Our attack taxonomy is illustrated in Table 1, and categorizes intrusions that occur in the DARPA Intrusion Detection Evaluation dataset, which was collected in a simulated military environment by MIT Lincoln Lab . In this dataset, each event to be monitored is a network connection, and the resources being attacked are mainly the network services (e.g., http, smtp, etc.) and system programs on a particular host in the network. We use the taxonomy described in Table 1 to first categorize the intrusions occurring in the dataset into ROOT, DOS, R2L, and PROBE, based on their intrusion results. Then within each of these categories, the attacks are further partitioned by the techniques used to execute the intrusion. The ordering of sub-categories is of increasing complexity of the attack method. Attacks of each sub-category can be further partitioned according to the attack targets. For simplicity, the intrusion target dimension is not shown.
Criticality measures the importance, or value, of the target of an attack. This measure can be evaluated according to a resource's functional role in an organization or its relative cost of replacement, unavailability, and disclosure . Similar to Northcutt's analysis, we assign 5 points for firewalls, routers, or DNS servers, 4 points for mail or Web servers, 2 points for UNIX workstations, and 1 point for Windows or DOS workstations. Lethality measures the degree of damage that could potentially be caused by some attack. For example, a more lethal attack that helped an intruder gain root access would have a higher damage cost than if the attack gave the intruder local user access. Other damage may include the discovery of knowledge about network infrastructure or preventing the offering of some critical service. For each main attack category in Table 1, we define a relative lethality scale and use it as the base damage cost, or baseD. By assigning damage cost according to the criticality of the target, we are using the intrusion target dimension. Using these metrics, we can define the damage cost of an attack targeted at some resource as . For example, a DOS attack targeted at a firewall has DCost = 150, while the same attack targeted at a Unix workstation has DCost = 60.
In addition to criticality and lethality, we define the progress of an attack to be a measure of how successfully an attack is in achieving its goals. For example, a Denial-of-Service (DOS) attack via resource or bandwidth consumption (e.g. SYN flooding) may not incur damage cost until it has progressed to the point where the performance of the resource under attack is starting to suffer. The progress measure can be used as an estimate of the percentage of the maximum damage cost that should be accounted for. That is, the actual cost is . However, in deciding whether or not to respond to an attack, it is necessary to compare the maximum possible damage cost with the response cost. This requires that we assume a worst-case scenario in which progress=1.0.
Responses to intrusions that may be automated include the following: termination of the offending connection or session (either killing a process or resetting a network connection), implementation of a packet-filtering rule, rebooting the targeted system, or recording the session for evidence gathering purposes and further investigation [1,19]. In addition to these responses, a notification may be sent to the administrator of the offending machine via e-mail in case that machine was itself compromised. A more advanced response which has not been successfully employed to date could involve the coordination of response mechanisms in disparate locations to halt intrusive behavior closer to its source.
Additional manual responses to an intrusion may involve further investigation (perhaps to eliminate action against false positives), identification, containment, eradication, and recovery . The cost of manual response includes the labor cost of the response team, the user of the target, and any other personnel that participate in response. It also includes any downtime needed for repairing and patching the targeted system to prevent future damage.
We estimate the relative complexities of typical responses to each attack type in Table 1 in order to define the relative base response cost, or baseR. Attacks with simpler techniques (i.e., sub-categories x.1 in our taxonomy) generally have lower response costs than more complex attacks (i.e., sub-categories x.2), which require more complex mechanisms for effective response.
Some features cost more to gather than others. However, costlier features are often more informative for detecting intrusions. For example, features that examine events across a larger time window have more information available and are often used for ``correlation analysis '' in order to detect extended or coordinated attacks such as slow host or network scans . Computation of these features is costly because of their need to store and analyze larger amounts data.
Based on our experience in extracting and constructing predictive features from network audit data, we classify features into three relative levels, based on their computational costs:
We can assign relative magnitudes to these features according to their computational costs. For example, level 1 features may cost 1 or 5, level 2 features may cost 10, and level 3 features may cost 100. These estimations have been verified empirically using a prototype system for evaluating our ID models in real-time that has been built in coordination with Network Flight Recorder .