7. Conclusion and Future Work

It is very important to establish the cost-effectiveness of intrusion detection because the ultimate goal of an IDS is to protect the information assets that are at risk and are most valuable to an organization. In this paper, we have examined cost factors that are relevant to intrusion detection, which include development cost, operational cost, damage cost, and response cost. We have shown that it is necessary to use an attack taxonomy along with organization-specific security policies and priorities to measure these cost factors. We studied the trade-off relationships among these factors and defined consequential cost to be the cost associated with the predictions of an IDS. The total expected cost of an IDS is the sum of the operational and consequential costs. The cost-benefit of an IDS is manifested in its abilities to reduce this total expected cost. We presented a multiple model machine learning approach for reducing operational cost and a post-detection decision module for reducing consequential cost. Empirical evaluation using the DARPA Intrusion Evaluation dataset shows that our approaches are indeed effective.

As pointed out by Dorothy Denning, cost analysis (and risk assessment in general) is not an exact science because precise measurement of relevant factors is often impossible [8]. Cost-benefit analysis and modeling, however informal or incomplete, is often very helpful for an organization to determine appropriate protection mechanisms. The study of cost-sensitive modeling for intrusion detection is both challenging and extremely important. Our main contributions to this study are in the development of a framework for analyzing cost factors and building cost-sensitive models. In doing so, we offer a better understanding of the development and deployment of cost-effective IDSs.

One limitation of our current modeling techniques is that when cost metrics change, it is necessary to reconstruct new cost-sensitive models. For future work, we will study methods for building dynamic models that do not require re-training. These techniques will help reduce the cost of re-learning models due to changes in intra-site cost metrics and deployment at diverse sites with inherently different cost models.

We will also study how to incorporate uncertainty of cost analysis due to incomplete or imprecise estimation, especially in the case of anomaly detection systems, in the process of cost-sensitive modeling. We will also perform rigorous studies and experiments in a real-word environment to further refine our cost analysis and modeling approaches.