VI. Use of Metrionome During Red Teaming
We have successfully used Metrinome during internal security testing of software artifacts developed under the Secure Tactical to Enterprise Gateway (STEG) [20] R&D effort. To evaluate the security benefits of STEG, we build an internal threat model that decomposes attacks into three main categories, namely, attacks that cause (1) loss of integrity, e.g., by corrupting service logic or changing data in transit, (2) loss of availability, e.g., by crashing critical components or exhausting shared resources, and (3) loss of confidentiality, e.g., by getting unauthorized access to sensitive information. The attacks are then further decomposed into sub-categories for each category (i.e., Integrity, Availability, and Confidentiality). The model can be visually represented as attack graphs, with annotations for defenses and logical arguments.
Figure 7. STEG Attack Tree for Loss of Integrity
Figure 7 shows the resulting attack graph for integrity. The graph reads from left to right and first branches out into high-level attack strategies, e.g., Impersonate Client and Publish corrupted IOs. The next levels then provide functional refinements for the attacks. Attack refinement may lead to multiple alternatives (branches). The next level of the attack graph is annotated with mitigated by, indicating the defensive component that addresses the particular attack represented by the branch. Note that an attack strategy may have multiple mitigating defenses (indicated by the mitigated by annotation on a branch). For the cases where mitigation is verified by experimental observation or logical arguments, the attack graph is shown with an additional level, annotated with verified by describing how we determined that the STEG prototype actually addresses the threat.
We used Metrinome to establish and document correct security functionality by measuring a number of metrics listed in the attack tree, including TLS authentication failures, identity mapping failures, authorization failures, and anti-virus filtering failures.
VII. Conclusion and Future
Metrinome has proven to be an effective component in supporting runtime assessment and monitoring, demonstrations and scientific experimentation during execution of the STEG R&D effort. In particular, the integration of end-to-end testing into the continuous build cycle has helped identification and mitigation of run-time bugs.
Going forward, we expect Metrinome to grow as it is adopted by other efforts with extended requirement sets. In particular, we have plans to (1) make it easier to add custom functions without the need to recompile the Metrics Server through a plugin framework, (2) provide capabilities for more complex graph generation, e.g., by providing boxplots via integration with R [21], (3) provide the ability to define metrics over metrics and metrics capturing trends, and (4) implement an adapter layer for ingesting messages other than Logback.
References
[1] Eoin Keary, Integration into the SDLC(Software Development Life Cycle), Retrieved Nov 06 2013, https://www.owasp.org/images/f/f6/Integration_into_the_SDLC.ppt
[2] JUnit Homepage, Retrieved Sep 06 2013, https://github.com/junit-team/junit/wiki/Getting-started
[3] Nessus Vulnerability Scanner, Retrieved Sep 06 2013, http://www.tenable.com/products/nessus
[4] HP Fortify My App, Retrieved Sep 06 2013,
[5] YourKit Profiler, Retrieved Sep 06 2013, http://www.yourkit.com/
[6] Grinder, Retrieved Sep 06 2013, http://grinder.sourceforge.net/
[7] Network Management Information System, Retrieved June 10 2013, http://www.sins.com.au/nmis/sample/
[8] Software Testing Automation Framework, Retrieved June 10 2013, http://staf.sourceforge.net/
[9] Java Simon – Simple Monitoring API , Retrieved June 10 2013, http://code.google.com/p/javasimon/
[10] Metrics, http://metrics.codahale.com, Retrieved June 10, 2013
[11] Ganglia Monitoring System, Retrieved June 10 2013, http://ganglia.sourceforge.net/
[12] Graphite – Scalable Realtime Graphing, Retrieved June 10 2013, http://graphite.wikidot.com/
[13] Splunk, http://www.splunk.com/ Retrieved June 10 2013.
[14] ArcSight, http://en.wikipedia.org/wiki/ArcSight
[15] Cody Burleson, “How to setup SLF4J and LOGBack in a web app – fast”, Apr 10 2013, https://wiki.base22.com/display/btg/How+to+setup+SLF4J+and+LOGBack+in+a+web+app+-+fast
[16] Fielding, Roy Thomas, “Architectural styles and the design of network-based software architectures”, Diss. University of California, 2000.
[17] Jenkins: An extendable open source continuous integration server, http://jenkins-ci.org/ Retrieved July 1 2013.
[18] Kelley Dempsey, Nirali hawla, Arnold Johnson, Ronald John-ston, Alicia Clay Jones, Angela Orebaugh, Matthew Scholl, and Kevin Stine, “Information Security Continuous Monitoring (ISCM) for Federal Information Systems and Organizations”, Retrieved June 25 2013 http://csrc.nist.gov/publications/nistpubs/800-137/SP800-137-Final.pdf
[19] Jason Miller, “Agencies struggle with continuous monitoring mandate”, Retrieved June 25 2013 http://www.federalnewsradio.com/513/2681377/Agencies-struggle-with-continuous-monitoring-mandate
[20] “R: Box Plot Statistics”, R manual, Retrieved June 3 2013, http://stat.ethz.ch/R-manual/R-devel/library/grDevices/html/boxplot.stats.html
[21] “Secure and QoS-Managed Information Exchange between Enterprise and Constrained Environments”, currently in submission to appear in Proceedings of ISORC 2014.