Learning Cyber Operations Through Gaming: An overview of current and up and coming gamified learning environments

Cyber warfare, cyberterrorism, and cybercrime are serious existential threats to the national security of the United States. This is driving a demand signal for expert cyber operators that is far outpacing the supply. As a result, there is a pressing need to rapidly establish innovative, effective, efficient and responsive cybersecurity education and training programs. One program, the Cyber Operators Academy Course (COAC) began to solve the problem but is just one component of a comprehensive cybersecurity education and training strategy. As a part of that strategy, other methods including well-designed games should be considered and interestingly enough, there are many parallels between COAC and what occurs in well-designed serious games or gamified learning environments. This article discusses games and gamified learning environments’ place in the cybersecurity training and culminates with brief overviews of four programs currently or imminently available.

Background

Cyber warfare, cyberterrorism, and cybercrime are serious existential threats to the national security of the United States. This is driving a demand signal for expert cyber operators that is far outpacing the supply. One of the root causes of this imbalance is that existing training programs either do not produce enough qualified personnel to meet the demand signal or those that are produced by the cyber education and training pipelines lack the skills to effectively counter the countless numbers and scale of cyber-attacks from criminals and our adversaries. Compounding this issue, the cybersecurity ecosystem continues to evolve at pace of change that is measured in days, as hackers continuously probe our defenses to identify vulnerabilities in this hyper-dynamic operating environment. As a result, there is a pressing need to rapidly establish innovative, effective, efficient and responsive cybersecurity education and training programs.

A current program within the Office of the Secretary of Defense’s Force Training Directorate called the Cyber Operators Academy Course (COAC) has shown that using a constructivist theoretical framework and a journeyman-apprentice learning model incorporating situated learning, problem-based learning, experiential learning, and cognitive apprenticeship is not only innovative but highly effective way to train cyber operators. Treating cyber operations as a cognitive “trade” led the course designer to use the cognitive apprenticeship learning model (Collins, A., Brown, J. S., & Newman, S. E. ,1987) and a learner-centered curriculum learning environment to effectively teach and amplify skills such as innovation, problem solving, and critical thinking; and enhance inductive thinking processes. The outcome was a learner who developed a thirst for learning along with a strong intrinsic motivation to “teach oneself” (autodidacticism). These approaches took complete novices and brought them to a high level of competence as cyber operators in six months of intensive learning (Gallagher, 2016).

Within COAC, students were divided into learning teams or “fireteams” and are based on the U.S. Marine Corps’ primary infantry fighting unit and the U. S. Army’s Warrior Leaders Course (Department of the Army, 2016). Fireteams worked together collaboratively and cooperatively to bond as a team, solve assigned problems, and compete with each other and in external events. Fireteam leads were assigned to each fireteam and provided mentoring, scaffolding, direction and motivation. These leaders generally acted together as coaches and facilitators in lieu of traditional instructors and were highly qualified subject matter experts in offensive and defensive cybersecurity operations.

This program, though effective requires six months of immersive team-based experiential learning, problem solving and competitive game play between the fire teams (i.e., capture the flag). Due to a requisite high level of subject matter expertise and the need for virtually 24/7 availability for coaching and tutoring, this approach places high demands on the fireteam leads. Due to these conditions, COAC is but one component of a larger comprehensive cybersecurity training strategy. As a part of that strategy, other methods including well-designed games should be considered and interestingly enough, there are many parallels between COAC and what occurs in well-designed serious games or gamified learning environments.

Why Games?

Games and serious games support both generational differences (as they are ingrained within the culture of Generation X, Y and earlier) and a varied, ubiquitous set of technological opportunities that can now be tracked and be leveraged for learning (Gallagher, 2013). In 2016, the video game market in the United States was valued at an estimated 17.69 billion U.S. dollars, approximately three billion more than in 2011. It is projected that the market will be worth 20.3 billion by the end of 2020 (https://www.statista.com/statistics/201073/revenue-of-the-us-video-game-industry-by-segment/). This statistic shows the magnitude of the investment currently made in readily available games. As noted by Chatham (2011), the world is changing at an incomprehensible pace and the military must not only adapt to these changes it must be able to leverage a changing and evolving workforce. Training had to keep up on both fronts and the popularity and ubiquity of computer games suggested that game informed training might be an answer. This led to the development and deployment of such games as America’s Army and DARWARS Ambush. The plausibility of using game based training for cybersecurity builds on a strategy that has been in place within the military for several years.

Well-designed games typically leverage constructivist learning models putting the player into specific situations or problems forcing decision-making and the need to induce rules for incremental success. The game meets the player at his or her emotional, cognitive and/or psycho-motor skill level providing strong emotional connections, context, or goal matching to the environment, puzzles, or problems faced. Games focus on engagement leading to flow (Csikszentmihalyi, 1975) providing a sense of total immersion and intrinsic motivation. Game designs typically rely on the achievement and their broadcasting (typically using leaderboards) of increasingly more difficult goals and, depending on the type of game, collaboration within a team to do so. These are examples of game mechanics that define how one can interact with or within a game through basic actions, processes, and control mechanisms. Other examples of game mechanics are points, levels, and challenges (Bunchball, 2016). Games essentially can allow understanding to develop from interactions with the gaming environment, provide puzzlements facilitating the players’ desire to solve and therefore learn, and potentially allow for the social negotiation of meaning through collaboration either virtual or face to face (Kirkley, Duffy, Kirkley, & Kremer, 2011). These design properties place games within the realm of a constructivist theoretical framework.

To specifically create the conditions that foster the cyber operations cognitive skills, games should also have the features that can stimulate the cognitive processes necessary for these skills. Features (and sub-features) are an addition to the familiar MDA (mechanics, dynamics, aesthetics) model of game design (Salen & Zimmerman, 2004) that come before mechanics and represent general design tenets or desirable characteristics that are translated into the mechanics of a specific game. This creates the hybrid model FMDA. Features in this hybrid model, in turn integrate into the game’s runtime dynamics evoking a particular aesthetic during gameplay (Gallagher & Prestwich, 2012). The emotions corresponding to the aesthetics in the model have themselves been modeled by Lazzaro (2004) producing what she discusses as the four keys to unlock emotion: hard fun, easy fun, altered states, and the people factor. Using both models, features can be explicitly aligned to the desired emotion targeted.

Relationship between features, mechanics, dynamics, and aesthetics with the addition of sub-features (circled representations) in the FMDA model.

For games designed to foster the cognitive skills necessary for cyber operations, the features are unstated/non-explicit rules; unstated/non-explicit changing of rules; dynamic shifting of environments; open ended choices, and implicit reinforcement for actions or choices leading to goal achievement. Taken together, these features lead to the development of cognitive skills including cognitive flexibility, transference, and metacognitive awareness (mindfulness/goal setting). All these attributes contribute to increases in problem solving ability which is related to executive functions. Over the years there have been several studies considering the relationship between playing video games with these features and executive competencies spanning visual attention to fluid intelligence with positive results (Gallagher 2013). Generally, the aesthetic or emotional category aligning to these features within a game is that of “Easy Fun” – the games awakens a sense of curiosity with many options combined with ambiguity and incompleteness as well as detail (Lazzaro, 2004).

One crucial core mechanic that becomes critical is that of time. Time introduces a quantifiable tool for judging performance to game play and can motivate players to not only reach the goal but reach it rapidly and force metacognitive activities to occur in the micromomentary. This is essential for developing expertise. Timed play can include anything where time is measured to the consequence of the player: either rewards for quick action, negative reinforcement for slow action, or actual time limits on the player’s gameplay.

Other than pure game play, many environments may have some combination of gaming mechanics with other uses especially for training. This leads to the concept of gamification. Gamification applies game mechanics to typically non-game activities including training to drive desired behaviors. There are 10 typical mechanics gamification uses for motivation and engagement (Bunchball, 2016):

• Fast Feedback
• Transparency
• Goals
• Badges
• Leveling Up
• Onboarding
• Competition
• Community
• Points

By incorporating the above and other features, mechanics, and aesthetics, games with the right design or well-designed gamified learning environments can develop not only domain specific knowledge but crucial cognitive capabilities as well. Games have historically been used in hacker and cyber competitions. Currently, commercial games, targeted serious games, and gamified learning environments are becoming available to specifically target the types of content knowledge, problem solving and autodidactic behavior necessary to learn cyber operations.

Games place within cybersecurity and cyber operations training

Ever since the hit movie Wargames debuted in 1983 with the iconic phrase, “Shall we play a game?”, the idea of hacking and games, especially wargames and blow up the world games such as Thermonuclear War become indelibly linked (Brown, 2008). As a homage of sorts to Wargames and founded in 1993 by Jeff Moss, DEF CON (also written as DEFCON, Defcon, or DC) is one of the world’s largest hacker conventions, held annually in Las Vegas, Nevada. Attendees to DEF CON include computer security professionals, journalists, lawyers, federal government employees, security researchers, students, and hackers with a general interest in anything that can be “hacked.” The event consists of speaking tracks on computer- and hacking-related subjects, as well as social events and contest or games (DEF CON, 2017).

Besides such contest as lock-picking, robotics, and scavenger hunts is a game called Capture the Flag (CTF). CTF is most likely the best known and is a hacking competition where teams of hackers attempt to attack and defend computers and networks using certain software and network structures. Over the years CTF has been emulated at other hacking conferences as well as in academic and military contexts for such broad uses as entry exams to universities and measurement of skills in cyber protection teams.

CTFs

Used widely for cyber security competitions, Capture the Flag (CTF) games or contests are usually designed to serve to give participants experience in securing a network and/or a machine’s operating system, as well as conducting and reacting to the sort of attacks found in the real world. Typically, skills required to successfully play a CTF are reverse-engineering, network sniffing, protocol analysis, system administration, programming, and cryptanalysis. CTFs typically fall into two types: attack/defense and jeopardy.

Attack/defense CTFs require that each team defend a given machine or small network. Scoring is accomplished on successful defense as well as success at offense or attacking others’ machines or networks. This is represented by a “flag” usually a long hex encoded string that is either prevented from being captured or is planted on an opposing team’s system. Besides DEF CON another annual large scale CTF is held at New York University Cyber Security Awareness Week (NYU-CSAW) – the largest student-centered contest.

Jeopardy-style CTFs involve multiple categories of problems, each of which contains a variety of questions of different point values and difficulties. Teams attempt to earn the most points in the competition’s time frame (for example 24 hours), but do not directly attack each other. Rather than a race, this style of game play encourages taking time to approach challenges and prioritizes quantity of correct submissions over the timing (Wikipedia, 2017; Harmon, 2016).

Games or gamified learning environments for training cyber warriors

Even though games are a logical component to the overall cybersecurity training strategy and have a long legacy within the hacker and cyber communities, there aren’t many serious games or learning environments that have been available until recently. Over the last two to three years, several efforts have been working to produce games or digital gamified learning environments that are serious contenders for teaching deep cybersecurity concepts and skills while reinforcing the cognitive capabilities that make effective cyber operators. The rest of this article is devoted to describing four of these.

ESCALATE

Under development for the past couple of years by the company that has successfully led the development and execution of the Cyber Operators Academy Course (COAC) is a gamified learning environment called ESCALATE. Commercially launched in December 2016, ESCALATE is designed to support the acquisition of skills in the cyber domain which is typically a slow and intimidating process for many novices and professionals alike. Based on challenges not unlike a jeopardy CTF, it is intended to keep avid learners relaxed and engaged through many elements of gamification. For example, it includes the ability to form and compete in teams, provides team and global leaderboards, and awards points for challenge completions. Profile badges are also attained for accomplishing specific achievements.

Using a problem-based approach, challenges are complex with solutions that may not be intuitive or straight forward helping to develop the cognitive skills and problem-solving ability necessary for cyber operators. ESCALATE also incorporates “replayability” by uniquely generating a solution each time a challenge is attempted. Supporting scaffolding and implicit feedback, is just-in-time help based on system connected “helper” material and/or live coaches with struggling learners. However, coaches are there to elicit learner thinking strategies not just to give the answer. Using these elements, ESCALATE works to inspire learner confidences, instill a sense of community, and maximize “on keyboard” for learners that can collaborate and interact with the system and others 24/7.

ESCALATE was developed by Point3 and currently used in the third pilot of COAC as the primary online learning system. It tracks learner behaviors using xAPI (Experience API from ADL or Advanced Distributed Learning – adlnet.gov) and can produce analytics on useful learner behaviors and achievements. It is currently commercially available and for more information contact Point3 at https://point3.net

Learners interacting with Escalate, Copyright 2017, Point3

Project Ares

Project Ares is a gamified, artificial intelligence (AI) powered cyber training environment by Circadence. It uses real-world tools and tactics in immersive, virtual environments. Project Ares gives the learner access to an evolving library of mission scenarios and educational resources such as a how-to video library and various learning games. Learners can work alone or in teams to stop hackers, protect systems, and hone skills inside realistic or mirrors of the business and organizational environments they could eventually defend.

In addition to an overall gamified learning environment, specific learner features also include an AI component powered by IBM Watson™ and SparkCognition™ that acts as a coach, umpire, or even opponent. It provides real-time cyber-attack data that continually evolves and perpetually learns how threats appear, develop, and expose network systems. It provides AI-based monitoring and scoring (umpire), and in-context knowledge to learners and trainers (advisor). AI is also used for offensive and defensive opponents to increase challenge factors.

Learners can also prepare in the Project Ares Battle School, which enables asynchronous practice and review of cyber skills and knowledge. Key features include: cyber games for technical topics (i.e. Cylitaire, PortFlow), battle room for non-mission specific tactical practice, and a media center for videos, documents, and other key resources/websites in cybersecurity.

Skill badges and certifications can be earned on single cyber tasks and in large-scale, cooperative settings. As a MMOG (massively multi player online game), learners or players can work with others to cooperatively solve missions, follow others, communicate and develop a sense of community as they progress.

Using an instructor portal with dashboards for monitoring progress, performance assessment, trainers and instructors can facilitate after-action reviews through mission review and playback as well as providing real-time interventions. For the first year of development of the ADL Total Learning Architecture (TLA), Project Ares functioned as a learning activity provider for the first TLA test and demonstration of a reference implementation at Ft. Bragg. Consequently, it is fully xAPI enabled. Project Ares is currently in use or in collaborative mission development with various DoD components and Services. For more information contact Circandence at www.circandence.com.

Capture The Packet (CTP) by Aries Security was originally created by Brian Markus and two colleagues in 2002 as part of DEFCON and is a training tool that leverages 25 years of development and experience running this game at DEF CON. Designed to train to network defense and offensive capabilities, this system offers a user-friendly interface, capable of expanding challenges suitable for key users, or entire teams. Enhanced through a contract for the Office of the Secretary of Defense’s Force Training Directorate, the enhanced purpose of CTP is to provide a lightweight Cyber Training Capability with “Capture the Packet” functionality that provides a persistent, realistic, end-to-end cyber mission training resource. The status of DoD’s Cyber Mission Force team training capability suggested a lightweight, low-cost, cyber training capability usable from home stations that will support classified and/or unclassified tools was needed. These capabilities were limited by the lack of an integrated portable tool that can provide simple administration, ease of deployment, and semi-automated cyber training challenges. A new cyber training tool was needed to overcome limitations and improve the operational readiness and cutting-edge training of the anticipated 133 Cyber Mission Force teams. Enter Capture the Packet.

Leaderboard and racked equipment for Capture the Packet, Copyright 2017, Aries Security

Originally based on network packet analysis techniques and leveraging the current the Capture The Packet training simulator framework, currently CTP takes advantage of automated tools, techniques, strategies, scoring, and administration capabilities that already existed. It has the ability to train for a spectrum of major threats to obscure tactics. CTP has ready out of the box capabilities, and a portable, enclosed network. According to Aries Security, it allows you to test your team against live threats, evaluate offensive and defensive abilities, and have an actionable growth strategy from day one of rolling out the suite.

Using an existing portable, standalone, ruggedized, 6U system design, CTP provides real-time records of student performance, and skill evaluation of users when operating under time constraints in high-fidelity competitive real-world conditions. This system architecture is designed to easily support classified and/or unclassified training exercises.

Technically, CTP provides a player on player environment in which 10 teams of 5 students each can concurrently compete. The students control the operation of a single virtual machine that houses five binaries. Each binary is a custom service that contains two or more memory corruption level vulnerabilities and has at least one anti-debug or anti-reversing technology applied. Through this, students are capable of identifying, patching and reverse engineering vulnerabilities in executable code and operating systems. Students are responsible for finding vulnerabilities in agent services, constructing the necessary exploit payloads capable of retrieving token values from memory and submitting those token values or “steals” to a scoreboard in order to obtain points. Student teams would also be responsible for defending their services by patching their live environments against discovered vulnerabilities. Points are awarded for steals as well as deducted for any down time of any service. If a team’s service has been compromised, they can force a key reset at the cost of points.

As a standalone solution, CTP doesn’t need an Internet connection making it ideal for secure environments and easily deployed to CONUS and OCONUS locations. For more information contact Aries Security at www.ariessecurity.com.

Cyber Attack Academy (CAA)

Another project completed by the Office of the Secretary for Defense Force Training Directorate is the Cyber Attack Academy (CAA). Developed by Socratic Arts and although not a game per se, CAA provides a problem based approach within a role-playing scenario. It could be called a blended learning course that is self-paced, immersive, story-centered curriculum but that description may not be doing it justice. It also has live and AI-based tutoring providing scaffolds to take you from a complete novice to one who can do such things as reverse engineering within the first task. In a story-centered curriculum, students play an authentic role (e.g., that of a cyber operator) in a realistic story of professional work designed with a pedagogical intent, meaning that the story is designed to require the successful application of targeted knowledge and skills to achieve the goals set for them. Students will do the same work as professionals and will produce the same work products. As in professional practice, some work will be individual and other work will be team based. As they work, students can make use of structured performance support materials including a “plan of attack” for accomplishing the work and learning resources key to aspects of their tasks.

Students will also have access to knowledgeable human mentors who can provide help and advice and, more importantly, feedback on drafts of student work.” In most cases 80-90% of the questions that students ask can be anticipated (i.e., the questions recur regularly), so the learning environment is augmented with an artificial intelligence-based automated mentor. Integrated into the learning environment, it uses natural language processing to extract key semantic features from student questions in specific task contexts and then use those features to retrieve high-quality expert answers to those questions in both video and textual formats. The 10-20% of questions for which the AI Mentor cannot retrieve suitable answers will be referred by the AI mentor to expert human mentors. Their answers can then be incorporated into the AI mentor extending its capability. Automated mentoring of this sort is intended to offload routine question answering from the human mentors, enabling a substantial increase in the student-to-mentor ratio. This provides a scalable training solution enabling the government to address the significant shortage of skilled cyber operators more quickly and effectively.

In essence, CAA provides the following capabilities, functionality and/or attributes:

• A remote platform to learn the cyber operator trade through autodidactic learning with mentor support which can either be live or through an artificial intelligence-based mentor with a natural language processing interface.
• Interweaving an on-line, digital, problem based learning environment with live (either in person or via digital means) mentor/coaching.
• Cyber operators with a wide array of technical capabilities, inclusive of basic cyber operations skills in offense, defense, forensics, scouting, and hunting, cyber red team, penetration testing, acquisition and analysis of publicly available information, the Dark and Deep Web and other cyber operations topics achievable within the four to six- month duration of the course.
• Reinforcement of the roles of anthropology, sociology and ethnography to better understand potential adversaries and themselves
• Both remote individual and team learning and a culminating “All Against All” team-based Capture the Flag (offense/defense) exercise to conclude the course.
• The opportunity for a cohort to pass a performance-based industry standard assessment, such as the Offensive Security Certified Professional (OSCP) credential, or a similar industry defensive oriented cyber operations credential, and at a minimum achieve equivalency with Department of Defense’s (DoD) Joint Cyber Analysis Course.
• Instrumented to allow the mentor (live or AI-based) to analyze learner performance and more quickly provide advice to the learner.
• Mentors (live or AI-based) with both formative and summative evaluations of the overall performance of the course and cohort performance.
• Be accessible to the government via commercial internet service providers from a cloud-based and/or local digital repository, with a secure password protected log in or other means of ensuring only authorized learners have access to the course.
• Provide unlimited use of the course to the Government.

CAA has recently finished development and will soon be available for access through the Web. To look at the environment and a demonstration of the course go to https://www.schankacademy.com/cyber-attack-academy. For more information go to https://www.schankacademy.com/contact.

Conclusion

In conclusion, games, serious games and gamified learning environments are powerful tools for engaged and motivated learning experiences. With the right design, they are capable of teaching the cognitive skills required for cyber operations and can be an essential component of the strategic and comprehensive cybersecurity training strategy which is a national imperative. Games and digital gamified learning environments can and should be available to provide innovative, effective, efficient and responsive cybersecurity education and training solutions. The four games discussed provide an overview of where these environments are headed and what is or will shortly be available to help produce enough qualified personnel to meet the demand signal or those that are produced by the cyber education and training pipelines.

References

1. Brown, Scott (2008). WarGames: A Look Back at the Film That Turned Geeks and Phreaks into Stars. Wired. https://www.wired.com/2008/07/ff-wargames.
2. Bunchball (2016). Gamification 101: An Introduction to the Use of Game Dynamics to Influence Behavior. Retrieved from https://www.bunchball.com/gamification101.
3. Chatham, R. E. (2011). After the Revolution: Game-Informed Training in the U.S. Military. In S. Tobias & J. D. Fletcher (Eds.), Computer Games and Instruction. Information Age Publishing. Charlotte.
4. Collins, A., Brown, J. S., & Newman, S. E. (1987, January). Cognitive apprenticeship: Teaching the crafts of reading, writing, and mathematics. Center for the Study of Reading, Technical Report #403, 2-37.
5. Csikszentmihalyi, Mihaly (1975). Beyond Boredom and Anxiety: Experiencing Flow in Work and Play, San Francisco: Jossey-Bass.
6. DEF CON (2017, September 14). The DEF CON Story. Retrieved from https://www.defcon.org/html/links/dc-about.html.
7. Department of the Army. (2016). Warrior Leader Course. Washington, DC.
8. Gallagher, P. S. (2016). Assessing performance in an innovative cybersercurity pilot course. Paper presented at the Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) 2016. Orlando.
9. Gallagher, P. S., & Prestwich, S. (2012). Supporting cognitive adaptability through game design. In Proceedings of the 6th European Conference on Games-Based Learning, 165–173. Cork, IE: 6th European Conference on Games Based Learning.
10. Gallagher, P. S. (2013). Transforming Education through Neuroscience, Cognition, and Game Design. In L Wells, T. Hailes, and M. Davies (Eds.), Changing Mindsets to Transform Security: Leader Development for an Unpredictable and Complex World. Center for Technology and National Security Policy Institute for National Strategic Studies National Defense University. Washington, D.C.
11. Harmon, Timothy D. (2016, September 14). Cyber Security Capture the Flag (CTF): What Is It? Retrieved from https://blogs.cisco.com/perspectives/cyber-security-capture-the-flag-ctf-what-is-it
12. Hunicke, R., LeBlanc, M. & Zubek, R. (2004). MDA: A Formal Approach to Game Design and Game Research. AAAI Workshop – Technical Report 1.
13. Kirkley, J. R., Duffy, T. M., Kirkley, S. E., & Kremer, D. L. H. (2011). Implications of Constructivism for the Design and Use of Serious Games. In S. Tobias & J. D. Fletcher (Eds.), Computer Games and Instruction. Information Age Publishing. Charlotte.
14. Lazzaro, N. (2004). Why We Play Games: Four Keys to More Emotion Without Story. XEODesign, Inc. Oakland.
15. Salen, K., & Zimmerman, E. (2004). Rules of Play: Game Design Fundamentals. The MIT Press.
16. Schank Academy (2017, September 7). Cyber Attack Academy. Retrieved from https://www.schankacademy.com/demos/cyber/index2
17. Wikipedia (2017). Capture the Flag. https://en.wikipedia.org/wiki/Capture_the_flag.