Software Defined Networking for Army’s Tactical Network: Promises, Challenges, and an Architectural Approach

Software Defined Networking (SDN) and Network Function Virtualization (NFV) together promise to revolutionize the networking with unprecedented improvements in the speed of introduction of new services, programmability and reconfigurability, resource efficiency, and security posture. While early successes of these concepts have been in database applications, emerging beneficiaries are the enterprise and carrier networks, both wired and wireless. Our qualitative analysis of the SDN/NFV in the context of Army’s Tactical Networks (ATNs) uncovers significant potential but also identifies serious deficiencies in the standard SDN/NFV for this application. We identify a family of architectures, involving hierarchically distributed SDN Controllers, which have the potential of providing the promised benefits while mitigating the disadvantages. We then discuss the required research to add detail to this conceptual framework and to relate the architectural specifics and design parameters to the characteristics of the network. We briefly discuss a prototype novel SDN architecture, consistent with the Framework, for one SDN Controller controlling one network domain.

Introduction and Summary

In this article, we discuss potential benefits and challenges for SDN/NFV in the context of Army’s tactical networks. We discuss a family of architectures for SDN Controllers that could meet these challenges, while providing the promised gains in programmability, efficiency, and security. We also discuss the research needed to select specifics of this architecture as functions of the characteristics of the network.

The separation of control and data (forwarding) planes and the centralization of control functions in a controller are defining characteristics of SDN. The key characteristic of NFV is running multiple networking functions in one general purpose hardware platform as Virtual Machines (VMs) or Containers. In this paper, we will use ‘SDN’ to denote SDN and NFV together.

The separation of control and data planes allows a representation of all of the lower level networking infrastructure as an abstraction to the higher level control and management functions residing in the SDN controller, thus allowing introduction of new services without knowing the specific technologies of the underlying infrastructure. The result is a very high degree of programmability, which allows rapid service development and deployment, easier reconfigurability, and interoperability among diverse networks. In fact, many consider this programmability (and not the separation of forwarding and control planes) the defining characteristic of SDN [1]. With NFV, VMs and resources available to each VM are defined dynamically. In addition, a VM or one of its functions can be moved rapidly from one HW platform to another. Collectively, they permit rapid reconfigurability and adaptability, improved resource efficiency, scalability, redundancy, and security.

Figure 1: Basic SDN Architecture.

Control decisions are made by the SDN Controller based on the status of the network infrastructure and then communicated to the infrastructure nodes for actions. Thus, effective functioning of the standard SDN requires significant amount of information transfer between the SDN Controller and the network infrastructure and this information transfer needs to happen securely with minimal incremental latency. High availability, low latency, and secure communication are thus very critical.

Army Tactical Networks (ATNs) exhibit some unique characteristics. There is a strong hierarchy, mimicking the hierarchy in the force structure There are multi-hop peer-to-peer wireless communication networks (MANETs) within some layers of hierarchy. The data rates in the lower levels of hierarchy are typically low and connectivity is unpredictable. Many devices have limited energy supply. Devices may not have physical protection and may operate in enemy territory. The device could be captured by the adversary, thus providing authenticated access to malicious actors. Unit Task Reorganization (UTRs) are frequent and there is a need for supporting multiple classification levels. Virtual Private Networks (VPNs) are routinely used but are manually configured. A natural question is the value of SDN in such an environment.

In addition to the general benefits mentioned above, SDN can help automate VPN establishments, UTR execution, and distributions of security certificates and patches. It can also improve Network and Cyber Situation Awareness and deployment of policies. On the other hand, unpredictable connectivity and low data rates are perhaps the biggest challenges for the deployment of standard SDN in ATNs since the standard SDN relies on very frequent communication and information transfer between the SDN Controller and network devices. Other big challenges stem from a highly centralized architecture making the SDN Controller a single point of failure as well as the target of cyber-attacks.

As discussed below our proposed family of architectures addresses the above challenges and minimizes the reliability and security risks associated with a highly centralized SDN architecture.

We discuss the work that needs to be done to add specifics to this architecture, relate design parameters to network parameters, prototype key components, and provide early validation.

Figure 2: Army Tactical Hierarchy and Corresponding Networks.

Software Defined Networking (SDN) and Network Functions Virtualization (NFV)

Figure 1 shows the basic architecture of SDN.

SDN enables abstraction of the network infrastructure and data plane in the SDN Controller (SDNC) so a multi-technology, multi-vendor infrastructure is presented to higher layer applications and management functions as a unified network in a standard format. NFV allows creation of multiple virtual entities (VEs) on a single general purpose hardware platform. VEs can be implemented using Virtual Machines (VM) technology [2] or the lighter weight Containers (CONT) technology [3].

As mentioned earlier, we will use ‘SDN’ to refer to SDN and NFV together.

The above properties of SDN enable the network to be ‘softwarized’ and ‘virtualized’ at the SDNC. The following are key benefits:

• Rapid development and deployment of applications
• Separation of networking HW and SW businesses
• High degree of network programmability and reconfigurability
• Potential to enhance the cyber security posture
• Scalability of control and management
• Efficient use of network resources
• Broader situation awareness and faster responses to events
• Easier deployment of cloud computing and other centralization initiatives
• More efficient use of IT experts, very precious resources in tactical environment

Overall, SDN could do for the network infrastructure what the host virtualization has done for the computing infrastructure, namely make the complex network topologies and architectures vendor-independent, easier to monitor and manage, and efficient to operate. While the Southbound interface (between SDNC and network infrastructure) has been standardized (OpenFlow and other solutions), work is ongoing to define the Northbound interface (between SDNC and applications) standards. Applications development and deployment, network management, and configurations could then be much simpler, faster, and cheaper. The combined North and Southbound interfaces will also facilitate development of a common situation awareness, specification and deployment of network management policies, implementation of distributed firewalls, and defense in depth.

Of course, SDN raises concerns about the centralization of controls, resulting in a single point of failure, an attractive target for cyber-attacks, and a traffic bottleneck. In particular, the effectiveness of the standard SDN architecture relies on secure, reliable, and high speed connectivity between the network infrastructure devices and the SDNC.

When multiple network domains are involved, with possibly multiple ownerships, we may need multiple SDN Controllers. IETF is working on Interfacing SDN Domain Controller protocol (SDNi). Multiple SDN Controllers communicating via SDNi will also provide some scalability, incremental deployment of SDN, and support of diverse policies.

Characteristics of ATNs

Figure 2 shows a schematic of ATN architecture. ATNs reflect the hierarchy in Army’s force structure, starting from individual soldier to squad, platoon, company, battalion, brigade, division, and corps. Higher levels of ATNs provide connectivity to the enterprise network. ATN is divided into Lower Tactical Internet (LTI), Mid-Tier Tactical Internet (MTTI), and Upper Tactical Internet (UTI). Within each level of hierarchy at lower levels, Line Of Sight (LOS) connectivity is typically provided by distributed wireless mesh (MANETs) with thin connectivity outside the mesh. Multiple MANETs exist at a given level. Inter-MANET connectivity Beyond Line Of Sight (BLOS) could be provided by SATCOM or other communications relays. At higher levels of hierarchy, there is a mix of terrestrial wireless, SATCOM, and wireline connectivity. In addition to the networks designed for people-people communications, situation awareness, and C2, there are special purpose networks for FIRE support, Logistics, medical, etc.

Virtual Private Networks (VPNs) are used extensively in ATN. Manual setups are time and resource consuming. As the missions and tasks within a mission change, people may get reassigned to different units. These Unit Task Reorganizations (UTRs) are quite frequent in Army tactical environment. They require significant time-consuming reconfigurations. ATNs also support communications at multiple classification levels.

Data rates available within a level of hierarchy are typically low, especially in LTI and MTTI, where they range between 100 kbps and 2 – 5 Mbps. Tactical SATCOM providing BLOS connectivity also have low data rates (100 kbps – 1 Mbps). Also, individual radios are mobile and the entire MANET could be on-the-move. This mobility results in changing RF environment, unpredictable connectivity, and unpredictable data rates. Adversarial jamming adds to the unpredictability of RF links. Given that the user devices may also be network switches and routers, the network topology is governed by the user mission and not by the placement required for optimal RF links, thus increasing the unpredictability. A unit may get physically separated from the rest and may have very limited connectivity with the peer units or with the higher level.

ATNs support communications at different classification levels. Some parts of the network infrastructure may be shared among different classification levels while the others are segregated. Some segregation happens at physical layer, the other happens logically using Virtual Private Networks (VPNs), Multi-Protocol Label Switching (MPLS) and other logical isolation mechanisms.

Possible SDN Architectures for ATN

Given the mission criticality of ATNs, a single SDNC controlling all networking functions in an entire tactical network (say, for an entire Brigade) would not be a desirable solution in any case. It would be a single point of failure for the entire network and a very attractive target for kinetic and/or cyber-attacks. Unreliable and low data rate communication between network devices and SDNC make it even more difficult to give the responsibility of very time critical control functions for a large number of users to a Centralized SDNC. Multiple classification levels add to the difficulty.

On the other hand, SDN concept may offer key advantages to ATNs, in addition to the benefits mentioned above for all SDNs:

• Enabling automation of several very important functions involving tedious manual processes today:
  • Setting up networking in a new location
  • UTR
  • VPN set up and maintenance
  • Management of PKI certificates and patch deployment
• Efficient use of network resources and spectrum, detection of cyber-attacks, and appropriate responses to such attacks via better situation awareness (SA)
• Enabling many breakthrough capabilities such as dynamic spectrum allocation, cognitive networking, and moving target cyber defense.
• Facilitating policy deployment (for network management, quality of service, priorities, cyber defense, etc.).
• Minimizing user interactions with the control plane could reduce the ability of an adversary to inflict major damage using a cyber-attack.

These potential benefits motivate us to investigate SDN architectures that can take advantage of the key concepts but mitigate the concerns expressed above.

Figure 3 is a simplified depiction of the family of architectures we propose for SDN to support ATN. We have a pyramid of SDNCs. Each SDNC is assigned the primary responsibility for at least one Network Domain (ND) and uses a standard southbound protocol (such as OpenFlow) for communication with network devices in that ND. This relationship between an SDNC and ND is as shown in Figure 1. An ND may span one or more levels of the ATN hierarchy. Peer SDNCs are responsible for peer NDs. These peer SDNCs communicate among themselves using the emerging SDNi protocols. SDNCs at a higher level of the pyramid control higher levels of the ATN hierarchy. SNDCs between levels could also communicate using SDNi but the content may reflect the hierarchy. In addition to the primary responsibility for an ND, an SDNC also acts as an indirect higher level Controller for the tree of SDNCs subtending to that SDNC. Figure 4 illustrates direct and indirect control domains of an SDNC.

Figure 3: Hierarchically Distributed Family of SDN Controllers.

We now highlight several important features of this family of SDN architectures.

1. Each SDNC is physically close to the ND it controls and SDNCs are physically separated from one another when NDs are.
2. The pyramid structure limits the size of an ND controlled directly by an SDNC and reduces the fraction of time devices in an ND cannot communicate with the SDNC controlling that ND.
3. Unlike in the standard SDN, we do not require that all control plane functions for an ND are handled by its SDNC. An important architectural decision is the set of control plane functions that we keep tightly coupled with the data plane and managed by distributed controls as is done currently. Based on the discussion above, the functions which require very time critical communication between data and control plane are candidates for not moving to the SDNC. The decisions may be different for different NDs, especially for NDs at different levels of hierarchy.
4. Like the standard SDNC, each of our SDNCs would create an abstraction of the NDs it directly controls and present this software abstraction to the applications and management. However, as discussed in Figure 4, a higher level SDNC provides indirect controls for the NDs controlled by the SDNCs on the tree. This higher level SDNC should have an abstraction of the entire tree of NDs it controls indirectly, in addition to the abstraction of the ND it controls directly. During operations, it should also have SA of the entire tree of NDs and can use this broader SA to make decisions with broader impact than decisions made by a lower level SDNC can have. An interesting set of questions surfaces.
   1. The first set of questions are the granularity of the network infrastructure and SA an SDNC should carry for indirectly controlled NDs and how these abstractions are generated. It makes sense for the higher level SDNC to get a summarized view of indirectly controlled NDs with a greater focus on inter-ND connectivity and edge-to-edge behavior of the traffic through the ND. The SDNC controlling an ND directly could prepare a summarized version of the current status of that ND and send that summary to the next higher level SDNC controlling that ND indirectly. The higher level SDNC can prepare a summary of the summaries about these multiple NDs and send it up to the next higher level SDNC. This approach creates increasingly broader and less granular abstraction of the network as we move up the hierarchy of ATN. The same will apply to the Situation Awareness if we follow a similar approach in propagating SA up the hierarchy.
   2. Similar questions arise about the control flows from higher level SDNCs to lower level SDNCs and then to the NDs. A higher level SDNC could use its broader SA to select effects that optimize the network behavior across all NDs it controls directly or indirectly. For directly controlled NDs, these effects are mapped into actions/controls that are communicated to the ND devices. For indirectly controlled NDs, the selected effects will get communicated to the next lower level SDNCs. Each SDNC at that level adds its more detailed knowledge of the NDs to arrive at actions for the NDs controlled directly by that SDNC or more detailed effects for the next lower level, and so on. The following are some examples of effects selected at higher level: Need for connectivity or additional capacity between two units that are separated without adequate communication capacity (this may get mapped into an action of deploying a communication relay at a lower level); Additions and deletions needed to carry out an UTR across many units; Response to a cyber-attack identified by correlating information from several lower level SDNCs.
   3. 4a and 4b above reflect the application of Mission Command Philosophy [4] to the control and management of ATNs. Namely, the SA is broader but less granular at higher levels of hierarchy. On the other hand, the intent and desired effects selected at higher level are mapped into more detailed actions and controls closer to where the effects are to be realized. This is critical for scalability of network and human resources. It is also very important when the connectivity between levels of hierarchy is unpredictable and an SNDC needs the ability to take local actions based on the status of its ND and guidance and intent from higher levels.
5. The pyramid structure of the SDNC/ND network requires greater processing capacities and data rates at higher levels of the hierarchy, requirements that are easily met because, as we move up the hierarchy, the mobility reduces, enabling increases in the network data rates and device processing capacities. This architecture also supports hierarchically distributed firewalls with increasing complexity as we move up the ATN and SDNC hierarchy. Such an arrangement would facilitate defense in depth.
6. Pyramid architecture for SDNCs allows peer level connectivity among SDNCs at the same level of hierarchy and cross level connectivity between SDNCs at adjacent levels of hierarchy. This richer connectivity can be used to mitigate concerns about the centralization of controls, processing overload, single point of failure, and cyber-attacks.As shown in Figure 5, we can use more than one peer SDNCs to control one ND with one SDNC as the primary and the others as backups. The backups are primaries for other NDs.
7. The pyramid structure of the SDNCs in Figure 3 is used for the cyphertext (black) network carrying encrypted traffic between various plaintext (red) enclaves [4]. Plaintext (red) enclaves may exist at all levels of ATN hierarchy and may have different classification levels. Traffic originating in one red enclave does not typically go to a red enclave at a different level of hierarchy entirely in the red network. Black network connects the pairs of red enclaves. One or more red enclaves can together form a red ND managed by a red SDNC. If traffic is not encrypted, the hierarchy of SDNCs applies to the entire tactical network.

Figure 4: SDNC2 Controls ND2 Directly and ND1s Indirectly.

Figure 5: Primary and Backup SDNCs.

An Experimental SDN Architecture and Prototype Implementation

As discussed above, NDs at tactical edge (e.g. LTI) may keep some control functions in the forwarding plane. As an example, we consider a hybrid architecture where the mobile nodes in such an ND have local autonomy in making forwarding decisions and SDNCs perform less time critical functions like the following:

• Policy dissemination and enforcement: As an example, nodes could act as distributed firewalls sending suspicious packets to the SDNC for further action. Rules for identifying suspicious packets are disseminated by the SDNC.
• Monitoring and re-configuration of nodes: For example, SDNC can re-configure the nodes per UTR policy.
• Gateway functions in networks with multi-level security: The SDNC could use virtualization technologies to provide cross domain solutions.
• Interfacing with other networks and acting as default router: When a packet needs to be forwarded to a location outside of the ND, the SDNC can be the default router for this operation. Direct connectivity to a node belonging to another ND is under consideration.
• Overriding of locally computed routes: An example of route override is when the SDNC decides to blacklist a node and instructs all the impacted nodes in the ND to modify their routing tables accordingly.
• Modification of routing parameters: SDNC may indirectly affect routing by modifying the cost of the links used in the computation of routes in its ND.

Our ND uses Transparent Interconnection of Lots of Links (TRILL) [6] for local routing. TRILL enables mobile nodes to function as Routing Bridges (RBridges) using IS-IS routing protocol [7] at link layer, thus combining the advantages of bridges and routers. TRILL offers the following advantages:

• Optimum (minimum cost) multicast & unicast forwarding
• Fast Convergence times
• Minimal configuration due to link layer operations
• Robust loop mitigation and/or preventions with Time to Live (TTL) marking.
• Scaling to large numbers of MAC addresses
• Equal Cost Multi Pathing (ECMP): Load-splitting among multiple paths
• Multi-pathing support for multicast traffic
• Native support for V-LANs.

Figure 6: Display of SDN Controller Interface.

The control plane consists of a combination of the locally run IS-IS protocol and policy based instructions delivered by the SDNC. Flow Table entries are updated by both the IS-IS protocol and by the SDNC policies. SDNC updates to Flow Tables occur at a slower time scale than IS-IS updates. When there is conflict between IS-IS updates and SDNC updates, the SDNC updates generally override the former, subject to stability constraints.

This hybrid capability is currently being tailored to support scenarios where wireless nodes are automatically reconfigured (IP addresses, frequency spectrum, keys, etc.) based on the requirements of a newly imposed task on the network.

Broad Set of Questions to be Addressed

Section 5 describes a single SDNC and single ND architecture with basic routing function in the data plane, which could be advantageous in some NDs in ATN, especially in LTI. The long term goal is to architect the multi-controller SDN for the entire ATN. The family of architectures and functions defined at a high level in Section 4 (Figure 3) above provide a good starting point for discussing the broad set of questions to be addressed:

• Basic cyber security features to build the minimally required trust.
• Choice of control functions that should remain coupled with the data/forwarding plane of each ND. Other control functions will then be moved to the separate control plane in the directly controlling SDNC. The choice may vary among NDs and will be determined by the characteristics of the network connections in that ND and between ND and SDNC, time criticality of the function being considered for move to SDNC and security implications of the move.
• Comparisons of the alternatives among hierarchically distributed SDNC Networks defined above, and development of a quantitative approach to select the best alternative for a given set of network parameters. In particular,
  • Defining NDs and corresponding directly controlling SDNCs. More but smaller NDs create more SDNCs and allow more redundancy, but also increase communication requirements and could result in the use of less powerful SDNCs
  • Selecting the number of peer SDNCs serving as backups for an SDNC. Larger number will lead to an increased availability of control functions residing in SDNCs and less time in purely autonomous mode. Even greater agility and cyber resilience could then be provided by moving individual functions among SDNC platforms. However, larger number will also lead to ND abstractions maintained in more SDNCs. The resulting increase in the memory and synchronization burden need to be factored in the decision.
  • Number of layers of hierarchy in SDNC network.
  • Choice of a higher level SDNC or a peer SDNC as a backup for an SDNC, depending on the expected connectivity and data rates available between SDNC and ND, additional burden on SDNC, and security profiles of the two choices.
• Approach to developing network abstraction at directly controlling SDNC and at indirectly controlling SDNCs. The latter will require aggregation logic and algorithms. A related question is the network Situational Awareness (SA) for the management of the network. What does the network SA mean to a tactical radio user? To the commander of a tactical unit? To the staff of a brigade commander? How do we build this SA hierarchically?
• Approach to building Cyber SA raises questions similar to the ones for the network SA.
• Appropriate use of Moving Target Defense and other agility and deception mechanisms that are facilitated by the hierarchically distributed SDNCs. This may involve SDNC centralization to coordinate changes in the network designed to offer a moving attack surface. It may also involve moving functions away from the compromised platforms, VMs, or Containers.
• Use of VMs vs Containers to implement VEs. Containers are much lighter weights and so more of them can fit in a platform. However, Containers in one platform share the Operating System software so the security implications need to be factored in.
• Use of SDNC to help set up and manage VPNs.
• Use of SDNCs to facilitate UTRs
• Cross Domain Solution between SDNCs.
• Overall security architecture
• Architecture for hybrid SDN, non-SDN operation

References

1. Bloch, K, “Software defined networks (SDN) – Enabling virtualised, programmable infrastructure”, IEEE Conference on Local Computer Networks Workshop, October 2013.
2. Smith, James; Nair, Ravi, “The Architecture of Virtual Machines”. Computer, 38 (5), 2005, pp: 32–38.
3. T. Andersen, “Past, Present and Future of Containers”, First International Workshop on Container Technologies and Container Clouds, March 2015
4. Col. C. J. Ancker, III, U.S. Army, “The Evolution of Mission Command in U.S. Army Doctrine, 1905 to the Present”, Military Review, March-April 2013.
5. Bharat Doshi, “A Prefix Space Partitioning Approach to Scalable Peer Gateway Discovery in Secure Virtual Private Networks”, IEEE MILCOM’05, 2005, pp. 2735-2741
6. R. Perlman, et al, “Routing Bridges (RBridges): Base Protocol Specification”, IETF RFC 6325 July 2011.
7. R. Callon, Use of OSI IS-IS for Routing in TCP/IP and Dual Environments, IETF RFC 1195 December 1990.