Identifying the Gap: The System Engineering Framework
Development of a SIS encompasses three aspects: Business, System and Software. Though there is overlap among these aspects, specific responsibilities can be attributed to each Aspect.
Figure 2 General Aspect Characteristics
Various agile frameworks exist in both the Business and Software Aspects. The Business Aspect has frameworks such as the Business Capabilities Lifecycle (BCL) and the Defense Science Board Agile Framework (Force 2009, Hand and Little 2012). The Software Aspect can utilize frameworks such and Scrum; however, there is a large gap between the Business and Software Aspect frameworks which does not allow for complete synchronization between all Aspects. The Business Aspect framework defines deliverables at a high level with typical delivery times of 12-18 months whereas the Software Aspect is more granular defining a single delivery in weeks. The System Aspect needs to provide a framework which allows for the management of multiple asynchronous Software Aspect deliveries to complete the desired capability defined in the Business Aspect (Figure 3). With the vast complexity of evolving SIS this framework gap does not allow for easy incorporation of system engineering best practices such as interface management, modular / open designs, configuration management or risk management.
Figure 3 Missing System Aspect Framework (Graphic by Kelly Helms)
Figure 4 Agile System Engineering Framework
An Agile System Engineering Framework was developed for the System Aspect to foster agile practices in the Software Aspect as well as retain the ability to rapidly respond to changes from the Business Aspect (Figure 4). This framework also enables the application of systems engineering best practices to be used throughout system development.
The Agile System Engineering Framework defines three main phases: Release, Increment and Integration. Each phase is completed with a retrospective. The Agile Systems Engineering Framework is comprised of a single release divided into a series of Increments, with each Increment containing one or more Sprints and Integration blocks. A retrospective assesses each phase and provides lessons learned in order to improve the current processes the next time the phase is implemented. The Agile Systems Engineering Framework provides a good foundation for agile system engineering; however, to further incorporate system engineering best practices and agile practices further descriptions such as input / exit criteria and activates to be complete during each phase was defined to further assist in the operational implementation of the framework. These phase descriptions are defined below.
Release Phase
The product of the release is the delivered system. It starts with a Release Planning meeting(s), consists of multiple Increments and is completed with a Release Retrospective.
Input Criteria: High Level Design
Exit Criteria: Finished product to be fielded
Release Planning
Activities in this Phase
- Requirements Engineering
- Increment Time Estimation
- Identify Key System Interfaces
Exit Criteria: Prioritized Release Backlog
Increment Phase:
The Increment Phase receives the prioritized Release backlog from Release Planning Phase. The output of an Increment is an item that is placed under configuration management. Each Increment consists of one or more Sprints and Integration phases.
Input Criteria: Prioritized Release backlog
Exit Criteria: Finished “Configuration Item”
Increment Planning
Input Criteria: Prioritized Release Backlog
Activities in this Phase
- Decompose the system into functional items
- Identify high risk items
- Identify Key System Integration Points
- Identify / further define Key System Interfaces
- Specify temporal dependencies among Sprints; i.e., determine which Sprints can be conducted concurrently and which must be conducted sequentially
- Select what can be done at each Increment based on the prioritized release backlog
- Identify the personnel / resources / skill set that should be involved in the Increment
- For each Sprint
- Identify ‘customer’
- Specify the definition of ‘Done’
- Specify / Identify expected outputs / specifications for each Sprint
Exit Criteria
- Prioritized Sprint backlog(s)
- Incremental program plan identifying Sprints and Integration points
Sprint
A Sprint consists of a time-boxed window for producing a potentially shippable product to be integrated into the system in the parent Increment or Integration Phase. The Sprint block is where development of any kind occurs and is handled as a black box within the Agile System Engineering Process. Here, a form of “black box trust” occurs allowing each Sprint to develop the specified product freely provided the product is completed using the minimum required specifications and interfaces provided in the Input Criteria. The Sprint development risk is managed by a combination of the input specification / interfaces and the development time-boxed window. In general, the shorter the development time, the less the investment. A product could include software and /or hardware but may also be items like a Commercial-Off-The-Shelf (COTS) product evaluation assessment. Multiple Sprints can be underway concurrently.
Input Criteria
- Sprint Backlog
- Specifications / Interfaces
- Customer Identification
- Definition of “Done”
Activities in this Phase: Item Development
Exit Criteria: Completed / user-accepted product(s)
System Integration Phase
The integration phase combines various elements of the overall SIS. These elements could be a combination of hardware and / or software produced by the Sprints and / or the incorporation of Government-Off-The-Shelf (GOTS) / Commercial-Off-The-Shelf (COTS) products required by the SIS. This is the phase where high risk pieces can be incrementally constructed to assess the feasibility of the combination of components.
Input Criteria: Completed items and specifications / interfaces for the systems to be integrated from previous Sprints, Increments and / or GOTS / COTS.
Activities in this Phase
- System Integration
- Specifications / interfaces validation / review / refinement
Exit Criteria: User accepted, integrated system or subsystem
Juggernaut was provided with the Agile System Engineering Framework and Descriptions to facilitate the gap between the Business and Software Aspect (Figure 5). In addition, Juggernaut was provided a list of agile practices found effective in both the Business and Software Aspects (Table 2).
Figure 5 Complete Framework Interaction (Graphic by Kelly Helms)
The Framework and Practices were delivered to Juggernaut in two face-to-face training sessions. During these meetings the case study primary researcher was also provided an overview of the product under development. In addition to the training meetings, quarterly teleconferences and virtual meetings were held to provide progress and feedback on the implementation of the agile systems engineering process. Information was also collected and shared through email correspondence which totaled over 75 email exchanges throughout the development process.
| Incremental Development | Small Teams |
| Iterative Development | Time-Boxing |
| Short Time-lines | Lean Initiatives |
| Retrospectives (Lessons learned) | Prototyping |
| Empowered/ Self-organizing/Managing teams | Continuous User Involvement |
| Prioritized Product Backlog (Requirements) | Co-located Teams |
Table 2 Agile Systems Engineering Practices (Kennedy and Ward 2012)
Upon acceptance of the Framework and Practices, Juggernaut reengineered its project plan to conform to the new framework. Based on analysis from the Release Planning Phase, three Increments were planned, each containing Sprints that were designed to coincide with key integration points in which a combination of internally and externally developed hardware, firmware, enclosures and / or software needed to be integrated and tested by the Quality Control (QC) group.
At these key integration points, the hardware and software were required to have certain functionality and the QC group had to have the necessary outside resources, equipment and personnel to run specific integration tests. As defined in the process, each integration point had a set of predetermined input and exit criteria used to measure whether a successful integration was accomplished.
Reorganizing the project into the new structure allowed for the identification of critical system components and interfaces. High risk items could be identified and completed early and integration risks could be mitigated by performing incremental integration throughout development. The new structure also allowed for more accurate project tracking since each sprint constituted a completed item as defined by the predetermined “definition of done” versus attempting to track items using an estimated percent complete methodology. No item should be given credit until it is 100 percent complete.