AAS Best Practices:
1. Use Scrum of Scrums carefully when coordinating multiple teams. Scrum is the most often used Agile method in today’s environment, and primarily involves team management practices. In its simplest instantiation, a Scrum development environment consist of a single Scrum team with the skills, authority and knowledge required to specify requirements, architect, design, code, and test the system. As systems grow in size and complexity, the single team mode may no longer meet development demands. If a project has already decided to use a Scrum-like project-management technique, the Scrum approach can be extended to managing multiple teams with a “Scrum of Scrums,” a special coordination team whose role is to (1) define what information will flow between and among development teams (addressing inter-team dependencies and communication) and (2) identify, analyze, and resolve coordination issues and risks that have potentially broader consequences (e.g., for the project as a whole). A Scrum of Scrums typically consists of members from each team chosen to address end-to-end functionality or cross-cutting concerns such as user interface design, architecture, integration testing, and deployment. Creating a special team responsible for inter-team coordination helps ensure that the right information, including measurements, issues, and risks, is communicated between and among teams. But care needs to be taken when the Scrum of Scrums team itself gets large to not overwhelm the team. This can be accomplished by organizing teams—and the Scrum of Scrums team itself—along feature and service affinities. We further discuss this approach to organizing teams in our Feature-Based Development and System Decomposition practice. Such orchestration is essential to managing larger teams to success, including Agile teams.
2. Use an architectural runway to manage technical complexity. Stringent safety or mission-critical requirements increase technical complexity and risk. Technical complexity arises when the work takes longer than a single iteration or release cycle and cannot be easily partitioned and allocated to different technical competencies (or teams) to independently and concurrently develop their part of a solution. Successful approaches to managing technical complexity include having the most-urgent system or software architecture features well defined early (or even pre-defined at the organizational level, e.g., as infrastructure platforms or software product lines).
The Agile term for such pre-staging of architectural features that can be leveraged by development teams is “architectural runway.” The architectural runway has the goal of providing the degree of stability required to support future iterations of development. This stability is particularly important to the successful operation of multiple teams. A system or software architect decides which architectural features must be developed first by identifying the architecturally significant requirements for the system. By initially defining (and continuously extending) the architectural runway, development teams are able to iteratively develop customer-desired features that leverage that runway and benefit from the quality attributes they confer (e.g., security).
Having a defined architectural runway enables technical risks to be uncovered earlier, thereby helping to manage system complexity (no late surprises). The consequence of uncovering underlying architectural concerns such as security, performance, or availability late—that is, after several iterations have passed—often is a significant rework rate and schedule delay. Delivering functionality is more predictable when the infrastructure for the new features is in place so it is important to maintain a continual focus on the architecturally significant requirements and estimation of when the development teams will depend on having code that implements an architectural solution.
3. Align Feature-Based Development and System Decomposition. A common approach in Agile teams is to implement a feature (or user story) in all the components of the system. This gives the team the ability to focus on something that has stakeholder value. The team controls every piece of implementation for that feature and therefore they do not have to wait until someone else outside the team has finished some required work. We call this vertical alignment because every component of the system required for realizing the feature is implemented only to the degree required by the team.
However, system decomposition could also be horizontal, based on the architectural needs of the system, focusing on common services and variability mechanisms promoting reuse.
The goal of creating a feature-based development and system decomposition approach is to provide flexibility in aligning teams horizontally, vertically, or in combination, while minimizing coupling to ensure progress. Although organizations create products in very different domains (embedded systems to enterprise systems) similar architecture patterns and strategies emerge when a need to balance rapid progress and agile stability is desired. The teams create a platform containing commonly used services and development environments either as frameworks or platform plug-ins to enable fast feature-based development.
4. Use quality-attribute scenarios to clarify architecturally significant requirements. Scrum emphasizes customer-facing requirements—features that end users dwell on—and indeed these are important to success. But when the focus on end-user functionality becomes exclusive, the underlying architecturally significant requirements can go unnoticed.
Superior practice is to elicit, document, communicate, and validate underlying quality-attribute scenarios during development of the architectural runway. This becomes even more important at scale when projects often have significant longevity and sustainability needs. Early in the project, evaluate the quality-attribute scenarios to determine which architecturally significant requirements need to be addressed in early development increments (see architectural runway practice above) or whether strategic shortcuts can be taken to deliver end-user capability more quickly.
For example, will the system really have to scale up to a million users immediately, or is this actually a trial product? There are different considerations depending on the domain; for example, IT systems use existing frameworks, so understanding the quality-attribute scenarios can help developers understand which architecturally significant requirements might already be addressed adequately within existing frameworks (including open-source systems) or existing legacy systems that can be leveraged during software development. Similarly, such systems have to deal with changing requirements in security and deployment environments that necessitates architecturally significant requirements to be top priority when dealing with scale.
5. Use test-driven development for early and continuous focus on verification. This practice can be summarized as “write your test before you write the system.” When there is an exclusive focus on “sunny-day” scenarios (a typical developer’s mindset), the project becomes overly reliant on extensive testing at the end of the project to identify overlooked scenarios and interactions. Therefore, be sure to focus on rainy-day scenarios (e.g., consider different system failure modes) as well as sunny-day scenarios. The practice of writing tests first, especially at the business or system level (which is known as acceptance test-driven development) reinforces the other practices that identify the more challenging aspects and properties of the system, especially quality attributes and architectural concerns (see architectural runway and quality-attribute scenarios practices above).
6. Use end-to-end testing for early insight into emerging system properties. To successfully derive the full benefit from test-driven development at scale, consider early and continuous end-to-end testing of system scenarios. When teams test only the features for which they are responsible, they lose insight into overall system behavior (and how their efforts contribute to achieving it). Each small team could be successful against its own backlog, but someone needs to be looking after broader or emergent system properties and implications. For example, who is responsible for the fault tolerance of the system as a whole? Answering such questions requires careful orchestration of development with verification activities early and throughout development. When testing end to end, take into account different operational contexts, environments, and system modes.
At scale, understanding end-to-end functionality requires its elicitation and documentation. This can be achieved through use of agile requirements management techniques such as stories as well as use of architecturally significant requirements. However, if there is a need to orchestrate multiple systems, a more deliberate elicitation of end-to-end functionality as mission/business threads should provide a better result.
7. Use continuous integration for consistent attention to integration issues. This basic Agile practice becomes even more important at scale, given the increased number of subsystems that must work together and whose development must be orchestrated. One implication is that the underlying infrastructure that developers will use day to day must be able to support continuous integration. Another is that developers focus on integration earlier, identifying the subsystems and existing frameworks that will need to integrate. This identification has implications for the architectural runway, quality-attribute scenarios, and orchestration of development and verification activities. Useful measures for managing continuous integration include rework rate and scrap rate. It is also important to start early in the project to identify issues that can arise during integration. What this means more broadly is that both integration and the ability to integrate must be managed in the Agile environment.
8. Consider technical debt management as an approach to strategically manage system development. The concept of technical debt arose naturally from use of Agile methods, where the emphasis on getting features out quickly often creates a need for rework later. At scale, there may be multiple opportunities for shortcuts, and understanding technical debt and its implications becomes a means for strategically managing the development of the system. For example, there might be cases, where to accelerate delivery, certain architectural selections are made that have long-term consequences. Such tradeoffs must be understood and managed based on both qualitative and quantitative measurements of the system. Qualitatively, architecture evaluations can be used as part of the product demos or retrospectives that Agile advocates. Quantitative measures are harder but can arise from understanding productivity, system uncertainty, and measures of rework (e.g., when uncertainty is greater, you might be more willing to take on more rework later). Several larger organizations have started to look into technical-debt management practices organizationally.
9. Use prototyping to rapidly evaluate and resolve significant technical risks. To address significant technical issues, teams employing Agile methods will sometimes perform what in Scrum is referred to as a technical spike, in which a team branches out from the rest of the project to investigate the specific technical issue, develop one or more prototypes to evaluate possible solutions, and bring back what was learned to the project so that it can proceed with greater likelihood of success. A technical spike may extend over multiple sprints, depending on the seriousness of the issue and how much time it takes to investigate the issue and bring back information that the project can use.
At scale, technical risks having severe consequences are typically more numerous, and so prototyping (and other approaches to evaluating candidate solutions such as simulation and demonstration) can be an essential early planning but also recurring activity. A goal of Agile methods is increased early visibility. From that perspective, prototyping is a valuable means of achieving visibility more quickly for technical risks and their mitigations. The Scrum of Scrums practice mentioned earlier has a role here, too, for helping to orchestrate bringing back what was learned from prototyping to the overall system.
10. Use architectural evaluations to ensure that architecturally significant requirements are being addressed. While not considered part of mainstream Agile practice, architecture evaluations have much in common with Agile methods in seeking to bring a project’s stakeholders together to increase their visibility into and commitment to the project, and to identify overlooked risks. At scale, architectural issues become even more important, and architecture evaluations thus have a critical role on the project. Architecture evaluation can be formal, as in the Software Engineering Institute’s Architecture Tradeoff Analysis Method, which can be performed, for example, early in the Agile project lifecycle before the project’s development teams are launched, or recurrently. There is also an important role for lighter weight evaluations in project retrospectives to evaluate progress against architecturally significant requirements.