A digest of organizational structures in quest for continuous delivery

🇬🇧 🇧🇷

The DevOps movement, as vividly portrayed in the novel The Phoenix Project, came up as a cultural shift to break down the silos in large organizations, better integrating development and operations teams through collaboration. However, this collaboration can happen in different ways from an organizational perspective: developers and infrastructure specialists can be part of different departments or can be together in a single cross-functional team. With advancements in PaaS offers, it is possible even to envision developers themselves taking operations responsibilities.

Our research at IME-USP (University of São Paulo) aims to investigate how software-producing companies are organizing their development and operations teams. We are taking this endeavor by interviewing software professionals to understand how things are really happening in the real world. With our research, we hope to provide a theory to support organizations in designing their organizational structures toward continuous delivery and providing guidance on the consequences of a given organization structure choice.

Based on the careful analysis of the conducted interviews, we elaborated a theory describing the organizational structures used by industry in the real world regarding how the work of developers and infrastructure engineers can be coordinated in the pursuit of continuous delivery. In this digest, we present our current understanding of such organizational structures: segregated departments, collaborating departments, API-mediated departments, and single departments. Each of these structures has core properties, commonly seen in organizations with a given structure, and supplementary properties, which may or may not be present, but, when present, supports the structure explanation.

Figure 1 - The four organizational structures of our taxonomy

The presentation of this taxonomy has already been accepted for publication in the Information and Software Technology, a very reputable academic journal in our field.

Download the full paper here

Between the paper submission and acceptance (almost one year), we have been working to evolve our taxonomy based on its use with practitioners. The most relevant changes from this evolution are the renaming of concepts, aiming for more objectiveness and less confusion for the taxonomy users. In this digest, we present the taxonomy in its latest version (June 2021).

Figure 2 - High-level view of our taxonomy in its latest version (jun 2021)

In the following, we present each structure alongside its core and supplementary properties.

Segregated dev & infra departments

Segregated, or siloed, departments is “the pre-DevOps” structure existing in large organizations, presenting limited collaboration among departments and barriers for continuous delivery. This structure is considered to be the problem that DevOps came to solve.

Figure 3 - With segregated departments, operators and developers are each one in their own bubbles. They do not interact directly too much, and communication flows slowly by bureaucratic means (ticket systems).

Core properties:

• Developers and operators have well-defined and different roles.
• Developers have a minimal vision of what happens in production.
• Monitoring and handling incidents are done mainly by the infrastructure team.
• Developers often neglect non-functional requirements (NFR).
• There are many conflicts among silos since each department aims its own interests, looking for local optimization, not the global optimization of the whole organization.
• DevOps initiatives are centered on adopting continuous integration tools rather than improving collaboration among silos.
• The lack of proper automated tests can hinder delivery performance and reliability.

Collaborating dev & infra departments

This structure focuses on collaboration among developers and the infrastructure team. It is the way the book “The Phoenix Project” portrays DevOps.

Figure 4 - With collaborating departments, operators and developers in different departments seek to work together, even if not easy, by direct contact and close collaboration.

Core properties:

• There is a culture of collaboration and communication among departments, which work in an aligned way.
• NFR responsibilities are shared among developers and the infrastructure team.
• The infrastructure staff is still in the front line of tracking monitoring and incident handling.
• While developers feel alleviated because they can count on the infrastructure team, stress can persist at high levels for the infrastructure team.
• There is no apparent correlation between this structure and delivery performance.

Supplementary properties:

• Infra people as development collaborator: infrastructure engineers have advanced coding skills and contribute to the application code-base to improve the non-functional properties of the application.
• Infra people occasionally embedded in product-centered dev teams: sometimes, collaboration occurs with an infrastructure professional spending some time in tight collaboration with a development team, especially at the beginning of a new project.

API-mediated dev & infra departments

In this case, there is an infrastructure team (also called the platform team), and it provides highly automated infrastructure services (“the platform”) abstracting the infrastructure to empower product teams.

Figure 6 - The platform team provides automated services to be used, with little effort, by developers (the platform API mediates the interaction between developers and the platform team). The platform team and developers are open to hear and support each other.

Core properties:

• The existence of a delivery platform enables the product team to operate their own business services in production and minimizes the need for infrastructure specialists within the product team.
• Although the product team becomes fully responsible for NFRs of its services, it is not a significant burden since the platform abstracts away the underlying infrastructure and handles several NFR concerns.
• The product team is the first one to be called when there is an incident; the infrastructure people are escalated if the problem is related to some infrastructure service.
• Product teams become decoupled from the members of the platform team.
• Usually, the communication between the development team and the platform team happens when infrastructure members provide consulting for developers or when developers demand new capabilities for the platform.
• The infrastructure team is no more requested for operational tasks.
• The platform may not be enough to deal with particular requirements; usually, the platform is tailored for the typical case of functionality built in the organization.
• Infrastructure specialists possess coding skills.
• Organizations with platform teams present the best outcomes in terms of delivery performance.
• Not for small organizations.

Supplementary properties:

• Cloud façade with specialized API: the organization provides a platform that consumes a public cloud (e.g., AWS or Google Cloud), but in such a way that developers may not be aware of it.
• In-house administered open-source platform: the organization manages an open-source platform, such as Rancher.
• Custom platform: the organization has built its own platform due to the specif needs of the organization.

Single dev/infra department

A single department takes responsibility both for software development and infrastructure management. It is more aligned with the Amazon motto “You built it, you run it,” giving more freedom to the team along with a great deal of responsibility.

Figure 5 - A single department takes care of both development and infrastructure.

Core properties:

• Communication and standardization among cross-functional teams within a single organization must be well managed to avoid duplicated efforts.
• A challenge in forming cross-functional teams is guaranteeing they have members with the necessary skills.
• Not prevalent in large organizations.
• Although this structure is commonly associated to the usage of cloud services, such usage is not obligatory.

Supplementary properties:

• Dedicated infrastructure professionals: the team has at least one infrastructure specialist. It may be the case that the department has an infrastructure team dedicated to a group of developers. In such case, the difference to collaborating departments is the common hierarchy (same boss) to both development and infrastructure people.
• Developers with infra background and attributions: the team has at least one developer with advanced infrastructure knowledge.
• Lightweight infra effort: there is no need for advanced knowledge in infrastructure.

Other supplementary properties

• Enabler team: provides consulting and tools for product teams, but does not own any service or infrastructure in the production environment. Consulting can be, for example, on performance and security. Tools provided by enabler teams include the deployment pipeline, high-availability mechanisms, and monitoring tools. Another type of enabler team are committees to coordinate the work of development and infrastructure groups. We found them in all organizational structures.
• With a platform: the organization possesses a platform able to provide deployment automation, but without following the patterns of human interaction and collaboration described by the core properties of API-mediated departments. This includes organizations with a single cross-functional team building a platform and organizations having siloed-style interactions between developers and the platform team.

To know the details about how we conceived these structures, read the full paper!

Download the full paper here

Summary

Now we briefly provide for each discovered structure: (i) the differentiation between development and infrastructure groups regarding operations activities (deployment, infrastructure setup, and service operation in run-time); and (ii) how these groups interact (integration).

• Segregated dev & infra departments
  • Development differentiation: just builds the application package
  • Infrastructure differentiation: responsible for all operations activities
  • Integration: limited collaboration among the groups
• Collaborating dev & infra departments
  • Development differentiation: participates/collaborates in some operations activities
  • Infrastructure differentiation: responsible for all operations activities
  • Integration: intense collaboration among the groups
• Single dev/infra department
  • Development differentiation: Responsible for all operations activities
  • Infrastructure differentiation: Does not exist
  • Integration: –
• API-mediated dev & infra departments
  • Development differentiation: responsible for all operations activities with the platform support
  • Infrastructure differentiation: provides a platform automating much of the operations activities
  • Integration: interaction happens for specific situations, not on a daily basis

Just a few more considerations

First, if one classifies a specific organization as adhering to one of the above structures, it does not mean that the particular company will necessarily follow all the structure characteristics. Each organization is unique and has some adaptations for its context. Nonetheless, the model can assist reasoning in understanding organizations and in supporting decision-making regarding organizational changes.

For now, we believe our taxonomy provides the following primary benefits:

• Offers concepts to distinguish clearly collaborating departments and single departments. In different scenarios, people use the word DevOps to describe both of them, which can be confusing and disorient organizations wishing to “adopt DevOps.”
• Distinguishes the “API-mediated department” as a separate organizational structure, making clear it has a different set of consequences when compared to collaborating departments or single department.

Another distinctive characteristic of our model is that it is grounded on data retrieved from software professionals in the real world.