There are a lot of articles out on the internet that tries to explain what the DevOps methodology is, and why you should use it. I believe that most of them will only get you a fraction of the way to understanding the DevOps methodology, why you should use it, and how you should transform your thinking to realize its benefits.
When doing research I have seen everything from "The Definitive Gide to DevOps" to "The Ultimate DevOps Handbook" attempting to describe DevOps in its entirety but stopping short of a 360-degree viewpoint.
Please, follow along with me as I try to remove some of the disillusionment and misuse of the term DevOps all while trying to bring clarity to the intent and characteristics of a methodology that has tremendous upside when applied properly.
Definition of DevOps
We should spend a little bit of time talking about what DevOps is not to help bring some additional clarity to the definition once we get there. First and foremost, DevOps is not a buzzword that does not have any formal meaning behind it.
DevOps is a full blown methodology that has real meaning, not just another word thrown into a generic business-centric buzzword soup. At its core, the generic definition of "DevOps is a the merging of Software Development (Dev) and Operations (Ops) teams into a single functional team" does fit at the 100k foot level, but it really does not do justice to the real intent.
What DevOps Is Not!
DevOps is not a replacement for rigorous planning, execution, and diligence in releasing software. While doing research, one of the major things that kept popping up was job postings. You can tell a lot about a company and its expectations from how a job is posted.
At the time of writing, LinkedIn had over 44,000+ job postings with the term DevOps in them. Indeed had nearly 11,000+ job postings. A common thread amongst them was the extensive skill set the market is demanding for Junior DevOps Engineers and how disproportionate that is against the more Senior DevOps Architect roles. This is a clear sign that DevOps is not being applied appropriately at these companies and they are trying to skip rigorous planning, execution, and diligence while looking for individual contributors that can come in and plug those gaps.
The biggest red flags are things like:
- Junior DevOps Engineer jobs needing extensive Kubernetes experience
- Senior DevOps Architects with hyper-specific AWS experience and certifications
- Any posting looking for a Full Stack Developer with the word DevOps in the job posting
- Extensive experience with 15 or more technologies in a 4 year time-span
- Broad experience with a full range of DevOps-related technologies paired with extensive experience with a singular niche technology
- Job postings that do not mention anything about flow, monitoring, observability, or agile software development
With the examples provided above, it is clear that the hiring managers are being asked to go through a digital transformation by the leaders of their division or company and assume that they will be able to bring in talent to fill the gaps instead of ensuring they educate themselves before trying to jump in feet first.
DevOps is also not a shim between an old way of delivering monolithic technology and more modern delivery approaches. Sure, we may choose to slowly step our way into an overall adoption, but that transition should be well planned and executed to make sure the transition does not get stopped in the middle.
Most of the feedback online about failure in the adoption of DevOps practices will center around it being ineffective and cumbersome. This will happen when it is not fully adopted by both technology groups and the non-technology groups of a business.
An ice cream shop is not going to go through the effort of adding a conveyor belt to their ice cream cone serving process only to never turn it on and let ice cream cones get stacked up at the scooping station when they should be making it to the cash register.
When thinking of DevOps, most people will jump directly to automating infrastructure and will stop there. Infrastructure as Code is only the start of a DevOps journey. It is generally low-hanging fruit that can give you a quick bump in productivity by bringing some consistency and reliability to the equation.
If your organization is not looking forward to streamlining IT operations, application development, application performance monitoring, continuous delivery, static code analysis integration, along with any of the other many facets of DevOps; then you are not really reaping the benefits of a full practice.
Because of the nuances of DevOps and DevOps Practices, we need a couple of parallel definitions with different focus areas to really disambiguate what DevOps really is.
Methodology-Focused Definition of DevOps
DevOps is a method of software delivery lifecycle management that focuses on enhancing delivery flow for complex software ecosystems. This methodology centers around planning and communication routines and strategic planning in which everyone participates, an organization being willing to take small risks to test out market conditions and pivot when they do not work out, and finally ensuring expectations of the definition of success are aligned throughout an organization.
Technology-Focused Definition of DevOps
DevOps is a method of software delivery lifecycle management that brings the practices of Software Development (Dev) and Operations (Ops) together under a single team to promote; quality, speed, stability, and security. Work will be moved from large monolithic releases to small incremental updates, changes, enhancements, and bug fixes that align under clearly communicated future strategic planning. Delivery of work will move from manual deployments into more automated hands-off deployments.
Why DevOps and Agile are like Peanut Butter and Chocolate?
One of the primary principles of DevOps is the ability rapidly iterate, learn, adjust, and iterate again. When trying to deliver in a more waterfall / monolithic pattern, projects will try to peer into their crystal balls to build buffer time into their project plan to account for failures or issues. After the buffer time has been used up, the scope starts to get cut to meet a deadline. When the scope is cut, you are left with a sub-standard product.
Suppose you flip the delivery method on its head from large and monolithic to lean and agile, as you execute your plan. In that case, you are able to absorb new information and adjust expectations accordingly. Scope no longer needs to get cut in order to meet delivery deadlines. The technological principles of DevOps lend themselves very well to this agile approach. Since DevOps technology teams are focused on being lean and observable, these micro pivots are much easier to absorb while still moving a project forward.
Business expectations defining what delivery and success mean can be at odds with DevOps success. Business leaders expect that customer and partner expectations are being met while agile teams that are high functioning care about ensuring that a consistent and reliable pace is set to ensure quality products are delivered. They also care about the consistent flow of value being produced more quickly.
When a business team puts pressure on a technology team to release a full product faster, you will generally see quality suffer and you will see more adverse impacts on future planning and initiatives. When taking a more agile approach to delivery, quality issues that show up can more quickly be remedied because the methodical movement of bite-sized work is much easier to understand and pivot around.
Lastly, the idea of market value is not lost on a technology team. I have witnessed far too much of a direct customer/partner influence dictating conditions of success that are unobtainable. This is generally due to conflicting messaging, information, and expectations because these two parallel teams are not speaking on the same terms. By prescribing fully to the agile methodology, meaning the business entities selling products align fully with technology teams delivering products, a full story can be developed to ensure that customers/partners are delighted by the value being delivered.
What Business Problems is DevOps Solving?
Now that we have a definition of DevOps through a few different lenses, and we understand what DevOps is not trying to define itself as let's directly tackle the business problems that DevOps is trying to solve.
DevOps is Trying to Solve the Consistent Flow of Value
Businesses everywhere struggle with trying to understand when they are going to start seeing a return on their investments. This is generally realized once a product hits the market and it starts to gain traction. In a more traditional methodology of building and delivering software that value may not be realized until well after a product hits the market simply due to bugs and quality issues.
DevOps adoption helps correct this by focusing on flow. When paired with agile, value is consistently being pushed to production on a daily basis, but it is gated by feature flags. This allows for real-time feedback to enter the system and get back to responsible parties much more quickly.
For example; An Agile team pushes a small bit of incremental code for a new product line through the DevOps assembly line and it gets deployed to production. At the same time, a partner manager is meeting with a group that is interested in this new product line. Feedback is provided that this group finds some of the language on the new product line confusing. This issue could be fixed very quickly instead of waiting for the next major release to be completed. Everybody wins!
Log4J is another example of impacted delivery flow. Java has deep roots in technology produced by big businesses. When Log4J, the de facto standard for logging in Java frameworks, was found to be compromisable teams knew that they needed to update their tech stacks to avoid damage to the companies that they work for. If a company was on a monthly or worse quarterly release cycle using more traditional delivery methodologies, they could be sitting out there exposed until their next release. A team employing DevOps tools would need to update their library version, let continuous testing tell them if that broke anything, and finally let automation push their updated code to production. If set up correctly, the DevOps toolchain could be fast enough to push new code within minutes of the CVE being released. Worse yet, for teams using more traditional delivery methodologies, there were several updates that needed to be applied and then re-certified through all of the labor-intensive manual quality gates.
A team delivering against the DevOps model is going to enhance a business's flow of value simply by utilizing the core principles of automated testing, repeatability, faster resolution of bugs and issues, and minimizing rework. A business is also going to reap the benefits of an enhanced security posture by ensuring that changes to remediate vulnerabilities are low-impact and very well-tested.
Solving for Visibility and Communication
How does a team know if they are producing the value that the market wants? How does a business get reassurance that the team is working effectively? How does a business know if they will get MORE value if they scale up or scale out the work being delivered?
DevOps can answer all of these questions when implemented appropriately. When implementing a DevOps approach, a business is going to get an immediate uptick in productivity because two silos were just broken down. Combining software development and operations teams together will inherently increase the flow of bi-directional communication. But that combination of teams is just a jumping-off point.
A team delivering against the DevOps model should be responsible for providing more visibility than a traditional monitoring view of the world would provide. A more traditional view would be looking for the up/down state of a service or website. DevOps teams should be measuring their rate of flow, providing visibility and communication into that flow, and finally should be reporting on anything that may be impacting that flow.
If there are a slew of production-related issues which is now distracting the team from delivering against planned targets, the team must speak up and the business around them should help clear roadblocks to get them back on track again. Sometimes this means re-prioritizing work, sometimes this means investing in stability, and sometimes this means investing in more automated quality gates.
Since a DevOps team will focus on providing visibility and communication, any issues like the aforementioned ones will surface quickly and it is up to the support structure around that team to jump into action. If that visibility and communication are ignored or not actioned on quickly enough, large swaths of time and value may be lost.
Solving for Responsibility
A DevOps team is one team with one goal. As I mentioned earlier, an immediate benefit to DevOps is breaking down communication silos. A secondary benefit is the creation of a team that is specifically responsible for a product or portion of a product. By moving all of that work back into a single team, they will focus on quality delivery because it is their responsibility to deliver quality. Nobody else is going to be waking up in the middle of the night for production issues anymore. There is nobody else to shift blame to when something isn't working as expected.
A self-organizing team is going to also spread information and knowledge around much more effectively. Adult learning patterns are a pretty mixed bag, and it is tough for an individual contributor to disseminate information to 500 of their colleagues, but is much easier when that team is sized to be fed by 2 large pizzas.
There will also be equality in the prioritization of work. It is fairly easy to deprioritize work that is not in high interest when that work could greatly benefit someone downstream. When a singular team is responsible for delivering a flow of value, they will be much more likely to prioritize improvements over individual contributor self-interest.
Solving for Operational Inefficiencies
Handoffs are costly to a business. It may feel like the right decision to stand up a group to build something and have a second group stand up to run something once built. When trying to be as operationally efficient as possible, handoffs are not the way to accomplish it.
By merging software development and operations teams together into a single set of DevOps practices, a business can gain operational efficiencies immediately. An operations team is not suffering through lingering bugs that may have a quick resolution. Software development teams are not waiting for operational activities to be performed against their projects which need external resourcing.
What are the Business Benefits?
When evaluating if DevOps is the right model for your next software development project, understanding the benefits of DevOps to your business outcomes should be at the top of the list. Adopting DevOps should not be taken lightly, but it has numerous upsides. A few of the most notable ones are listed below:
- Flow: When paired with agile software development, DevOps processes will bring a more consistent and reliable flow of value to your business. Breaking work down into smaller chunks that can be released more quickly and more frequently allows a business to realize value from its investments much more quickly.
- Quality: DevOps processes do not eliminate bugs, rework, or outages. DevOps needs people, and people are not perfect. Through continuous integration and continuous deployment processes, a team running DevOps methodologies will be able to catch and remediate issues much more quickly than a team running a slower and larger release cadence.
- Flexibility: When running an agile development process, pivots will need to be made when direction changes. The use of configuration management tools along with the use of automated testing tools allows for quick pivots by being able to quickly either remove or roll back changes that were done as part of experimentation.
- Release faster and work smarter: I once heard a development leader I was working for say; "If you are proud of your code the first time you release to production, you waited too long". I didn't really understand it at the time, but it really makes a lot of sense when you look at agile teams and DevOps teams. Because of their continuous delivery nature, they are able to move lots of work into production quickly, learn, iterate, and repeat. A business gets a lot of value from this because its customers and partners can give feedback much more quickly.
- Simplifies Software Development and Operations Focus: By breaking work down into smaller user stories, a team moves with greater intent and focus on their work. By managing the work instead of the method, people get time-sliced and context switch which is detrimental to quality.
- Responsibility: By tearing down the silos and building a singular united DevOps team, or DevOps teams if your project is large enough, a business can build specific product responsibilities. Adopting DevOps is about the enablement of people, and businesses need to enable their people to both produce value, but also be responsible for that value.
- Consistency – Due to the nature of continuous integration and continuous delivery, a business gets consistency with DevOps practices. Any code that is going to make it out to production is going to go through these pipelines.
- Measurement: It is extremely tough for any business to build real actionable measurements without some kind of consistency. DevOps teams are focused on consistency to support continuous integration and continuous delivery. Through this consistency, measurements can be achieved which provide real actionable insights into what is going on with product development teams.
- Security: As discussed previously, handoffs are costly. Security teams are even more expensive than handoffs. Today, tools like SAST and DAST scanning can take care of just about every baseline attack vector an application will have. By integrating those tools into a continuous delivery pipeline, a business will get a shorter feedback loop for defects and will get quicker remediation on those issues. Security teams can then go and focus on the high-value work that their salary demands.
- Shifting Left: A singular integrated team working together to provide value will consist of many different practices including Software Development, Quality Assurance, Operations, Infrastructure, etc. By getting all of these disciplines onto the same team, you can improve software quality by shifting left much earlier in the project. By getting a focus on quality and testing early, nasty defects which may sink a project can generally be caught much earlier in development.
What are the Characteristics of a Healthy DevOps Culture?
A culture that is living the DevOps lifecycle is going to exhibit a few specific characteristics. These characteristics are going to look a little bit different than what you would traditionally see a Development team do. Here are a few of those characteristics:
- Foster a Collaborative Environment: Collaboration is at the heart and soul of DevOps culture. To really improve a development process and transform it into one that embraced continuous integration and continuous deployment, everyone needs to be communicating and collaborating on the same shared goal. Better collaboration and communication will lead to greater trust in your teams.
- Impose End-to-End Responsibility: DevOps focuses on lowering the number of handoffs that need to happen in the continuous delivery pipeline. Everyone is responsible for ensuring they deliver quality, and speaking up when quality is being sacrificed in the face of other challenges.
- Encourage Continuous Improvement: In order to reach success with DevOps, a team may review the same bugs, features, requests, or updates multiple times to improve the outcomes of their changes.
- Automate (Almost) Everything: Automated processes are at the center of the DevOps technology stack. To support continuous integration and continuous delivery processes, a DevOps team needs to ensure that any work being submitted can make its way quickly to a production environment where observability can start to shorten feedback loops. This automation will start at infrastructure as code, to configuration management, through deployments, and rollback processes.
- Focus on the Customer's Needs: DevOps relies on feedback so that micro pivots can be made to ensure that the market is getting what it needs. The debates about pure technological efficacy can be put to the side knowing that the core goal is customer delight.
- Embrace Failure, and Learn From It: Big bang/all or nothing deployments usually condemn mistakes, bugs, or performance issues as failures. A DevOps culture will embrace failure as a chance to learn and grow. Since work has changed from large monolithic requests to small bite-sized chunks of work, any "failures" can be learned from creating a more healthy and happy working environment. I have heard DevOps engineers say; "A failure is just another chance for more continuous testing!"
- Minimization of Waste: When trying to stay lean and support continuous integration and continuous deployment process, there is too much room for waste in the system. A healthy DevOps culture is going to see waste for what it is and do its best to eliminate it. This allows a DevOps team to stay lean and focus on delivery flow.
What does DevOps do for My Role?
This is the ultimate question that we are all trying to answer when reviewing published articles like this one. I think we have enough information at this point to break DevOps processes and DevOps methodologies down into something tangible per business role.
When operations teams are integrated into a functioning DevOps agile team, they should really start by embracing DevOps workflows. By utilizing DevOps tools like version control integrated monitoring tools, software development lifecycle, and others; they are no longer the recipient of work once completed. They should be shifting left in the continuous delivery cycle to integrate monitoring and reporting as early in the cycle as possible. By creating a DevOps report of the environment health and performance, earlier feedback can be given to the team about issues and bugs as they are uncovered. Other roles on the team are also able to now contribute to a more stable operations practice because all of that tooling lives in the same codebase under version control.
Getting environmental configurations into version control is critical for DevOps. With the proliferation of APIs in cloud computing, making configuration changes is easier than ever. Exposing those configurations through configuration management in a software project means that a broader team can bring their expertise to the project which generally increases quality. Infrastructure teams are well versed in the soft edges and best practices of infrastructure components, and they can lend that expertise to a project to also increase stability and security.
Software Development Teams
DevOps adoption for software development teams can be a bit tricky especially when converting software from a monolith delivery approach. One of the first steps should be to ensure automated tools are utilized to perform testing, both functional and unit testing, to ensure that quality is not impacted. This may be a bit of a stretch for the development team since they are going to need to extend functional testing to infrastructure, not just the application code. Next the team is going to need to adjust their software development approach by starting to break up their application into more defined domains and responsibilities. From there they can increase collaboration and you will find that individuals in other roles will start to contribute to the software project.
Quality Assurance Teams
Quality efforts will start to transition from manual testing in a specific testing environment into a method of continuous testing on every commit. Testing can be one of the most expensive parts of an application to ensure that regressions and new bugs are not introduced, but computers are much better and much more accurate at performing these repetitive tasks than humans are.
Product Management and Product Owners
Integrating directly into DevOps teams for these roles is a bit more difficult but is very necessary to promote a healthy DevOps culture. To understand the full picture of flow and continuous delivery, the Product team will need to live the software development lifecycle with the broader technology team. This will allow for bi-directional expectation setting to occur. A DevOps engineer will have a direct line of communication with the Product team around challenges and road-blocks which will help inform continuous micro pivots to clear those challenges and road-blocks. Product teams will also be able to inform the technology team about the continuously changing market conditions and help educate the team on what success looks like.
Account Management, Business Development, and Client Services Teams
Account Management, Business Development, and Client Services teams will need to embrace a more agile software development management style. Generally agile is looking to sell the cars that are on the lot. Brining in signed work to be completed by a certain date is not necessarily viable anymore. Since the software development practice is changing to something which reflects more prioritization of global effort instead of prioritizing specific initiatives, those new requests will need to enter the prioritized flow. Once completed, instead of big ceremonious launches, customers can be integrated into the DevOps lifecycle to start testing much earlier in the process. The end result is a well formed, well executed product being turned on in production instead of the dopamine releasing activity of testing code in production for the first time at launch.
Scaling DevOps can be challenging, especially when DevOps teams are just starting out. By looking for consistency and operational efficiency through technology instead of brute force labor, Executive Sponsors will see improved collaboration which in turn reduces waste and increases production of value. Consistency of this approach will also make a business far more measurable. When supporting a DevOps lifecycle for new services, an Executive Sponsor should also be in a position to invest in their teams with an expectation of value hitting the market more quickly than previously seen. The Executive sponsor should enable the team to automate processes, utilize agile application development practices, employ site reliability engineering, integrate IT operations, and utilize continuous testing to achieve the ultimate goal of delivering quality products that the market finds valuable.
Frequently Asked Questions
Below are some frequently asked questions about the DevOps methodology:
What is DevOps?
DevOps is a set of practices and philosophies that aims to improve collaboration between development and operations teams in order to deliver software faster and more reliably. DevOps emphasizes automation, continuous integration, continuous delivery, and other practices that streamline the software development process.
What is the purpose of DevOps?
The main purpose of DevOps is to improve the speed, quality, and reliability of software development and deployment. DevOps aims to bridge the gap between development and operations by fostering collaboration, automating processes, and using agile methodologies.
How does DevOps improve software development?
DevOps improves software development by streamlining the process and reducing the time it takes to deliver software to users. DevOps practices such as continuous integration and continuous delivery allow developers to quickly and easily integrate and deploy code changes, which helps reduce the risk of errors and improve the overall quality of the software.
What are the key principles of DevOps?
Some of the key principles of DevOps include automation, collaboration, continuous integration and delivery, and a focus on measurement and improvement. DevOps emphasizes the use of tools and technologies that facilitate these principles in order to improve the speed and reliability of software development.
What are the core tools of DevOps?
Some of the core tools of DevOps include version control systems (such as Git), continuous integration and delivery platforms (such as Jenkins), containerization platforms (such as Docker), and configuration management tools (such as Puppet or Ansible). These tools help automate and streamline the software development process.
How does DevOps foster collaboration between development and operations teams?
DevOps promotes collaboration between development and operations teams by breaking down silos and encouraging cross-functional communication and cooperation. DevOps practices such as agile methodologies, pair programming, and shared ownership of code help facilitate collaboration and improve teamwork.
What are the benefits of adopting a DevOps culture?
Some of the benefits of adopting a DevOps culture include faster software delivery, improved collaboration and communication, reduced risk of errors, and increased efficiency and productivity. DevOps can help organizations stay competitive by enabling them to quickly and reliably deliver new software features and updates.
How does DevOps integrate with agile methodologies?
DevOps and agile methodologies both emphasize a focus on rapid iteration and continuous improvement. DevOps practices such as continuous integration and delivery can help support agile development by allowing developers to quickly and easily integrate and deploy code changes.
How does DevOps support continuous integration, continuous delivery, and continuous deployment?
DevOps practices such as continuous integration and delivery facilitate the rapid and reliable integration and deployment of code changes. Continuous integration involves automatically integrating code changes into a shared repository, while continuous delivery involves automating the process of testing and releasing code changes. Continuous deployment takes this a step further by automatically deploying code changes to production as soon as they are ready.
We have explored the answer to the question "What is DevOps?". We have illustrated or discussed:
- How DevOps can impact the software development lifecycle in a positive way.
- A DevOps culture and how your teams will transform when embracing that culture.
- How a DevOps approach is different from a more traditional monolithic delivery approach would view flow.
- How a standardized test process can help improve security and stability.
- We have discussed how CI/CD can help manage infrastructure, development processes, quality, and expectations to obtain a competitive advantage in the market.
- We have discussed how important DevOps research is and which practices are important DevOps practices
I hope this article gives you better insight into what DevOps really means, and how the DevOps movement can help drive improvement in your business outcomes.