A very common question that I see over on Reddit is "How do I do DevOps?" or "How do I get better at DevOps?". The answers to these questions are usually technology focused. I think that the routine answers to this question fail to meet the fundamental understanding of what DevOps really is.
On its face; DevOps blogs, articles, YouTube videos, etc. will classify DevOps as a series of tools that are composed to do some form of infrastructure as code or configuration management. While this is a small component of DevOps, it is not the whole story.
DevOps is more about enabling delivery flow than it is about any singular tool or composition of tools. Basic human nature will tell us that we always want to get more for less, and DevOps is the "branding" behind that idea in technology. I have more to come on this topic down below.
With this basic understanding, I want to get into the meat of the issue first.
- How to gain experience with DevOps
- DevOps Deployments: Fast, Easy, and Fun!
- Mastering DevOps: From Outcomes to Craftsmanship
What is DevOps?
DevOps is a cultural shift from development and operations teams working in silos into a single cohesive team focused on delivering value to the market in a consistent and high-quality way.
The DevOps model brings a series of benefits to any business looking to make the jump. Those benefits center around:
DevOps teams embrace agile development as their core delivery methodology.
There is also a DevOps culture to embrace. The characteristics of a well-developed and functioning DevOps culture are as follows:
Fostering a collaborative environment, Impose End-to-End Responsibility, Encourage Continuous Improvement, Automate (Almost) Everything, Focus on the Customer's Needs, Embrace Failure and Learn From it, Minimization of Waste
As you can see, neither of these lists is primarily technology focused. They are far more experienced-based than processes-based characteristics. When attempting to implement DevOps, integrate DevOps team members, or practice DevOps; remember that the experience you are creating is as important as the technology solutions you are putting you into the market.
▶ Key Insight
Think of Agile and DevOps as a social contract between all parties involved in the delivery of a solution. Everyone in the pact has a baseline understanding of the expectations through training and experience. From there, the team is able to self govern and self manage their way to success based off this foundational common understanding.
Without Agile and DevOps, teams are contending with their personal experience, personal background, and individual expectations trying to put the world on their backs to move mountains.
While we want everyone to bring their personal skills and expertise to the table, constantly trying to figure out how to get individuals to work together seamlessly poses almost no value to a business or the market.
What is the Difference Between DevOps and Traditional Software Development?
By now, most people have a sense of how DevOps works differently than typical development methods. This document has differences on the basis of its concepts, mechanism, and operation. DevOps is highly efficient at integrating technology with the development of new technologies. Traditionally, developers look to test while operations prepare documents for the testing process. The operations team doesn't really know what's going on, they develop and monitor the plan themselves.
In a more traditional approach, software deployments are going to be a large and grandiose event. Multiple individuals from different teams will be pulled together to spectate or participate in the event. Project Management will ensure that time is carved out in schedules to accommodate pre-planning, executing, and postmortem activities to fully capture every facet of what happened on launch day.
In my experience, if you are asking the question "How do I accelerate development?" then you are probably using a more traditional development approach.
"Traditional" software development will fall into one of two categories: Waterfall or IC (Implementation Chaos).
Waterfall: hen working in waterfall, perpetual checkpoints are introduced into delivery called Milestones. Everyone on the project is rushing to deliver against Milestones regardless of the complexity of their project. There are nearly and infinite number of ways to deliver any of these milestones and rarely does anyone on the team really own anything substantial. The first thing to suffer here is quality.
IC: Implementation Chaos is even more interesting because there are no rules or frameworks, just the sheer force of will to get a project completed. I am using the term "interesting" very loosely here. IC is the equivalent of that hell on earth project that a lot of us have either participated in or heard about from our colleagues.
DevOps tries to solve all of this by engraining a few simple principles into the mix. Use feature flags, always be ready for production, automate everything, and iterate quickly. This cuts infinity down to something more manageable and gives some clearer direction for technology teams to work from.
How Do I Practice DevOps?
So, this is a bit awkward. Remember how I started this by saying that technology is not the answer for "How to practice DevOps?" That is still the case, but I do think that you need a solid technology foundation before diving into DevOps. This is not necessarily true for project management roles, but if you are in a technology role, this section is a good basis to build upon.
As noted above, DevOps engineers follow a set of characteristics. It will be very difficult to build on those characteristics if you are not enabling yourself to investigate them technologically. So, follow along as we dive into a bit of the DevOps process before we discuss really practicing DevOps for reals.
DevOps ensures that the flow of value (generally custom software) is enhanced to be faster, more reliable, higher quality, and continuously deployable. Knowing that the technology space is a vast expanse with nearly infinite ways to accomplish this, what can an aspiring DevOps engineer do to break into this space? I think the simplest answer is to do your research on delivery patterns more than specific technologies.
▶ Key Insight
Practicing DevOps is going to be different for technical and non-technical roles:
Practicing DevOps is going to mainly be around understanding and building empathy around what the technical team members are trying to achieve. Build skills in supporting their efforts and balancing external forces like management and customers. Some people will take this too far into a purely defensive posture to defend their technical team members, but that is the wrong attitude. The right place to live is constantly applying critical thinking to smooth things out in a way that everyone wins.
I believe there are some obvious things that someone can do to embrace DevOps such as learning popular automation tools like Ansible, Chef, GitHub Actions, and others. Additionally, technical roles should spend time building empathy with their non-technical counterparts. The rest of this article is primarily technology focused so follow along down below.
Deliver Software for Humans
Technology delivery is done by humans with preconceived ideas about what is or is not important when it comes to things like quality delivery. This means that there is no single "Best" or "Right" way to do DevOps.
Any technology that is put in place must work for the people who are consuming it. Think about it similarly to front-end design on a website. I personally don't really care for flashy front-end designs which are loaded to the hilt with highly stylized graphics. I enjoy a much cleaner and minimalist aesthetic. This means that I am going to gravitate toward websites that present a more clean aesthetic to me and I will generally abandon a website that does not.
You can think about how you accomplish a DevOps flow in exactly the same way. Something which is overly complex with too many rules or considerations may not play well with the consumers of your pipeline where it may be just what the doctor ordered with other teams. This is why understanding and researching delivery patterns are so important to start with.
Find your Passion
The next step is to identify a use case that you are passionate about. Spend some time looking at a variety of different automation tools that can help take the identified use case and get it into a continuous integration and continuous delivery model.
Utilizing a service to kickstart your effort, like GitHub pages, is a really good starting point. Open up your favorite editor or pull down some templates to create a simple static website. Keep it really basic and talk about yourself and your skills as the primary content. Follow the GitHub pages Quickstart guide to get your site launched.
This will help us understand the fundamental flow of releasing a piece of software out into the wild. This will also help build some baseline skills around version control systems.
Software Delivery Process
Once you have a technology project that is yours, start to add some flair to your pages. Integrate a build process that starts to optimize your content to improve metrics like Google PageSpeed. Here are some example activities you could perform:
- Push your images to optimizers like Shortpixel to boost
- Inject tracking code like Google Analytics to your website
All of these activities require some kind of intermediate step that can have a positive impact on your delivery flow. But, like any software development project, they can also introduce bugs that will have a negative impact. By working through these scenarios, you will build an understanding of the soft edges of software development techniques and gain more skills as a DevOps engineer.
Make a Pivot
Once you have all of that in place, think about what you are doing from a market competitiveness standpoint and take a hard pivot. GitHub pages may be an elegant starting point, but there is not a huge swath of companies looking for specific experience there.
The industry is going to either be hosting its content on servers or containers. Since you already have a continuous delivery pipeline setup, it is probably time to start looking at continuous integration. Containers are a great way to practice the skill of building integrations between technologies.
Start by installing a tool like Docker onto your workstation. From there, pull down a container image from Docker Hub. Add a Dockerfile to your code repo which will be used to run a container build. Once packaged, you should be able to run your container and see your website running live right from your computer. This is also the foundation of learning and understanding configuration management techniques.
Develop Automated Testing
The previous steps have taken you from ground zero to a continuous integration pipeline running directly on your workstation. Pretty cool right? The problem is, the steps have only covered software development teams and infrastructure teams but we have not discussed quality or quality assurance teams yet.
I would be willing to make an assumption that the website you have running in your container right now does not look nearly as good as what is running out on GitHub pages. Utilizing tools built right into the web browser like "Web Developer Tools", you may find some interesting 404s being introduced into your site that used to just work.
Part of any good continuous integration pipeline is automated testing. Any form of testing can be scary at first to anyone who has not done it in the past. You can start off pretty simply either by picking up some automated tools or by simply writing bash scripts with curl to validate the assets on your website are loading as expected.
If you really want to get fancy here, you could run a Selenium in a second Docker container to supercharge your testing efforts. Regardless of the tool, you should be focused on building an automated process that you can ultimately trust to work and provide real results. Work on quelling some of the initial noise and false positives.
Dabble in Some Software Development
This website (https://www.valewood.org) is a combination of a few different technologies. I start by doing WYSIWYG editing in WordPress. From there, I am using a plugin to generate static content that can be pushed to a GitHub Repository.
From there, a few build pipeline steps need to happen for this site to finally make its way to Cloudflare Pages. One of those steps is to build a search index. Following a software development process, I am able to offer enhanced functionality without the need for expensive backend servers.
Google itself is a HUGE search index running on tens of thousands of servers, so how does this website do it? It actually isn't that complicated, and I am using tools that are already in the ecosystems discussed earlier in this article.
In the linked repository, there is a script, build_index.mjs, which is fired off as part of build.sh. That script is a NodeJS script that is responsible for finding all of the static articles on the site, reading them, and creating search_index.js and search_previews.js files at the root of the website. Those files can then be served up to Lunr.js on a custom-built landing SERP page effectively giving my website a micro Google.
By integrating software development processes into my project, and while still hosting a 100% static blog site, I am able to offer some rich functionality without the cost, operations teams and security teams/risks that hosting on an EC2 might bring with it.
▶ Key InsightI believe that most of the people who are going to land here are probably more on the infrastructure delivery side of the house than the development side of the house. I would encourage anyone who is purely infrastructure focused spend a bit of time in the development space learning how to do a bit of programming. Learning to walk in someone elses shoes does a world of good in building perspective and growing overall as a technologist. This could be something as simple as writing some HTML all the way through learning some PHP and writing a wordpress plugin.
Find a New Hosting Provider
As mentioned before, there is not a huge market for DevOps engineers who are really great at GitHub pages. This means that you will need some exposure to other hosting providers. This is where this article is going to get a little bit tricky because everything mentioned above has been free. You could technically host your containerized website application from your home internet if you would like, but I feel that defeats the point of getting relevant exposure and practice.
DigitalOcean can be a really good and cheap hosting provider for anyone just starting out. You can get a virtual machine for about the same price as a cup of coffee. Start by pushing your container to DockerHub. From there, log in to your virtual machine from DigitalOcean and install docker. Pull your image and turn it on. Depending on how things are set up, you may need to either open a firewall port or do some port forwarding, but you should have a running and fully tested website ready for the world to consume.
If you really want to flex, DigitalOcean also offers Kubernetes hosting for roughly the cost of a foot long from Subway per month. More and more of the market is ending up with Kubernetes as a great way to orchestrate both infrastructure and code changes under a single definable schema.
Continuous Delivery (CI/CD)
To really tie all of this into a nice neat bow, the next step should center around continuous integration and continuous delivery and triggered actions. Your code lives in GitHub, your website lives in DigitalOcean, and you build containers on your workstation. Wouldn't it be great to cut your workstation out of the mix? You are really not that far away from achieving that.
An emerging trend in the industry is called GitOps. Again, you can utilize automation tools for this, but let's start really simple. DigitalOcean has a really great guide on how to set up this deployment process in detail.
Any DevOps engineer worth their weight is going to ensure that their solutions are monitored and available. Right now, the risk is pretty low that your static website running on a containerized web server is going to have an availability concern, but it is good to get practice in this space.
To get into the practice of utilizing DevOps tool to set up automated monitoring, there are some great services out there that either have free trials or are very inexpensive. On my personal projects, I have integrated into Updown.io‘s API to automated testing of my sites availability. I have then used cron jobs to hit the API and pull back downtime information. It is not the most elegant solution in the world, but it has helped solidify some of the concepts around monitoring.
If you are willing to spend the money, services like Pingdom are also great. They will give you continuous performance monitoring of your site.
Peers are a Great Resource
Lastly, don't be afraid to get a peer review from someone. Because DevOps is about flow, assume that your way of thinking about a problem is NOT the best way to approach it. By getting a peer review you are getting a more global view of what is or is not important to potential consumers of your delivery flow. Their feedback may be around adding features, reducing complexity, or pointers to more standard ways to approach a problem. All of that is good and should not be taken personally.
Flow of delivery
So, what is the flow of delivery? The flow of delivery is the idea that you should be able to quantify all of the steps from the second a task is started through the time that thing is released for consumption by the market. In software, this generally means The time it takes from when Tom Smykowski receives a request to the time in which it is available to the customer.
In the old world where a business would follow a slow monolithic software development practice, a development team would have any number of double-digit touchpoints all leading to a monolithic release of software going out for consumption. Each of those touch points continues to increase the cost and reduce the agility of the specific work product. This is unavoidable if you are not interested in optimizing that SDLC process.
Once your traditional software development process was completed, there would then be a handoff between the development and operations teams. Operations teams are responsible for running the software once delivered into the wild. This can be an extremely daunting ask.
In a DevOps delivery model, Tom should be able to bring in a request that is then broken down into smaller units fitting a continuous deployment model which are able to be deployed to production once committed to version control. This generally leads to a shift in overall spending from individual touchpoints over to automated testing. Customers can get their value much sooner by having smaller portions of their solution delivered to them in real-time. They are also able to provide feedback which makes the inevitable pivot much more achievable.
Delivery teams are able to shift from a traditional software development practice over to a continuous development cycle which provides value to the market much more quickly and reliably. When practicing devops, this is what you should really be focusing on.
Is DevOps Easy to Learn?
This question implies that there is a potential mastery of DevOps. I believe that the idea of DevOps is evolving and changing rapidly which does make it a bit of a moving target. You can compartmentalize DevOps into some small chunks like building a simple release pipeline which does make DevOps very easy to learn.
A different way to compartmentalize DevOps is simply trying to master the DevOps Lifecycle. This mastery is looking for ways to ensure that is software runs through continuous integration and continuous delivery models along with ensuring that all dependencies are up to date or being decommissioned at an appropriate pace. Learning this skill will also help you integrate into a DevOps team in the future.
I also believe that DevOps takes a mindset shift from focusing on getting a task done to focusing on getting something released. More time being spent on breaking work down into chunks and accomplishing delivering a work product through that breakdown and finally releasing that work product into production to to receive feedback and criticisms all ladders up to meeting a high standard of quality.
If you are focused on answering the question "how does technology X do a thing", there is room for that but, your DevOps journey is going to take a bit longer. Spend your time asking questions like:
- How can I get better at integrating code changes?
- How can I provide better feedback loops for successes and failures with technology?
- How can I automate processes which are labor intensive?
- How can I manage infrastructure as code?
- How can I integrate everything I am doing into a version control system instead of manual action?
- How can I utilize processes from DevOps to enable an engineering team of my peers?
Once you start to understand the answers to the aforementioned questions, you can start to expand even further into aspects like the vast array of tools that the CNCF has to offer. Each of those tools is trying to answer similar questions.
I believe that the simplest jumping-off point on a DevOps journey is to build some simple deployment pipelines from something like GitHub and GitHub Pages or Cloudflare Pages utilizing some scripting and git-flow to get used to the nuances that are required when diving more fully into DevOps. Once you have your baseline DevOps practice environment setup, you can then pivot into building good habits that enable customer satisfaction through quality, flow, and automation.
How do you practice DevOps? First, read about traditional SDLC flows and read about DevOps best practices. Understand the differences and why the market must modernize to stay competitive. Next, get interested in enabling the flow through continuous integration and continuous delivery while practicing delivering your own technology in some simple use cases. Next, get some peer reviews to help grow your global perspective. Finally, start to build in a DevOps lifecycle into your project!
All of this will lead you to understand the DevOps philosophy from a technological point of view. Start to review the culture of DevOps, DevOps strategy, DevOps transformation, and the rest of the non-technical aspects of DevOps to become very well-rounded in this space.
Below are some additional resources which help understand the fundamentals of an SLDC and automated delivery: