Over the course of the last year, Promet has been working hard to increase our agility. If you've been in any software circles during the last couple of years you probably have only heard the term agile. If you're anything like me you actually probably tired of hearing the word agile.
Before we can really discuss what agile is I think it's important to have some background about why agile is important and why you should care.
Almost all organizations start with a project methodology known as the waterfall. The waterfall is a concept that Winston W Royce introduced in 1970. Dr. Royce worked with NASA in developing software for the space program. Simply put waterfall says that in a development process that there are individually definable steps that take place in a given order. Discovery comes before design. The design comes before development, development comes before delivery. Simple? Is it the way it always works right?
In the very same document that Dr. Royce introduced waterfall, he also says that it's a flawed model because what happens is that as we are designing something or as we are developing something, we start learning things about what we're doing that causes to go back and revisit earlier steps in the process. Simply put what he says is that when you start on building a complex system there is no way possible to know all of the steps that you're going to need to know in order to deliver the functionality that you want at the end of the process.
Requirements are rarely fully defined at the time of discovery. Furthermore, you may not even know the functionality that you need at the end of a software development process until you have begun working on that piece of software. Agile processes acknowledge that you don’t know everything at the beginning.
I can't remember having talked to anyone that works in software that really likes waterfall for anything other than projects that are simple projects that they have done the exact same thing before. Why do business guys invariably want a project plan that marches you down a constrained path? Why do we have contracts that specify known delivery points? I think the answer is that waterfall is easy to understand. Waterfall doesn't beat you over the head and tell you how hard it's going to be down the road. Waterfall doesn't require you to acknowledge you don't know everything.
In working, researching and planning to add agility to our organization I've read a lot of blog posts, a lot of books and talked to a lot of people. I got really tired of hearing the term agile.
Casual use of the work agile implies several qualities that I believe are counterproductive. First, it implies that the test for “Agile” is a true or false measurement. This is not true. Agility is defined as the ability to react to change. Agility is a relative measurement. Just like some athletes have greater agility than others. Processes and organizations have different amounts of agility.
The word Agile does convey the meaning of having agility but it doesn't prepare us for the concept that being agile is the ongoing state of developing more agility. It's not a binary measure of something one possesses. It is an everyday struggle to try to do things better to try to adapt to the situations that are presented before you as you operate your organization.
At Promet, everyone in the company is tasked with adding agility to our organization. I want to share some of the learning’s that we have discovered as we are continuing to increase our agility.
A history of agile
The term agile wasn't used in the context of software until a group of industry thought leaders gathered in Snowbird, UT in 2001. The term was first used in this manner and published in the Agile Manifesto.
During the 90’s there were people talking about doing things faster and those usually revolved around a formal process having everything listed. If you're here and you're in this presentation you probably know or have some feeling of why this doesn't work and you’re wondering how to get better.
Many current agile approaches are based on the Agile Manifesto. If you haven't read it I strongly recommend that you do; it's a short list. There is the list of principles as well. Don’t overlook the principles. My personal journey into agile would have been shorter if I had read and followed these at the beginning.
There is a number of different frameworks one can pick from when shopping at the agile methodology store. Wikipedia has a considerable list: Extreme programming, scrum, rapid development, Kanban.
We chose the Scrum framework. Why? It's widely used, some of the team members in our organization already had experience with it, and there's training available.
Scrum is named after one of the ceremonies of the process. Scrum is a rugby term. Scrum, as it turns out, comes from the word scrimmage. The scrimmage is a modification of the word scrimmage which was borrowed from American football. Scrimmage, in turn, derives its form as a reflex of the word skirmish. Scrum teams have a daily skirmish that is called a scrum.
Scrum is really based on collapsing the essentials of software development into some easily understandable, easily definable segments so that we can determine what needs to be done and get working on it as quick as we can so that we can figure out what the heck we actually need to do
There is no way that in a short article or presentation all of the finer points of scrum can be explained. What I will try to do is give you the essential points that scrum requires and you can fill in the rest on your own journey.
Scrum has 3 roles. It's really important to separate who does what so that everyone is trying not to do the same thing. There also defined interaction points between each of the roles.
- Product owner: the product owner is responsible for the business value of the project.
- Team: the team is self-organizing. The team isn't told what to do by some master plan the team picks who is doing what based on who is best suited for it. The team relies on the other team members to support each other.
- Scrum master: scrum master is not a project manager. In order to succeed, in order to gain agility, this is one of the critical pieces that have to change in your organization. A scrum master is more of a coach and a facilitator. He helps all of the team members understand what the requirements are. He clears out the roadblocks for the team. He takes care of the boring pieces of dealing with the product owner.
There are 4 ceremonies that are defined by scrum:
- Sprint planning: the team meets with the product owner to choose a set of work to be delivered during a Sprint.
- Daily scrum: a daily meeting to share struggles in progress. Also called the daily standup. You really should do it standing up so that is uncomfortable if someone talks for more than 15 min. The idea here is clear short effective communication.
- Burn down chart: this is a chart that displays the total backlog of items that your team is working on and the progress that has been made today.
- Sprint retrospectives: the team looks for ways to improve the product and the process.
- Product backlog: the prioritized list of desired product outcomes and features.
- Sprint backlog: a set of work from the product backlog that the team agrees to complete in a Sprint broken down to the task.
- Burn down chart: At a glance look at the work remaining.
Core concepts revealed
Okay so maybe there are some more details to scrum.
The team loyalty is to the product. By communicating what the product is about, what the product should do and who the product is for, we make the team responsible for delivering the product. The loyalty of everyone involved is to deliver a functional well-tested product.
In Sprint planning, the team has broken down the desired functionality and features into iterations. At the end of every one of those sprints, we have to have the potentially deliverable code. One of the cool things about Drupal is your code is deliverable at the end always. One of the uncool things about Drupal is your code is deliverable at the end always.[EO5]
What are the struggles with scrum?
Sometimes when developers or even project managers hear the word agility, iterations, and deciding what to do each day; they take it to mean that we wake up every day with a completely new vision of the project and we can start out doing whatever the heck we want; today is a completely free and open experience.
We have potentially deliverable code at the end of each Sprint. That code has to be thoroughly tested. That means instead of waiting until the end of a project to test what's happening in the code we have tested each iteration. Drupal presents a particular challenge here as it doesn't really lend itself to unit testing.
Changing our internal expectations in the way we work. SOW's, contracts, customer communication, have all been challenges.
Scrum really does require that the team be a team. Everyone has to understand what's expected of them and be committed to getting it done. Everyone has to support other team members and be looking out for where they are going. Ask each other what you did yesterday to struggle. Everyone has to be willing to listen to the other team members. Well not only listen, everyone must actually act upon the information.
Using scrum with Drupal has some particular challenges. Formal scrum recommends that teams be 7 people plus or -2. There are a few Drupal projects that I've been on that will support this larger team. One of the tenants of scrum is also that a team be all co-located. Most Drupal firms that I know have people spread out over different locations. Drupal has its own issues with the way it's built and how to test Drupal.
What can you do today?
Get started. No really get started. Part of the tenant of agility is that you acknowledge that you don't have the answer to everything. It's better to get started right now than to wait until you have a perfect plan.
Start having daily standups. They are simple. They are easy. They can easily add substantial productivity to your team. The stand-up format has a few simple easy-to-understand rules. Have them standing up. They should not go over 15 min. everyone has 3 questions:
- What did you do yesterday?
- What are you going to do today?
- What roadblocks do you have?
Developers, graphic artists, system admin's and UI professionals all have a unique insight into projects. They are expensive, well-trained people. Use them. Explain the project to them, get them to understand it, work them as a team. Although we have seen an increase in the amount of time that we spend in meetings, we have seen higher quality projects go out and reduce the overall time that is required to deliver them.
As much as I hate to say it, project managers stink! You and your team will be more productive when you see the role the project manager changed to be more of a team coach and facilitator. The traditional role of a project manager as the guy who beats the drum, marching everyone down a path and demand performance is really less than optimal for software development. A project manager should be a champion or defender for his team, should help them get rid of the obstacles that are in their way. He should clear out the path for his team to go.
The greatest savings that I've seen is from something that I've seen discussed very little, although it exists as point number 10 in the agile principles. I'm going to restate what it says in a different way because I think it makes more sense. The greatest impact you can have on your project in terms of productivity and grief is not by managing how to get more done. But rather by increasing the amount of work that is not done. What I mean by that is define requirements in the simplest way you can. Look for the simple way to get things done. Work with the product owner to get the most useful, least complex resolution to the problem at hand.