Top 8 Mistakes Implementing Agile Methodologies

Agile development practices have been around for years and there seem to be evolutionary new flavors introduced almost daily.  As a testament to its mainstream appeal, even PMI is releasing an Agile certification, opening the floodgates to an organization that has long been seen as the keeper of the Waterfall keys.

As more and more organizations scramble to work in an Agile way, it seems that there are as many failed or ailing attempts as there are successful ones, though I have seen studies putting the successful adoption rate between 20 and 75% — a huge range to be sure.  Looking in from the outside and having worked with organizations making that jump, I have seen a number of trends that I want to call out.  Unfortunately, these are not cited with scientific statistical rigor, but I am hoping they are helpful nonetheless.

Just for full disclosure, I manage a Project Office responsible for delivering a wide variety of development projects in a consulting environment.  This affords a certain perspective favoring the business outcomes of adoption rather than the technical ones.  Further, I am focused on using the correct methodology to meet the situation while building reusable organization assets and collecting performance metrics.  So with that, here we go!

  1. Treating Agile as just a Development Practice rather than a Business Process — Working in an Agile way requires an understanding and realignment of your project selection, portfolio management, budgeting & funding process, requirements gathering and acceptance process.  Assuming that none of the business functions will be affected and that just the development teams will work in an agile way leads to a cognitive disconnect between the business and the delivery side — a gulch that is often there before you even try to implement agile.  This is especially true when Agile is a grassroots effort that is greenlighted without support.
     
     There needs to be a clear understanding of what the organization will get out of the new process and where there are opportunities to remove barriers before they are even an issue.  Failure to address these alignment issues will just lead to communication gaps and the business side demanding that you revert to the pervious practices.
     
  2. Using Agile Coaches who don’t have a Stake in a Delivered Product — Bringing in an agile coach can be a huge benefit to your organization as teams usually learn best by doing.  Usually a team (and the business side) will need to go through a full release cycle before the process really sinks in and an agile coach can help you to identify risks and guide the team through this.  Unfortunately, the coaches often will act as academics or shamans who want to sit in an advisory role with no real team deliverables.Always insist that the coach work with the team in a real-life product cycle and plan for the learning curve in your capacity planning.  You’ll get a ton more out of it and will see better adoption. This also means that you’ll be working with a coach who is a practicing professional and able to get things done.  Better yet, tie compensation to adoption metrics or success criteria for the release. 
     
  3. Forgetting to do Sprint and Release Retrospectives — One of the biggest benefits to agile is the virtuous cycle of continuous improvement, building on the lessons from the previous iteration.  When the going gets tough or there is a perceived lag in the project, this is usually the first area to be cut.  This just bleeds the value out of the Agile process and perpetuates problems rather than learning from them. Repeat after me…   “I will not cut Retrospectives! I will not cut Retrospectives. ” (Especially as they pertain to capacity planning.)
     
  4. Treating Agile Development as a Replacement for Planning or Requirements — Organizationally this is usually in response to poor planning or badly defined requirements.  The thinking usually goes, “We end up in trouble trying to deliver against half-baked specs and unrealistic project plans, but if we have neither of those things, we’ll be free to just deliver (and no one can say we failed.)”  That last part is usually subconscious.
     
     If you can’t come to an agreement with the development team as to what you want to deliver or aren’t willing to staff the team appropriately, there is little hope that working in an Agile way will improve things much.  Yes, letting the team interact with the business directly will help to specify what is asked for, but in my experience, the issues usually lies in poor arbitration between parties with conflicting needs.  Agile don’t directly fix that.  Failure to address the root problems can lead to additional churn and rework.
     
  5. Not Having a Vision of Done — This one might be a bit contentious, but every project needs a picture of done with acceptance criteria associated with it.  Usually this is a set of stories, grouped into a release, with UAT criteria associated with them.  As a consultant, I usually have to be much more specific than that as building a backlog against unlimited time and dollars usually doesn’t get us invited back.  Just make sure that you have a clear picture of whether done is “good enough with a list of nice to haves” or if there is a more stringent criterion that you need to be shooting for.  Not identifying that up front and structuring the team for it leads to bad things.
     
  6. Not Preparing the Business for Participation — It is essential that the business side of the organization have daily participation in the project.  Most likely, they are used to an intense upfront requirements phase and then are ignored until UAT.  Sure, they will still need to have initial engagement as the Product Backlog is built and the release plan made, but the ongoing participation often comes as a surprise and must be managed carefully.  Often they are happy to do it, but springing that on them with little notice or input can hurt feelings and lead them to a feeling that the development team is needy and unsure of themselves, rather than following a prescribed process.
     
  7. Starving the Factory — Trying to treat each iteration as its own waterfall or expecting that all story elaboration will be done in sprint often leads to work starts and stops — especially as a team scales.  Planning your stories and digging into the business drivers and needs one sprint ahead allows you to engage with your QA teams and Business Analysts to provide the fuel that the development team needs to keep the factory rolling.  This is not a substitute for having the developers interact with the business directly, but it does take some of the burden off of them and can leverage the specialized skills of your BAs and QA folks.  For me, this seems to be the best blend of sustained agility and supporting a scalable and even geographically distributed team.
     
  8. Expecting an Agile Experiment to Scale — So you wanted to prove that Agile can work and will be able to be adopted into your organization.  You’ve pulled 4-6 of your top architects and high-potentials into a mini project because you expect them to be leaders of additional projects when you scale.  The project goes great, the team blazes through the deliverables, and you’re ready to go!  Right?  RIGHT?
     
     This is a great way to train leaders in the overall flow and tenets of Agile, but it doesn’t always paint a good picture of how this will work in a real environment, especially if you work with distributed teams, outsource groups, or 9-5 developers who are phoning it in.  Once you have those leaders, make sure you do a real PoC with a realistic team to work the kinks out, especially around communication and code acceptance.

Wow, that was more typing than I had anticipated and I am sure as soon as I hit publish, I’ll think of 5 more that I just have to include.  Maybe there will be a part 2.  Hopefully this will save you a few headaches and help you to navigate the agile waters.

Advertisements

Making a work item only editable by its creator in TFS 2010

I was recently confronted by someone building a TFS template for a client that had an interesting business rule they had to support.  They needed to be able to allow only the creator of the Work Item to edit or make changes to it once it was created.  There is a lot of magic you can do with the TFS rules and field comparisons, but this one was a bit harder than most.

When TFS uses a work item, there are actually three passes that are made through the fields putting the rules in an order of precedence (For the full explanation check out this article):

  1. The first pass sets and copies values we well as dealing with any conditional rules (WHEN and WHENNOT, etc.)
  2. The second pass evaluates any fields that were changed in the first pass and then evaluates the WHENCHANGED rules, releasing the work item for editing
  3. The last pass is on commit when the Work Item is saved to the database.  This catches any rules triggered during the editing process and finally the SERVERDEFAULT rules.

The Solution

We can take advantage of this ordering to get that locking functionality to work by creating new rules on System.ChangedBy

<FieldDefinition reportable=”dimension” refname=”System.ChangedBy” name=”Changed By” syncnamechanges=”true” type=”String”>
      <COPY from=”currentuser” />
      <FROZEN />
</FieldDefinition>

How it works

  1. Since Copy is processed before everything else, we can use it to force the value on load before any of the other rules are posted (like the frozen one).  Then we freeze the value (the rule doesn’t fire if the field is empty).
  2. If the user that created the work item goes to edit it again, they are allowed as the value it tries to copy is the same so it never commits to trigger the frozen constraint.
  3. If a different user tries to edit the work item, the value is copied over top, but then the frozen constraint is triggered as it does the validation pass before it releases the work item into edit mode generating the error.

The downside here is that there isn’t a nice error message or prompt telling the user that they don’t have the ability to edit a work item that they didn’t create.   While that would be truly awesome, we might be content with the fact that the business rule is maintained (perhaps that makes an auditor somewhere happy.)

As an extension of this, we can also take advantage of the <FROZEN/> field by specifying the groups that are affected (or not in this case) by the change.  Here we allow Collection Administrators the ability to override, but keep the rule for everyone else.  Neat!

<FieldDefinition reportable=”dimension” refname=”System.ChangedBy” name=”Changed By” syncnamechanges=”true” type=”String”>
      <COPY from=”currentuser” />
      <FROZEN not=”[Global]\Project Collection Administrators” />
</FieldDefinition>

Setting Up Project 2010 for Agile Development

When teams start going down the path of agile development, often one of the first things they’ll do is throw away the project plan. This is almost never a good idea as the project plan can be a guide to help with capacity planning, release planning, and managing the project resources who are assigned to tasks.  MS Project diagrams can also be an excellent communication tool if built with communication in mind.  In the spirit of Agile development, we want to eliminate all activities that do not directly contribute to the team delivering a working product.  This means working with a project plan that can contribute to sprint planning and the achievement of release cycles.

Many developers and team leads have built a strong hatred for Microsoft Project as improper setup and unfamiliarity with the tool can lead to frustration as Project tries to adjust values and balance tasks across the team.  This posting builds on my previous post on Work vs. Duration for development teams helping you to set Project 2010 up to correctly from the beginning to make working with your project plan a much better experience. 

  Project 2010 Settings

 

All of the following settings can be accessed within the project options.   Click on the File tab, Select Options, then select Schedule from the pane on the left.

Hour per Day — It is generally accepted that you are only going to be getting about 6.5 hours of actual development from the developers on your project as the remainder of the time will be spent in meetings, research, collaboration and other activities that, while they do contribute to the project, do not contribute to the burn down.  Because you are entering your tasks in units of work, this will allow MS Project to plan for this overhead as it levels resources across the tasks.  This takes a bunch of the confusion out of the project plan and help MS Project to give you a reasonable result should you put more than one person on a given task, building in some overhead for the collaboration. 

You might be wondering what to do in a situation where not all resources will be working like this — Project Managers, Business Analysts, System Configuration guys and the like.  If you are mixing both types of roles on one project plan, you have to get a little bit fancy, but it is not too bad.   In this case, you would need to build separate calendars for development and time-based people.  Then on the resource management screen, you get to choose which calendar you would like each person to follow.  Perhaps this would be a good topic for a future posting…

Hours per Week — This is an extension of the Hours per Day above and the two should always be adjusted in tandem to match.

New Tasks Created — This should be set to Auto Scheduled.  Project 2010 lets you designate which line items on the plan should be leveled.  In practice, you should let most things be handled by the resource leveling function and focus on the dependencies for order.  I find it is easier to just let project do this from the beginning.  This also allows you to see the task relationships right away as the linking makes it much clearer than all those lines that you have to chase like some kind of maze.

Duration is Entered in — This should almost always be set to Days.  This your barometer of how long a task will take to get accomplished with the currently-assigned resources.  For tasks that are effort-driven (development) these should be calculated units rather than something entered. 

Work is Entered in — This should always be set to Hours.  This is the units that the development estimates are recorded in. 

Default Task Type — This should be set to Fixed Work.  This means that the default task will be entered in units of Work rather than time.   This is true for most development projects.  You’ll notice that this automatically sets the check-box for “New tasks are effort driven“.

In practice, this means that you will need to think about the work that is being done and come to a conclusion as to whether it is effort-based or time-based.  With this setting, you will need to go into the properties of the time-based work items and uncheck the “Effort Based” property.  If you think you are going to have a lot of these types of work items, you might choose to add the Effort Based column to the GANTT view so you can do this quickly.

You might be asking yourself what happened to Story Points?!  What’s all this talk of hours?  Microsoft Project is a planning and sourcing tool and should be used at the point where you are planning out the actual work to be executed.  As you go into your sprint planning cycle to initiate the next sprint, there are two ways you can approach this.  From a capacity perspective, you might take the look at the historical relationship between Story Points delivered and Sprint Capacity to put this in at the Story-level.  The other option is to do this as a sprint-based work breakdown structure as you decompose your stories into individual tasks and take the summation of the individual estimates.  Both are valid approaches, but are dependent upon your discipline and the depth of your sprint planning cycle.  I prefer the latter approach, but I understand that project constraints sometimes don’t allow for that level of decomposition.

 

Project 2010 Settings for Agile and Scrum Projects
Project 2010 Settings for Agile and Scrum Projects

Using MS Project 2010 for Software Development Projects – Work vs. Duration

There is a general misconception that Microsoft Project, or many of the commercially-available project management applications, doesn’t work well for software development projects. When talking with Project Managers and Architects, their trouble usually arises from scheduling and resource leveling. They’ve usually been burned by situations where countless hours have been invested into building out a project plan only to find that a minor change skews the plan into an almost unusable mess. There might be many things going on at the same time, but it usually boils down to an understanding of the difference between Duration and Work.

There is a general misconception that Microsoft Project, or many of the commercially-available project management applications, doesn’t work well for software development projects. When talking with Project Managers and Architects, their trouble usually arises from scheduling and resource leveling. They’ve usually been burned by situations where countless hours have been invested into building out a project plan only to find that a minor change skews the plan into an almost unusable mess. There might be many things going on at the same time, but it usually boils down to an understanding of the difference between Duration and Work.

Conceptually, this isn’t a hard distinction. Most people can tell you that Duration is the calendar time it takes to get a task done whereas Work is the number of development hours it takes to complete a given objective. However, in the middle of a project, the distinction is easy to miss especially when the business objectives and the actual project work-stream are expressed in different terms. The business stakeholders want to know when functionality will have to be handed off to other groups for activities like integration testing, security audit, and user acceptance testing – ultimately, when will the functionality be delivered. On the other hand, the development team is used to thinking in development hours and estimated work remaining. All of this is made more complex by the fact that MS Project’s default view is expressed in Duration.

Let’s look at a simple example based on the following Work Breakdown Structure (Not complete, just an example):

WBS Item

Initial Estimate
(Baseline)

Actual Hours

Work
Remaining

Traceability

1.0 – Users can log into the web site (Authorization) and are granted access according to their assigned authorizations 80 Hours 0 hours 80 Hours Use Case 1.0
     1.1 – Develop custom logon control for the upper-right in the Master Page header 20 hours 0 hours 20 hours Use Case 1.0
     1.2 – UX Design for logon flows 8 Hours 0 hours 8 Hours Use Case 1.0
     1.3 – Forgot Password Functionality 20 hours 0 hours 20 hours Use Case 1.0
Alternate Flow (Forgot Password)
          1.3.1 – Forgot Password Challenge Question 12 Hours 0 hours 12 Hours
          1.3.2 – Forgot User Name e-mail process 8 hours 0 hours 8 hours
Other tasks…  

 

I’ve put in a little bit more information into this WBS to prove a point. As you go through your initial planning (at the user story/ product backlog/ use case level) you will be looking at a time-based measurement. Likewise, when you are handing that work into the development queue, the estimates that you will arrive at in your sprint plan will be based on effort. Don’t work if you are using story points at the highest level. These eventually get translated into hours, and should be considered a substitution for the effort measurement.

Pasting this into MS Project 2010, it is tempting to just use the standard Gantt chart, but that leads you directly into the wrong path.

Notice that the column on the default view is Duration – that is, calendar time that is required to get the task done. This works if there is only one person (or resource) working at a time, but this quickly breaks down when you have a team or several teams working on the project at once. Instead, you need to add the Work column to the Gantt chart.

  1. Scroll the tabular view to the right until you can see the ‘Add New Column’ heading
  2. Click on the menu arrow to see the list of available fields
  3. Select the Work Field

Once the actual work is entered for the individual work packages, all items will automatically be marked as Effort Driven and the project plan can be leveled against the resources. Here I have added myself as the resource and have allowed MS Project to rebuild the schedule.

The Final Project Plan

Allocating an additional resource to some tasks in the project (John Gault, in this case) keeps the development effort the same, but MS Project 2010 shortens the overall duration of the project to take advantage of the additional work capacity. With the additional resource, we’ve shaved 2 days of duration off of the project.

Just be sure that when you add additional resources, you tell MS Project how this should affect the task.

Conclusion

With a little planning and attention to work estimates rather than only duration, MS Project can be a huge asset to the software development team. This work/effort-based tracking is vital to effective sprint planning and allows the project team to quickly visualize the workstream against the sprint capacity. This also helps to focus the team on critical path issues that might be missed in a taskboard exercise.