A bit more to IT

I’ve been meaning to write a bit about the 2009 report from Forrester Research mentioned in the title of this post for a while now because its a really excellent summary of many of the problems facing Agile at the moment. It covers much the same ground as my earlier blog post which asked whether the adoption of Lean Thinking could lead Agile toward a broader focus. Having worked in Agile for a few years now I can relate to what they are saying and I suspect many other folks who are interested will have the same viewpoint.

Forrester interviewed a number of tech industry professionals for this survey including those from QA, product management, support, consulting and sales & marketing. They were very careful to include people from all areas of the business and not just developers- which is where the traditional focus of Agile has been. Forrester suggest, that if the goal of Agile is to shorten the path between development and customer, Agile must evolve to become more than a set of development-centric guidelines. The positive message from their survey is that there is still time to achieve this because, for most organisations, Agile adoption is not in its mature phase. This is not to say that significant progress has been made it is only that this progress has been made in terms of development teams and people closest to them such as QA.

At this juncture one might ask the question how could improvements in the development team, and their interaction with those outside, be a bad thing? Development teams are certainly the core area of value creation in technology companies; to use the automotive analogy selected by Forrester they are the engine of the car. It is their contention, however, that Agile methods have tuned the engine without upgrading the other components of the car.   This conjures up a number of images in my mind including a one of a hot rod with terrible bodywork and a massive gleaming engine in a rusty engine compartment!

For many developers tuning the engine is enough.  Agile gives them a better way to work and removes some of the things that developers don’t like doing, such as writing documents.  These things are great for developers but they have limited benefits outside the development team.  By improving, or optimising, only one area organisations will not be able to achieve the bigger goals attainable by an Agile engagement which recognises that development relies heavily on many other areas.  For example there is no point in having frequent releases if other departments are not able to fully market, implement or support the technology.

A number of people have noticed the changes to the wider organisation that the adoption of Agile development brings. The problem is that the changes outside the development team, though positive, are by-and-large accidental and relatively poorly understood.  How then are we to effect the changes required to attain the goal of Agile engagement?  Some people have argued that it cannot be achieved within the traditional scope of Agile and that new inspiration is required.  A growing number of people believe that Lean thinking is the way forward though as yet we don’t have many examples to look at.  As for Forrester they leave that question for a future report.

Just a moment ago I did a Google search for Agile Development in Edinburgh and much to my surprise this blog came up first.  What a nice surprise!  Must start adding more posts because over the past few months I’ve not really been updating it.

The subject of user acceptance testing is pretty crucial for agile development teams.  Its certainly an issue which we have given quite a lot of thought to in our XP team in the past few months.  There are two points I want to make in regard to this, the first concerning our overall approach to UAT.

To break it down for those people who are not aware of the debates around automated UAT in agile there are basically two schools of thought.  One says that the developers and the customers should sit down before the coding starts for a particular feature and put the tests together for it.  The rationale behind this is that this will initiate a discussion between these two parties during which a better understanding of the requirements will emerge.  This ongoing discussion between developers and customers is particularly important in agile as there is no formal requirements spec adopted by most of the main methods.

The other way of thinking suggests that the tests should not be written way before the production code is done because, in that state, they are not actually doing what they are supposed to namely documenting the code.  This view is informed by a Lean perspective which says that we should cut down on artefacts which don’t contribute to our overall goal.  In other words tests which are not documenting code are ‘inventory’ and should be cut out.  This second approach is one of the ways in which Lean thinking is influencing the day-to-day approach to agile.  I think we can expect to see more and more of this in the near future, which, as I’ve recently written is a good thing.

Whichever approach is adopted for testing it still remains crucial for agile teams.  Given the increasing popularity of agile and the importance of UAT for it one would expect there to be many different frameworks to allow automation of the process.  Not quite.  The ones we have tried in our team all have problems and while we have settled on using Selenium we often speak of it as being the ‘best of a bad bunch’ rather than ‘best of breed’.  Now ours is a very small team and it got me wondering how larger teams cope with these flaky frameworks.  Do they use them at all or are they in fact more reliant on the manual variey of UAT.  I don’t know of any studies which investigate this issue but they would make for interesting reading.

Now I’ve written about one of my favourite Social Informatics thinkers on this blog before- Rob Kling.  I’m pleased to say that another Rob has re-entered the blogshpere after a bit of a break.  I’m talking  here about Rob Lally who has, until recently, been at J P Morgan.  Rob always has something interesting to say on a number of subjects in and around software development and I find its worth surfing over now and again to see what he’s been posting.  One of his recent contributions, which I liked a lot, tackled the question of whether Agile is a characteristic or a culture…

Well I best start off by saying that I didn’t win though I did end up really enjoying the evening.  My presentation worked out quite well as a 20min piece and I was very pleased with the response to my visuals.  I think what probably let me down was my presentation style.  It was not quite as polished as I would like, however, as they say practice makes perfect.  I really enjoy presenting things and am very keen on doing a bit more work on this aspect.  And what of the other two candidates?  Well the quality of both the other presentations was very high and both were very interesting subjects.  The eventual winner, Elizabeth Hutchison, delivered a really effective presentation which really impressed the judges.

The whole event was well organised by the APM committee and the choice of venue was excellent.  What I most enjoyed about the evening was the feedback from people about my ideas.  It seems that managers from other industries were able to grasp the context of what I was saying which is very heartening considering the importance of convincing people of the benefits of doing agile.

Thanks to everyone who has wished me well for this event.  Once again my special thanks to the agileScotlanders for making this research possible and to Napier University for giving me this opportunity in the first place.

The slides I used in the presentation are here.

I’m going to do my presentation “‘If you have programmer’s on two floors, forget it’: The challenge of distributed Agile” next Monday 2.2.09 at 7.30pm at Napier University.  I’m very excited about the prospect of talking to this group who were so key in getting this research done in the first place.  I’ll put the slides from this presentation up here shortly.

Publicity details for the talk are here.

After an amazingly quick turnaround of a couple of weeks Clarke Ching has announced that his excellent business parable Rocks into Gold is now published.  Clarke is to be congratulated on the speed at which he has managed to get these topical ideas out to a wider audience.

This tale, which shows how a bit of agile thinking can help software people keep their jobs during the credit crunch, is available in three different formats from this site.  Even those whose pockets have already been adversly affected by the credit crunch can partake of this 20 min read because one of the versions is free.  I think that I’m going to go for the paperback version, even though it costs £1 more, because its more tangible than the download version.  Roll on the publication of Clarke’s larger book where the ideas outlined in Rocks into Gold are expanded upon.

At the moment I’m preparing two presentations for quite different audiences based on my dissertation about distributed agile.  One group knows a lot about agile and its basic concepts while the other probably knows very little, you get the gist.  I think it is safe to assume, however, that both groups will not know very much about outsourcing past what they have read in the popular press and possibly the trade papers.  The problem is that much of this stuff takes a very pro or anti outsourcing stance depending on who is writing.  Thankfully there is a body of research which one can draw upon to obtain a more balanced perspective on what is understandably very divisive issue.  In this regard I’ve found the work of Erran Carmel, among others, to be particularly valuable.

Now as I was finishing off my brief history of outsourcing for one of the presentations I realised that I should bring it up to date by considering the impact of the credit crunch.  At this point I hit upon a bit of a snag.  The credit crunch is such a recent phenomenon that the only commentary available is from the press or trade papers whose analyses and vested interests I’ve already found so unsatisfactory.  Therefore the answer to my post title question is its very difficult to say right now.  More rigorous and balanced viewpoints require a bit more time to complete than a quickly penned commentary and quite rightly so given the fast pace of events in the economy over the past year.

Now you may be wondering what all this has to do with distributed agile.  Well if the amount of outsouring increases as a result of the credit crunch then I fully expect there to be greater emphasis on agile as companies try to gain more responsiveness from their offshore vendors.

My pal Clarke Ching is going to publish a short business parable next year called Rocks into Gold - A credit crunch parable for people who build software for a living. He’s going to give it away free online and also publish it as a paperback.  I’ve recently had a read of the pre-copy edited version and I have to say that its extremely topical and very well written.  Free copies of this version are available on his blog and I know that Clarke is keen to get feedback on this.  It contains some of the ideas which are going to be in his new book Rolling Rocks Downhill which is soon to be published by the excellent Pragmatic Bookshelf.  I know that myself and many other people in the agile community in Scotland, and also further afield, are eagerly awaiting the publication of Rolling Rocks.

I was left wondering how many other software people are in the same boat as the hero of Rocks into Gold, Bob Billington, whose job is in danger because of credit crunch related cutbacks.  I’m not going to give away more of the story here because those that are interested can get a copy from Clarke.  Suffice it to say that a bit of creative and agile thinking saves the day.  My question is, do many of the folks whose jobs are in danger realise that options exist to save projects hit by credit crunch issues.  This type of easily accessible parable is sure to make the message that much easier to communicate.