Learning your scales

Recently, we looked at how a jazz combo is a great metaphor for the interactions and exchanges that make up a great small team.  The question then comes to mind, “surely, isn’t there more to it than that?”  The answer is an unequivocal yes.  There is quite a lot to it.  Groups that reach this level of comfort and collaboration have a lot of experience and expertise under their belts.  Let’s start to explore the journey that gets us there.

When I was a kid, I was very lucky in my music education.  My high school was the host to an annual jazz festival, featuring all of the local high school and university jazz bands.  Because of this, I was exposed at an early time to a lot of different styles and techniques for music.  The festival was actually prestigious enough to be able to attract some top named talent each year who would give a concert and workshop.  We had folks like Dizzy Gillespie, Louie Bellson, and Clark Terry work one on one with us.  This was the opportunity of a lifetime, and it happened once a year!

I’ll never forget the first workshop.  Here I was a young high school student.  I considered myself a pretty darned good player, but I was going to learn from one of the founders of the bebop movement, Clark Terry.  This was a chance to get past all of the usual, boring stuff all of the teachers told us, and get into how jazz is really made.  Mr. Terry had us play a little, went over a few things, and then asked if anyone had any questions.  I asked him how I can really improve my soloing skills.

Now before we answer this, I should take a small detour for those who may not have ever been involved in jazz.  During each song, there is a break where certain players will get in front and improvise.  No music is written down, just the chord structure(some might say architecture) of the song.  It is very challenging and intimidating at first, but absolutely the most fun you can have once you learn how.

Now, where was I?  Oh yes.  Clark Terry was about to teach me the secret to great jazz improvisation.  He looked at me and said “Learn your scales man.”  What?  Learn my scales?  Scales are about as basic as it gets.  Once you learn how to make noise without making the dog howl, you move on to scales.  No real songs just up and down in each key.  Guess what?  He was right.  I know, big surprise a jazz legend was right about how to play jazz.  Once I learned those scales so that I wasn’t “thinking about what to play next”, the improvisation came easy.  I was merely playing something that “felt right” because the actions and structures of those scales were burned into my subconscious.

Programmers need to do the same thing.  Once we learn the basics of control structures and loops, we need to learn our scales.  It is not enough to just learn them once, or read about them in a book either.  We must practice them every day.  We apply them to the work we do, and we aren’t willing to compromise on their quality.  The more we do this, the more natural it becomes.  Just as I might not consciously say during a solo “now I am going to play a B-flat minor with a diminished seventh” I might not while writing a piece of code “and now, I will implement a strategy pattern”.  I will obviously be aware of it, but it will be so built into what I do as a programmer that I won’t have to stop and think about it.  My awareness will be built into what I am doing.

So what is the programming equivalent to scales?  There are actually a few activities that I feel fit into that category.  Most of them are discussed quite often but lets take a look at them again.

Test Driven Development

Writing a test prior to writing your code is not very controversial anymore.  Everyone recognizes the value in doing this.  And yet, most programmers will maybe write a few tests to get started, but then quickly move to an attitude of “I don’t have time right now, I’ll get to them later”.  This is totally understandable, since the idea of writing tests first still feels counter intuitive.  So do minor scales.  It takes repetition and practice to get this activity into our “muscle memory”.    Sometimes what we are doing is uncomfortable, or feels like it is wasting time.  We do it slowly at first.  But over time, it becomes an instinct.

This instinctive activity is not going to happen just by doing it at work either.  Just as Uncle Bob Martin asserts in his latest book Clean Coders, we need to practice at home also.  Find a project or site where you can practice writing tests and making them pass.  Get used to the idea of writing a test and then working to the “green bar”.  We’ll talk more about practicing in a later post, though.  Suffice to say, you need to practice, and a true professional will practice at home.


Another form of scales is Object Oriented principles.  The best description of the principles that lead us to true success is the acronym SOLID.  This has been identified and explored in many forums, so I will only briefly touch on them.  For a deeper dive, I would refer you to Robert C. Martin’s book: Agile Software Development, Principles Patterns and Practices.  Here is a link to an early article he posted on the subject. Meanwhile, for your dancing and dining pleasure, here are the SOLID principles, in brief:

  • Single Responsibility – An object should have one and only one reason to change.
  • Open-Closed  – a class should be open to extension, but closed to modification.
  • Liskov Substitution – Derived classes must be substitutable for their base classes.
  • Interface Segregation – Clients should not depend on methods they don’t use.
  • Dependency Inversion – Modules should depend on absractions, not details.

One interesting parallel to our musical paradigm here is that, as you start toget very comfortable with staying within these principles, you start to notice things that are not quite right.  You are so used to looking at code that fits within the SOLID principles that it just seems natural.  Code constructs that violate these principles come across as out of tune.  Something we refer to in music quite often as “klunkers”.



The final scale I want to talk about is Design Patterns.  Patterns are another part of our basic language.  We need to work with them, and be comfortable with them.  I am not a huge fan of saying “Here is a problem, which pattern shall I apply to it?”  That feels artificial to me.  However, I also remember a time when I was just learning music that I did consciously say “Now I am going to play notes from the B-flat blues progression”.  As we are using patterns, we become more comfortable with looking at a problem and just knowing what we are going to do to implement it.  Later, as we are discussing it with someone, we will note that what we did was apply a Visitor pattern.

So, in conclusion, even though it isn’t anywhere near as much fun as just going out there and soloing, we need to learn and practice our scales.  It isn’t enough to read about them once, or to pull them out of our bag of tricks when we have enough time or remember to use them.  They need to be a part of what we do every day, and they need to be so ingrained in our brains that we don’t even notice when we use them.  Hey if its good enough for the greatest performers and innovators who ever lived, surely its good enough for us.  Now I gotta run.  I gotta go practice.

Hey what about Refactoring?   Refactoring is hugely valuable and extraordinarily important.  It is so important that I want to save if it for next time, when we talk about Practicing.

Posted in Agile Development, Agile Management, Scrum Development, Scrum Management, Software Development, Software Development Management | Leave a comment

Making Beautiful Music – A new way at considering small teams

<Adapted from my article in CM Crossroads titled “Making Beautiful Music Together”>

Have you ever watched a jazz combo? The performance starts with the leader counting off the rhythm, then stepping away. Then the drummer begins to lay down a beat. Even at this stage, the audience can feel a groove hit the room. Soon, the piano joins and adds both melody and harmony to the piece. Energy is flowing from the chords as the team starts to see and feel the direction of the piece. Now it’s time for the other instruments to join in the fun. A typical combo will have a couple of different instruments maybe a sax and a trombone, or some other combination. Whoever starts off will state the melodic theme for the song, although sometimes the whole group does this together. After that, everyone gets a chance to do a solo, in which they improvise on the main theme, key off of past experiences, and apply their musical knowledge. It is not uncommon for jazz musicians to jibe each other, making jokes and comments while they are playing. The energy in the room builds and builds as the musicians play together, sometimes one at a time, sometimes in tandem. When you watch a jazz combo really swinging, it can be hard to tell who his having more fun, the audience or the musicians.

This metaphor works quite well to understand what makes a small team desirable, and how to be successful with such a team.   In the agile community, we have asserted over and over again that we need small, cross-functional teams. And yet, what really is cross-functional? Can cross-functional really work? The more traditional view of software creation involves the need for separate, functionally focused teams that are experts at their domain. The teams only interact as they are passing work items from one to the other. When development is done, we hand off to test. Test will find defects and hand them back to development. And the dance continues in this light forever.  The jazz model demonstrates that this can, in fact work.

In a jazz combo, each member of the team has a specialty. The members play individually, but often together  they create a tapestry of music that becomes much greater than the sum of the individual contributions. A small development team works best this way. We have some set of programmers, testers, documentation specialists, and some representative of the business working together. Team members gain their energy from each other. They try new things and get feedback right away from anyone who wishes to listen and share.

The team members don’t need to just focus on their own areas either. A tester can veryeasily and effectively form a duet with a programmer. They will play off of each other with their ideas. The tester will write a test to express some piece of functionality that the software will have. Then the programmer will answer with the code that will cause the test to pass. So we write another test based on this back and forth interaction. In music this interaction is known as “call and answer,” and it is especially effective with the testing and programming cycle as it provides a rapid, tight feedback loop.  Communication is instantaneous, and allows for more freedom of action. More often than not, a programmer will pair with another programmer. This duet is very effective and powerful as well.  The programmers can build off of each other’s knowledge, and ideas that might “seem like a good idea at the time” are instantly reviewed by a peer.  This type of pairing turns code review from an intermittent, reactive process into a continuous, proactive activity., and should be embraced as often as possible.

Let’s explore some of the roles that are important in a development team. Usually there is some sort of coach or leader. In the Scrum world, you might hear about the ScrumMaster. Each of these names is meant to describe someone who is both a part of, and to some extent outside of, the team itself; in a jazz combo, this is the director’s role. Not every combo has a director, but many do. Sometimes that director is part of the team, only directing long enough to initiate and introduce a number to the audience. In software development, the director represents the team to the stakeholders, and helps plan the meetings, stand-ups, and the like; essentially counting off the beat. If the rhythm seems to be getting lost, the director can help the team identify this fact and help with corrective actions.

A team also needs an individual who has the ability to identify what needs to be developed. In agile, this role belongs to the product owner. Now consider a jazz combo’s basic rhythm section: The drummer lays out the shape of what is to develop; the bass takes this one step further, presenting the progression of chords that identify the order in which the chordsthat make up the actual harmonies and melodies will be played. Lastly, the piano comes in with the rich, fully realized chords. Accordingly, the product owner has to play all three roles of the rhythm section, explicitly: identifying the work to be done or the shape of the upcoming work. Ordering the work in order of priority sets the rhythm.  Elaborating the story in terms of acceptance criteria provides the chord structure, and lastly, the constant interaction with the team throughout the development cycle provides the fulfillment of the intention as laid down by the aforementioned acceptance criteria

And, of course, we also have the rest of the musicians who are like the testers and programmers, in that they all have some specialization, perhaps the instrument they play, or the particular way in which they play.  For instance, in the Duke Ellington Orchestra, not only was Cat Anderson known for being a great trumpet player but he was also known as a high-note specialist. If applied to the agile software development environment, not only are there specializations like programmer and tester, but there are also some folks who are best at User Interface, or at database work. There was never a rule that only Cat Anderson could play the high notes, or that he could only play certain notes, and there should never be a rule that only your “UI guy” can work in the UI. That would lead to a very thin team.  There would be no ability to build a depth of knowledge throughout the team.  Should a particular team member be unavailable, and the skill that she specializes in is required, the team is now hostage to her schedule.

There are many reasons why small groups are desirable. Members of a small combo are best able to work together and play off of each other’s strengths and weaknesses. They can react to changes that might come from the stage dynamics. Whereas many large bandsrequire a hefty amount of coordination and very little room for improvisation, the small combo thrives on improvisation. Everyone adds what fits best, and the feedback from the audience is immediate. The energy builds, not just from each contribution but also from the cumulative effect. The band doesn’t stop and argue when someone makes a change during a jam session, band members pick up the new tempo and use this change to make the music better than ever before; the same thing happens in software. The team is able to communicate and work together. The different players are not going through some intermediary, but directly to each other. The energy, the pace, and the quality of the product all come out through this tight, frequent interaction.

I’d like to take the next few weeks or even months to explore further how we can apply the lessons of our jazz brethren to the world of software.  We can talk about ideas like how to be successful with duets (pair programming), or perhaps how to conduct a jam session.  Hopefully, this will spark some ideas that can build a strong understanding of how to work in our Agile teams.  What other comparisons can we make?  How about what we do when a small combo begins to grow into a large orchestra?  What might that mean to us?  The possibilities are endless.

So now picture this: The team comes together for a planning meeting; the director establishes the tempo by identifying, with the help of the team, the velocity for the upcoming work; the product owner then lays down a groove, describing the melody and harmony of the iteration.  She does this by providing the depth of description and acceptance tests that show not just what we will be doing, but how each story interacts with the others. Now the rest of the team picks up the melody as shown by how the programmers and testers pair up and work on stories together. The team’s energy builds as the code is tossed back and forth in short phrases. Each member employs his strengths, but helps to contribute to the overall outcome wherever he can. At the end of the iteration, the audience expresses its appreciation for another fantastic performance.  Now we can chill for a little while, enjoy our success, and look forward to the next gig.

Posted in Agile Development, Agile Management, Software Development Management, Uncategorized | Tagged , , , | Leave a comment

Everything I learned about Scrum Teams I learned from M*A*S*H

I like to participate in discussion groups.  I enjoy the discussions themselves, and I also like “meeting” the folks who are participating.  There are a lot of questions that get repeated in those groups, but I personally feel that the conversations are various enough that this is a Good Thing.   There is always enough of a twist on each one that I learn something new.

One question that comes up a lot is who should be a Scrum Master.  Sometimes it is as simple as “Hey, we are starting to do Scrum, so we need a scrum master, who should we get?” Sometimes it is a bit more involved.  Many of these conversations really focus on turning project managers into scrum masters, as a sort of natural step in a transition environment.   While I understand this seems to be a convenient, comfortable step, I’m not sure it is as helpful as it originally appears.  In our never ending search for metaphors to explain ourselves, I am going to utilize that well known show from the 70’s and 80’s, M*A*S*H

Consider the role of a project manager.  A good PM is responsible for making sure that all of the variables for a project are identified and categorized.  He is also responsible for identifying and mitigating all of the risks in a project.  Most of this is done at the beginning of a project, and will then continue in a reactive manner throughout the course of the project.  Other tasks that are important to a PM are to identify and manage the budget for a project, and also to make decisions along the way as to changes and delivery.  To me, this is Colonel Potter, as played by the late great Harry Morgan.

Col. Potter understood that his job was not to tell the surgeons what to do, or how to fix a wounded soldier.  He absolutely was a figure of authority, but knew when to get involved and when to stay out of it.  When push came to shove, if there was a decision that the team wouldn’t or couldn’t make on their own, he was there to either offer some insight to help them come to a decision, or in some cases he knew that he had to be the one to make that decision.  One disclaimer here:  the level of authority for a wartime military commander is going to be much higher than in our world, but much of this still applies.  Leadership is not really different it just becomes that much more imperative.

So who should be a Scrum Master?  I see the epitome of Scrum Master as Radar O’Reilly.  He made sure everything got done.  People got used to relying on him without asking for something in particular, and he really made everything happen.  If something got in someone’s way, he knew what to trade, and with whom, to make that obstacle go away.  I also think it is worth noting that Radar was a Corporal for most of the series.  He led from a position of no power whatsoever.  He knew nothing about surgery, knew nothing about the military, but everything about relationships (albeit, not the romantic kind). There was never any doubt as to who really made things happen there, it was all Radar.  He made things run smoothly so the doctors and nurses could focus on healing the sick and wounded.

One last comment on the M*A*S*H analogy.  At the beginning of the series, the doctors did all of the surgery, and the nurses supported them.  Over the course of the series, you would hear surgeons use statements like “OK nurse, close for me.”  And then later, the nurses started doing Triage(determining which patients had to be operated on right away and which could wait) and in some cases even getting involved in the actual surgery itself.  So while everyone kept their specialization, they were able to branch out and help wherever necessary.

Sound familiar?

Posted in Agile Development, Agile Management, Scrum Development, Scrum Management, Software Development Management, Uncategorized | Leave a comment

Get over it already

Is it fair to call Quasimodo ugly?

Earlier this year, on the LinkedIn Agile discussion group, someone posted the question “Is it fair or accurate to malign the waterfall process as is rote when people push Agile and Scrum?”  The original question drew a lot of discussion, then it seemed to die down.  All of a sudden it reared its ugly head again and really requires a more thorough comment.  When I first saw the post, I laughed.  Here was a guy posting in the Agile discussion group a question that implied any negative talk about waterfall was “maligning” it.  Needless to say, the wording implied an assumed answer.  So my first thought was this was a troll, and the usual folks would quickly send the poster on his way, with scorn and derision.

What surprised, and somewhat delighted, me was how many people replied with thoughtful discussion.  Folks took the question, wording and all, seriously.  They started to explore the question of whether Agile had become dogmatic, or perhaps we were being unfair to waterfall.  The conversation was going to help everyone better understand why the world is moving to a better way of creating software, which is what we are calling Agile, and now also Lean.  This is, after all, what discussion groups do so well.  We learned so much through the discussion groups in the early days of agile, as if we were our own Socratic society.

I quickly became disillusioned though.  The discussion was not a serious examination of the pitfalls of the processes known as waterfall, and how to overcome them by moving to agile development.  They were not an open conversation on why, when it comes to software development projects, waterfall is the poorest of choices.  Instead, we saw a lot of folks very nicely saying that waterfall will work just fine if you have all of the information up front.  Or if you have a very clearly defined set of goals, and a strong understanding of the problem domain.  This would lead one to say “gee, if that is all it takes, why learn all these new processes?  Maybe we should just put a lot of time and energy into being able to get all the information up front, or into making sure we have a clearly defined set of goals and ensure they just don’t change during the project?”  Well I’ll tell you why.

***It just doesn’t work***

This is a good time to remember why Agile practices came into being in the first place.  We didn’t all sit down and say “You know, everything is going so well, and projects are really coming in on time and budget, with high quality code.  Let’s change everything.”  This all started because folks realize that projects were failing more often than succeeding.  Most of the projects that were considered success on the outside were the result of long hours and dangerous shortcuts.  Change became something to resist, even if the change was the right thing to do.  We spent a lot of time and energy trying to find ways to change this, usually time and energy that would have been better spent creating software.  Agile is about recognizing the difficulties and complexities of the software world, accepting them, and working in a way that harnesses the ability to change software at a minimal cost.

So no, everyone doesn’t get a trophy.  Waterfall is not a good way to approach software projects.  That’s ok.  There are many projects that are not software projects that would probably do quite well in the waterfall model.  Meanwhile, lets dedicate ourselves to spending this coming year to getting the most out of Agile, especially finding the time to improve the craft of creating the software.  Arguing about whether waterfall is still good is *so* 2011.

Posted in Agile Development, Software Development Management | Leave a comment

Can an Agile Team have Managers?

I have a confession to make.  I’m a manager.  I’ve been a manager for many years.  I’ve led agile development teams most of that time.  I’ve been told I’m a pretty good manager.  And when I look in the mirror, I don’t see any pointy hair.  And as a point of fair disclosure, I believe there actually is a place for managers in an agile team.

Ever since the first days of extreme programming, I’ve noticed a definite view in the agile community that managers become superfluous.  Self organizing teams, coupled with the principles and practices that ease the path toward product and project management, have removed the need for management.  This is a seemingly wonderful sentiment, but it just doesn’t happen that way.  Companies will always have some sort of management structure.  Who those managers are, and what they do, will change considerably though.

There tend to be two types of manager in a software development  organization.  One of those is the project manager.  The project manager is responsible for the successful delivery of the project.  His domain includes status of work items, tracking of progress, and ultimately the schedule of delivery.  As one can imagine, in an Agile development environment this role will change considerably.  About the closest relationship to an agile role would be the customer, or in Scrum terms the Product Owner.  What is truly different though is that the Product Owner is a member of the team, and “status” is not reported directly to or from him.  Many project managers are inclined to continue to track time-lines, and to worry about such things as “percent done” or other metrics that hold little meaning to the Agile community.  I like to remember how much of a change in focus this is, and I do my best to remember how hard such a change is.  The project manager who becomes a product owner is being asked to let of a lot of perceived power and responsibility, and move into the uncomfortable world of “petitioner”.  I say “perceived” because project managers never really had that much control over the course of a particular project, just lots of responsibility.  This is why there was so much emphasis on risk analysis and predictive analysis around the project plan.  We aren’t able to effect what will happen over the course of the project, just plan what our reaction will be to any particular contingency.  Letting go of this will become liberating over the long run, but in the meantime, there is a lot of change.  A lot of this is personal, so let’s give them a break.

The other type of manager is the functional manager.  Teams tend to be organized at some level around the type of job they do.  For instance, there is usually a development manager/director, and another for Quality Assurance. Much of what this manager is responsible is still very important in an Agile organization.  These functional managers tend to be responsible for many of the personnel administration issues, such as hiring/firing, budget, and many of the technology and purchasing decisions.  These traditional activities are important.  The change is the direction such activities come from and where the decisions originate.  The team becomes the primary focus of technical and, sometimes, personnel.  A functional manager is going to be supporting these decisions, providing the “cover” necessary for experimentation exploration.  This won’t change what the functional manager has to provide to the executive team.  The manager is still going to have to represent the progress that the team is making.  The manager is also going to be the one whose career is most affected by the success or failure of any particular effort, especially during a dramatic effort, such as the initial shift to Agile.

So in the end, we want to keep in mind the amount of change that each manager, no matter what their original role was.  Especially during the initial shift to agility, the managers are in extremely precarious positions.  Most of them are involved because they want to make things better, so lets give them a break.  Over time, their jobs may become completely unrecognizable compared to what they used to be, and this is a Good Thing.  In the meantime, they are going to need support in embracing this “new world”.

Posted in Agile Development, Software Development Management | 1 Comment

When is it Good Enough

I’m a woodworker.  I love to work with wood, and I truly enjoy the sense of satisfaction that comes from the work I do.  I think this is why the software Craftsmanship movement rings true for me.  I find many of the aspects of the craft of making things out of wood to fit perfectly with the craft of creating software.  Most of these parallels are fairly obvious, yet they are still quite valuable to explore.  This time, I’d like to relate a lesson that has stuck with me for years now.  As with so many things, it begins with a story…..

My best friend and I like to go to a woodworking show every November.  Some of the most prominent names in the craft give lectures and tutorials, and we get to try out cool new tools.  At one of these, a very well-known writer on  woodworking gave a lecture on  “Best Practices”.  He apparently had not known that he was going to be giving this lecture, and turned it into a bunch of tips and tricks.  He then, about ten minutes before the end of his allotted time, stopped, took off his glasses and looked at the audience.  He said “there’s something else I want to talk about today.  Give yourselves a break.  How many of you have spent hours on hours working on a piece of furniture, only to be upset with all of the  little mistakes along the way.  You finally say you’re done, and while other people are admiring your work, you only notice the one little chisel mark that you just couldn’t sand out.  Look, it is never going to be perfect, and you are going to frustrate yourselves if you can’t accept that.”

So…how do we know what is Good Enough?  How do we know that this piece of software is going to be sufficient?  Well, of course in the Agile world we start by saying we will have automated tests for everything.  Automating those tests up front is of course a great way of driving our design and  a certain level of quality.  The other piece of the puzzle though is to make sure we are not signing up for more than we can handle.  Whether we are using Lean/Kanban or Extreme Programming, we are designing our software process around making sure that we have the time to do what we want to do, without cutting corners.  Far too many problems arise out of  trying to cram that last feature in, or making a compromise on a design flaw because we just don’t have time to “do it right”.  So between Test Driven Development and agile techniques to manage work in progress, we have a nice set of tools that will help us stay on the right side of the quality equation.

But there is one more thing we need to do.  We need to learn to let go of this crazy notion that if we engineer something enough, if we do enough code reviews and enough design sessions, we will be able to make perfect software.  There is no such thing.  We want to make software that works well, doesn’t fail at inopportune times,  which could be as simple as while I’m trying to  write a document or as major as while I am trying to land an F/A-18 on a carrier deck, and that people want to use.  When you see that little chisel mark, ask yourself whether you need to put extra time and effort into fixing it, or is it something small that really doesn’t affect anyone.  So absolutely, make it the best you possibly can, make it beautiful and functional, but remember that until it is  actually delivered to a customer, it is worthless.   Set yourself some parameters around what is Good Enough for this project.  Don’t set ridiculously high standards because it sounds good, but be pragmatic.

And for Heaven’s sake, give yourselves a break.

Posted in Agile Development, Software Development Management, Uncategorized | Tagged | Leave a comment

Managing Technical Debt

This gallery contains 1 photo.

Debt seems to be on every-one’s mind these days.  You can’t open a newspaper anywhere in the world without seeing some article about some country’s national debt.  As we all know, countries aren’t the only entities that have debt.  Most … Continue reading

Gallery | Leave a comment