Todd Sedano

Software Engineering, Improv, Craft of Software Development

Performance Pair Programming

Performance pair programming is when one developer drives the entire implementation while the other watches. It is as if someone is solo programming with an audience. The performer prefers accomplishing a task more than taking advantage of the teaching opportunity.

When this occurs, there is no collaboration or discussion about how to proceed. Questions that the watcher asks are often ignored. Typically the Performer is unaware that their pair is not learning anything.

Since one of the most effective learning technique is “learn by doing,” the pair is missing a great learning opportunity.

The Origins of Collaborative Software Development

Collaborative Software Development is the combination of practices from Extreme Programming, Kanban software development, User centered design, and Lean Start-up.

Kent Beck’s Extreme Programming serves as the bedrock foundation for the practices at Pivotal. The company through Rob Mee experimented and piloted these ideas which formed the DNA for our software development practices. Over time these evolved. 1) The core technical practices remain the same. 2) Managing the backlog evolved from managing commitments for an iteration to just in time scheduling of work. By creating small stories, the need for story to task decomposition disappeared. Iteration planning meeting morphed into bi-directional communication about the immediate work to be done in order to create a shared context for the whole team. This removes unnecessary team task breakdown waste, mid-week checking in on programmers waste. It enables product to change the backlog whenever necessary by shifting stability from the iteration to the story. It enables developers to be present with the work and not focused into the future. 3) While Extreme Programming shifted the definition of features from engineers to product, XP did not specify how product determines these features (See below for more details.) XP enabled the developers to focus on the how and product to focus on the what.

When I first learned about Extreme Programming, it was described to me as “cowboy programming” and an excuse for developers to run amok with little discipline. I have been programming and teaching it since 2005 and can currently cannot imagine a saner way of developing software. I find it incredibly disciplined, applicable to many contexts, and I rely more heavily on it for challenging projects.

Collaborative Software Development’s programming in the present comes from Kanban software’s idea of no iterations. The team looks to design and product to periodically communicate and discuss the next bit of work to be done. Typically this happens weekly or more frequently. Since the team is not focused on a fixed amount of work for a period of time, changes in priority do not necessarily interrupt team productivity. If design discovers a problem with a user interaction, a story to fix the issue can be added into the backlog and prioritized as the next most important thing to be done. Problems can arise if the team has no context on the work to be done, and performing an impromptu shared understanding meeting alleviates this problem. If too much trashing happens, the team needs to understand why there are daily fluxes in prioritization.

When I first heard about no iterations, I thought the idea was crazy. However, having experienced it myself, I appreciate the ability to show up at work and see what is the next important thing to get done.

Collaborative Software Development’s validation of features by users comes from user-centered design and popularized by Lean Startup. The approach prioritizes the user’s needs over perceived business desires and suggests that conflicts between the two should result in examining the business desires in order to align with what the consumer wants. This idea also comes from Lean Thinking which defines delivering only what the user wants as value and delivering anything that the user does not want as waste.

Lean Software Development is not a direct influence on Collaborative Software Development. Several of the ideas of lean thinking for reducing do show up. (Co-located teams, short feedback loops, anyone can stop software development by starting a team huddle.) Just as it is problematic to apply construction practices to software development, so to it is to apply manufacturing practices to software development. In Lean Thinking, most of the attention is on optimizing the production of identical components on an assembly line, not discussing how to optimize the design process of creating new products. While some software products are simply configured, at Pivotal, most it is rare for us to build the same component over and over again. Whenever that occurs, we design it once or refactor the code so that adding more is a trivial exercise. Since the work that we do is not repetitive like an assembly line, applying manufacturing technics to software development can be problematic.

“Renaming the Iteration Planning Meeting”

Summary

I suggest renaming the “Iteration” “Planning” Meeting (IPM) to accurately reflect Pivotal’s current goals and practices for that meeting.

Context

Over time, Pivotal has evolved its own form of Extreme Programming software development methodology by incorporating ideas from Lean Thinking from manufacturing and Lean Startup.

Some of our terminology needs refreshing to reflect the way we currently work. In particular, “Iteration Planning Meeting” does not accurately reflect what typically happens in that meeting.

In Extreme Programming (1999) and Extreme Programming 2nd Edition (2004) the development team commits to the stories to be accomplished during an iteration or weekly cycle just like a sprint in Scrum. The commitment can change as the week unfolds. Once Product identifies the stories for the week, programmers decompose the stories into tasks. After volunteering for a set of tasks, each programmer estimates only their tasks on Monday during an Iteration Planning Game (XP 1999) or Weekly Cycle meeting (XP 2004.)

In contrast, Pivotal follows a “continuous flow” where the work “flows” through people as people start the next needed story in the backlog. We limit work in progress to a story per pair of programmers. At Pivotal, we’ve customized the Extreme Programming process so that the developers accomplish the work that needs to get done at that moment in time. At any point in time, we can not predict who will be working on what work tomorrow, but we can say who is working on what work now. As that work is finished, we simply start the next unit of work to be done. We do not commit to which stories will be done in any given week, instead we follow a Lean Thinking’s just-in-time mechanisms for identifying what to work on next.

At Pivotal, stories and tasks are not owned by specific individuals. In Scrum and Extreme Programming, one person owns the task for the duration of the task. At Pivotal, many people might work on a single story. While the team notes who is working on each story right now, the team understands that these associations will change. After finishing a task, a development pair simply works on the next available story.

We’re familiar with the multi-day story: On Monday, Bailey and De begin working from the top of the backlog. On Tuesday, De and Kadeer continue on the story. On Wednesday, Kadeer and Lesley finish the story. For the participants, this can feel like a relay race. The entire team might even rotate through a complex story.

Pivotal prefers very small stories in order to deliver value quickly, remove engineering waste, shorten the feature feedback loop, and support iterative and incremental development. The smaller stories often do preclude the need to explicitly list out any tasks. It is rare for a pair of developers to need to list out the tasks associated with a story. Usually, the pair can effortlessly figure out what needs to get done.

We do not follow a Scrum Sprint, or an Extreme Programming Iteration or an Extreme Programming Weekly Cycle. We’ve successfully incorporated Lean into the way we prepare and execute our work.

The concept of “week” describes the team’s work rhythm

The team’s rhythm aligns with the natural rhythm of a work week. Retros are scheduled on Fridays at the end of the week. The meeting signifies closure for the week. There are practical reasons to schedule the meeting on a Friday. After a weekend off, the team forgets the details of the previous week. Just watch the Monday stand-up for confirmation as team members struggle to recall their stories from Friday.

The team schedules a “shared understanding” meeting once a week in order to 1) share context from product and design with the engineers. While the engineers are busy implementing stories, product and design are looking into the future work, specifying what needs to be done. This sharing of context helps anyone on the team pick up any story in the backlog. 2) The meeting provides feedback from enginering to product about the complexity and risk of the work. A story that seems simple to product might be expensive to impement. This emergent conversation allows product an opportunity to course correct to a simpler solution.

The meenting typically covers enough stories to make sure that we have enough work in the backlog. Teams typically schedule them on Mondays, Tuesdays, and Wednesdays since any day will do. (This is unlike an Extreme Programming Iteration where these need to be scheduled at the beginning of the week so developers know who is working on what.) If the backlog was extremely full, we could just cancel the meeting. Just because the team’s natural cadence aligns with the week does not mean we are doing Extreme Programming iterations.

The next time you find yourself using the word “iteration”, see if the word “weekly” would do instead. “What is our velocity this iteration?” becomes “What is our velocity this week?” and “How many iterations are left in this project?” becomes “How many weeks are left in this project?” The term iteration is a proxy for measuring time. Since we aren’t committing to work to be done in an iteration, our usage of the terms are interchangeable.

In communicating our process to potential clients and the outside world, I’d prefer to be accurate in describing our work flow as a Lean just-in-time system. Using the term “iteration” potentially miscommunicates that we are committing to specific work each week.

Updating our terminology at Pivotal

Since Pivotal has evolved its own form of Extreme Programming to incorporate Lean’s just-in-time practices, I suggest we use the accurate terminology in describing the way we work.

Consider our weekly meeting for creating shared mutual understanding and pointing the next stories in the backlog. Ironically, the meeting is called an “Iteration Planning Meeting” even though we do not commit to which stories we are doing next week and no explicit planning happens in the meeting. While we do point stories in the meeting, examining whether we are on track, re-prioritizing stories, and assigning work to individuals all happen outside the meeting. For us, planning happens at the product manager, client liaison, and client level as Pivotal discusses whether the project is on track and whether key features will be done by a certain date.

During our “Iteration” “Planning” Meeting, it is entirely possible that the stories we do discuss might happen two or even three weeks from now, not during the next “iteration.”

When I’m anchoring a project, my chief concern with our “Iteration” “Planning” Meeting is to make sure that our backlog has enough pointed stories for the next week. The team running out of work would be disastrous.

Finding a suitable replacement name for this meeting is challenging as many of us have different, sometimes conflicting expectations and goals for the meeting. Here is what I have found n interviewing 10 pivots and product managers from four offices.

Common goals for the meeting: 1) Create a shared understanding of the work that is to be done. (Where are we headed? What will we be building? Are we building the right system?) 2) Communicate to the PM risks and story complexity. We typically use estimation as a forcing function to have this conversation. 3) Estimate the work 4) Verify that each story has clear acceptance criteria

Goals 1 and 2 are frequently cited in my interviews. Some pivots argue that we should not be pointing and thus goal 3 is not a goal for them in the meeting. Goal 4 is infrequently mentioned.

We could call the meeting a “Pointing Meeting” (PM) which unfortunately has the same abbreviation as Product Manager (PM). In discussing the goals of the meeting with numerous Product Managers, Anchors, and Pivots, I personally prefer the term “Shared Understanding Meeting” as it reinforces the goal of getting everyone on the same page.

Updating Pivotal Tracker

Pivotal Tracker might minor updates to its messaging. Pivotal Tracker describes each week as an “iteration” and has charts that talk about “iteration velocity” (aka weekly velocity) “iteration start date” (aka week start date). Fixing this would be a simple string replacement. Fortunately, many of the charts simply reference “Date” instead of “Iterations.” The terminology seeps more into the Help documentation than the tool itself. Keeping the iteration terminology in the help documentation makes sense given that other companies use tracker, and they might have sprints or iterations. Since the weekly markers are not labeled as iterations, one could use tracker for quite some time without knowing that the tool reinforced the Extreme Programming iteration concept from 1999.

My own A-ha moment

In the movie, the Matrix, Keanu Reaves follows a discovery and evaluation process to determine if he is “The One.” While in a waiting room, he sees a boy bending spoons and wonders how that is possible. The secret is that “there is no spoon.” I had a similar revelation while working on a Pivotal Labs project. In the middle of the work week, I was reflecting on our rhythm of weekly retrospections and weekly pointing meetings and realized that there were no iterations. We did not commit to explicit work each week. Every day felt very much like the next day. We weren’t ramping up on new stories on Monday to wind them down on Friday. Instead, we were present with the features in the backlog. We weren’t planning for or committing to which stories would be done in each week. I mentioned this to my pairing partner and he said, “Of course there are iterations, each Monday we start a new iteration!” The anchor overheard this conversation and said, “Actually, since our project started on a Tuesday, I setup Pivotal Tracker to have iterations go Tuesday to Monday.” With three developers on one team, we had widely different opinions of whether we had iterations and when they started. This conversation reinforced my observation that we were not using iterations as prescribed by Extreme Programming.

Conclusion

I recommend we replace “Iteration Planning Meeting” with a phrase that accurately describes the meeting’s purpose and goals. I suggest “Shared Understanding Meeting” as a simple draft proposal which is intended to encourage discussion about its disadvantages or the purpose of generating better proposals.

Update 5/10/2017: I now suggest “Story Discussion Meeting” or “Story Estimation Meeting”

Collaborative Software Development

I’m still editing this blog post and it is subject to change. While what I want to say isn’t changing, the words that I use to say it are changing.

Collaborative Software Development

Collaborative Software Development is an evolution of Extreme Programming method with iteration-less development. Collaborative Software Development starts with all of the practices of Extreme Programming (XP) and replaces the concept of iterations with Lean Thinking’s just-in-time production philosophy manifested in pulled-based practices. This achieves a high quality, extremely readable, and well designed software with a faster ability to respond to change.

Collaborative Software Development continues to rely on all the technical practices of XP including pair-programming, test-driven development as well as the remaining process practices of XP including on-site customer.

Commitment level shifts from the iteration to story

While there is a commitment to the story in-flight, the developers with the product manager may realize that the story needs changing. While working on a story, the developers might discover a corner case not considered by the product manager, or find a confusing user interaction flow that requires the user experience to be re-worked. For these emergent and “internal” changes, the story needs updating. (reduces the waste from more design-up-front approaches. This is an advantage of the process.) For “external” changes where the product owner changes the feature, typically the product owner creates a new story that reflects the new insight into the feature. From this perspective, there is a commitment between the product owner and the team that the stories written reflect our current understanding of the system and will not be re-written by product.

By switching from an iteration based to a just-in-time workflow, the commitment level changes from the iteration to the story. In Extreme Programming, there is a weekly contract between the product owner and the development team that the work for a particular week is fixed. Anything can change about the project, as long as the changes occurred between iterations. With Collaborative Software Development, the product owner is committing to the completion of any story in flight since it was at the top of the backlog when available developers pick it up and start working on it. In practice, product owners do not cancel in-flight stories. Stories might become blocked and be un-started. Theoretically it is possible that the direction of a project changes so drastically that you might stop working on a story, but in practice this is extremely rare.

Being in the moment

In this environment, developers learn to be flexible when showing up at work, just like actors performing improvisation without a script. Imagine a developer with expectations about their work for the day. At the beginning of the day, they pair up, and the pair may discover that the team’s needs will cause the developer’s expectations to be misaligned from reality. Developers learn to loosely hold any expectations and embrace what emerges in the day. Developers can be pleasantly surprised by what emerges. “Acceptance is the answer” to many of life’s problems. A developer can express their desires to the team. Examples include “I haven’t worked on this part of the code base recently” or “I haven’t paired with this person in awhile” or “I really enjoy working on UI stories.” However, if the build is broken, or the product manager needs help in accepting a story, then the reality of the moment will trump any of the developer’s plans.

Improvisors practice letting go of planning through exercises that reinforce handling of chaotic situations, rewarding spontaneity, and listening. In Lean XP, we learn to listen to our pairs, our tests, our code, our stories, our product owner, and our users.

“You Aren’t Going to Need It” (YAGNI) encapsulates part of this idea. Instead of worrying about future features, let’s focus on the current system and implementing the current feature. At its heart, its about being in the present with the code. By not committing to stories for each week, it shifts the conversation. Instead of discussing what was done last week, or what will be done next week, the team can focus on what needs to be done now.

Pros

This approach encourages flexibility in the developers. I don’t know what I’m working on today. If I do have plans, I need to be willing to let go.

This approach allows for quick re-distribution of the work. As developers finish a story, they will pick up the next important one. The PM can re-prioritize the backlog at any point and know that the most important story will be worked on as soon as a pair finishes their current work.

If everyone was committed to a task within a sprint, then explicitly coordination is required to determine which tasks should be postponed and dropped. Dependencies on those tasks could also be affected.

This simplifies the plan of work for the week Some engineering managers maintain intricate mappings of who is working on what and who is dependent upon what. If any one developer slips on their work, there is other work that is also delayed. Lean XP shifts the focus from developer dependency to story dependency. Stories that block other stories can be explicitly labeled. When the first story finishes, the second story can be unblocked and worked upon when it reaches the top of its backlog. There is no “chess” game of resource allocation and dependency management.

How is a SE kanban board lean? Instead of tracking about all of the requirements of a product, we can just track the stories in the next iteration or the features in the next release?

OS X Filesystem and Case Sensitive Bugs

Problem:

On one project, our test cases would be green locally, but then fail on CI due to filesystem case sensitivity differences between OS X (our development environment) and linux (our QA and production environments.). By default, OS X treats ~/project/myFile as equals to ~/projectMyFile. Thus if the code had a typo on an import, it might pass locally when it shouldn’t.

Solution:

I wanted to use the base pivotal OS X image and sprout wrap. The strategy is to create a case sensitive partition and symlink the workspace directory to it. Sprout wrap has issues with case sensitivity (pivotal_ide_prefs in particular), so we would run that from the user’s home directory, but run all project materials in the case sensitive directory.

Alternative solution: getting the pivotal OS X image on a USB key might allow us to configure the default partition’s setting on installation.

These steps could work with an existing machine, but I only tested on a clean install

Step 1) install the Pivotal Labs OS X image

Step 2) using Disk Utility, split the only partition in half. For the new partition select “Mac OS Extended (Case-sensitive, Journaled)”

Step 3) create a workspace directory in the new partition mkdir /Volumes/CASE_SENSITIVE/workspace

Step 4) symlink ~pivotal/workspace to the new partition’s workspace directory. ln -s /Volumes/CASE_SENSITIVE/workspace /Users/pivotal/workspace

Step 5) install sprout-wrap from ~, NOT from ~/workspace. I noticed that the pivotal_ide_prefs wouldn’t fully work with case sensitive on. (There may be other recipes that have issues too.)

Step 6) install your project code in ~/workspace

Note that everything in Applications et al. will be typical Mac Setup, but we were fine with that.

Estimated vs Actual Story Points

On my last project, I tried an experiment looking at estimation accuracy.

My results run counter the conventional wisdom of software engineering research and experience, but are consistent with my experience at Pivotal. Conventional wisdom says that engineers are optimistic and horrible at estimating work which is why some managers “double” estimates given to them.

On my last team, we tended to be cautious in estimating work and not overly optimistic about the risk and complexity of our work. This is similar to the “under commit, over deliver” adage.

Here is how we went about collecting the data 1) We limited our pointing scale to “Easy”, “Medium” and “Hard” — one of the pivots advocated this scale and I liked the simplicity of it. We mapped “Easy” to 1 story point, “Medium” to 3 story points and “Hard” to 8 story points. In the meeting we would hold up 1, 2, or 3 fingers, I would call out Easy/Medium/Hard and the PM would record the correct story points.

While I don’t have evidence for this, I felt that the IPMs were very efficient as there was less quibbling over minor point differences. We went with the majority point value. If a large number of people said Easy and a large number of people said Medium, we’d have a discussion. If most of the team said Easy and a few number of people said Hard, we’d have a discussion.

2) At the end of each story, the pair would assign a Pivotal Tracker label to reflect the actual point value. We used the labels “Actual Easy Points” “Actual Medium Points” “Actual Hard Points”, eventually we had to add a “Actual Zero Points”. I asked the pair not to look at the estimate on the story while recording the actual, but there could be some anchoring bias with the data.

3) I monitored tracker and reminded developers to put labels on stories they had finished but had not labeled.

Periodically, I would show results to the team. (After 1 month, 2 months, and end of project.)

  We were… (number of stories)
Estimate Conservative Accurate Optimistic
0 5 1
1 7 48 5
3 13 14 2
8 0 2
Estimate Conservative Accurate Optimistic
0 83.33% 16.67%
1 11.48% 48.28% 6.90%
3 44.83% 48.28% 6.90%
8 0.00% 100.00% 0.00%

Frustration Quote

I am frustrated when my expectations do not align with reality – Todd Sedano

Rethinking SEMAT Card Affordance

A possible solution

While I’m an avid player of card games and board games, the SEMAT card format does not reflect how I think about the alphas, which are a collection of states.

Here is a mock prototype of an alternative physical format for the SEMAT alphas.

Each alpha is a strip of cards folded much like a scroll with the “highest” state on the inside, and the lowest state on the outside. Starting with the “lowest” state, the user of SEMAT, can incrementally unfold the strip comparing the current state with the next possible state. If the next possible state is achieved, then the user can continue to unroll the strip.

Now it is impossible to accidentally loose a state in the alpha, and displaying the current state for all alphas in a project takes up roughly 1/6 of the room of the SEMAT board.

So what do we call these new SEMAT cards? SEMAT strips, SEMAT rolls, SEMAT scrolls? I’m open to suggestions.

Background — the problem

When I first saw a set of SEMAT cards, my instinct told me something wasn’t quite right. I collect playing cards. I like unusual cards sets such as my agile estimation cards, XP training cards, improv feeling cards.

When I laid out six cards for an alpha, it felt messy. I could easily get these out of order, and the order matters in a single alpha. If I piled up several alphas without a rubber band, I could easily mix them together.

I pictured myself introducing this at a training session. With agile estimation cards, I just hand out a deck, yet for SEMAT I would want to hand one alpha at a time. If SEMAT cards were printed in a deck, taking out one alpha at a time would be time consuming.

I do think agile estimation cards work well. There are four sets in one deck, just like regular playing cards. Assuming that my five year old daughter found the deck and randomized it, sorting it wouldn’t take too long with four sets. However SEMAT cards, there are many alphas, and sorting it would be tedious.

Abacus as an alternative metaphor

I stared considering an abacus. Each rod of the abacus could represent an alpha. Each bead on a rod could represent a state card. The space between the beads on the left and the beads on the right could represent the current state. Yet creating an abacus for SEMAT seemed unfeasible. Then it occurred to me, I could tape the SEMAT cards into a strip.

Improving Code Readability – Turning Comments Into Methods

I’m working with 21 developers to improve their code readability. Through a code read-through, they listen to another developer try and read their code. (See Code Readability Process for more details.)

In reviewing one programmer’s code, a sixty line method had a visual rhythm to it. There would be a blank space, a comment, then about ten lines of code, and the cycle would repeat. The comment would explained the code just following it.

The programmer realizes that the narrative is lost in their code, and feels compelled to add these comments to help the reader understand what is going on. These comments serve as section breaks or chapter headings.

Instead, the code could be split up into smaller methods, where each method name would clearly revel the intent of the code. The comment would be better served as a method invocation.

Here’s the pattern

1
2
3
4
//determine interest rate  (comment about the code intention)
code
code
code

Becomes

1
determine_interest_rate()

Here’s the original code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
    bool Tictactoe::determine_game(int row, int column, char move){
        bool flag = true;
        int i;

        // Check the row of latest play
        if(!flag){
           flag = true;
           i=0;
           while((i<dimension) && (flag==true)){
               if(board[row][i] != move){
                   flag = false;
               }
               i++;
           }
        }

        // Check the column of latest play
        if(!flag){
            flag = true;
            i=0;
            while((i<dimension) && (flag==true)){
                if(board[i][column] != move){
                    flag = false;
                }
                i++;
            }
        }
        ...
        return flag;
    }
  

Here’s the revised code.

1
2
3
4
5
6
7
8
9
10
     bool Tictactoe::determine_game(int row, int column, char move){
        bool winner_found;

        winner_found = check_row_of_latest_play(row) ||
                       check_column_of_latest_play(column) ||
                       check_...

        return winner_found;
    }
  

(Note that I have not shown other refactoring that I would do, as it would distract from the point.)

By changing the comment to a method call, the code is now more “self documenting” and the intent is clear by the method call.

When I suggested this to the programmer, she resisted the idea noting that a method call would affect performance. For a tic-tac-toe problem, this is a specious argument. However, is there merit to it? Will a modern compiler optimize this kind of refactoring? And this brings up a broader question, should we optimize code for performance or readability when we are writing it? Conventional wisdom says we should write code that is clean and easy to understand, and when we are done and have performance analysis with production data, then we know where to spend engineering effort to optimize critical sections. The one exception would be algorithm complexity and running times. (e.g. O(N) vs O(NxN)

I’m now curious, is this a “comment smell”? Can comments be indicators to us programmers that the code we just wrote isn’t very clear. The comment itself my inform us on how we need to refactor the code to make it more readable.