Todd Sedano

Software Engineering, Improv, Craft of Software Development

Illusion of Requirements

Title: The illusion of requirements; the requirements myth

“Requirements” is a pre-agile construct based on our false belief that building software was somehow like the construction of buildings.

“Requirements” limits the development team into thinking the end product is fixed and that the team’s goal is to implement a specification.

Agile development teams expect features to change over time. Even with thorough user research and validation, the product will change as soon as we receive user feedback. Even during the implementation of a user story, an emergent conversation on the team might change its acceptance criteria.

In contrast, features are a construct of preferences. Many preferences may stay the same, some will change over time, and most are made up on the spot Lichtenstein and Slovic 2006.

A simple example

Let’s consider a simple and common feature request: “the user logs into the system with a username and password.” If we consider this request to be fixed, we might not consider alternative solutions.

  • Some websites support magic links. You don’t type in a password, the system emails you an authentication link.
  • Some websites require two-factor authentication.
  • Some websites rely on a third-party single sign-on such as Google, Facebook, or Okta.

Several of these solutions also attempt to solve the problem of users creating easy to guess or easy to break passwords.

Treating authentication as a “requirement” misses these opportunities.

System characteristics

How then do we deal with performance, scalability, availability, recoverability?

Pre-agile has the notion of “non-functional requirements” which describe system characteristics or product quality characteristics.

Performance

Before agile, I wrote my share of statements like “the web page should load in under 0.5 seconds.” In reflection, my non-functional “requirements” were arbitrary limitations on the system.

Here’s the typical workflow for agile software development. * Conduct user research and validation to determine appropriate feature for the opportunity * Build a minimal implementation of the feature * Verify that the feature matches the opportunity, if not iterate * Improve performance where needed This is a variant of “Make it work, make it right, make it fast.”(http://wiki.c2.com/?PrematureOptimization)

Availability and Recoverability

Many of the system characteristics such as availability can influence Service Level Agreements. Pre-agile treats these as guarantees. If we build the system just right, we can guarantee that we will achieve a particular outcome. Agile treats these as goals. If we want to prove ourselves that the system handles disruptions, then we introduce disruptions much like Netflix Chaos Monkey. We keep iterating on the product and deployment process until we achieve the goal. Adding Chaos Monkey to a system could be considered a “non-functional story” or a chore, since it provides no user facing functionality yet improves the product’s quality.

Avoid pre-agile constructs

It can be hard to remove bias from pre-agile constructs. If you see yourself or your team using the following words, pause and reconsider your options: * specifications, * requirements, * the system shall…

In Summary

Avoid limiting yourself with pre-agile concepts. Recognize the flexibility of features in building software.

Here’s an academic paper demonstrating the negative impacts of framing potential system functionality as “requirements.”

For more information about the construction of preferences see http://www.cambridge.org/id/academic/subjects/psychology/cognition/construction-preference

Considerate Pair Programming - Part 1

Why We Pair

Hi. I have come to love Pair Programming. I enjoy the collaboration, cross-training, and better solutions that emerge while I pair.

Continuous Pair programming with Overlapping Pair Rotations removes knowledge silos to increase teams’ resiliency to team disruption [reference] and builds collective code ownership [reference] as described in Sustainable Software Development.

Pair programming is a complex practice requiring skill to master. Although Pair Programming has been around for 20 years and is well researched, some teams simply drop engineers onto a team expecting them to figure Pair Programming out.

As a practitioner, I am still discovering ways that I can improve the pair programming dynamic.

This article describes 1) several techniques that can improve your pair programming experience, 2) strategies on what to do when things get frustrating or go wrong, and 3) concludes with success stories of converting difficult pair-programming relationships into successful pair-programming relationships.

Like any relationship, such as friendships, family dynamics, dating, or marriage, when pairing works well, it is an amazing experience; when pairing falls apart, it can be frustrating and occasionally super hurtful. While unfortunate dynamics can happen while pairing, I have experienced the joy of finding each other in the rift.

Pairing exposes both our strengths and our weakness. How we handle our weaknesses may have a direct impact on the pairing relationship. One of us might not be comfortable exposing our weaknesses. If we are experiencing imposter syndrome, feeling that we are not qualified for the job we have, we might fear that our partner is going to find out that we are faking it.

I have been on a journey to be a better pair. While at Pivotal over 3.5 years, I have paired with ~39 pivots and ~ 20 client developers. I am grateful to each person who has helped me better understand myself and have helped me assemble and create these techniques. If you paired with me and I did not apply the technique, it may be because I learned it later.

Considerate Pair Programming - Part 2

Strategies for Success in Pair Programming

In my software development practice, I rely on several techniques for increasing the probability of success with a pairing relationship.

This article contains a toolbox of techniques. Feel free to pick a few to try out and then revisit the list in a few weeks.

Become invested in your pair’s success is the major take-away. When I am taking care of myself and looking out for my pair, an incredible friendship can form. Consider “what are my pairs strengths? And how do they map to my weaknesses?” and vice-versa. Have I always been invested in my pair’s success? No. My pairs probably have stories on how I could have improved in my pairing. Shifting from “what is in it for me” to “what is in it for us” enables new pairing dynamics.

Considerate Pair Programming - Part 3

When Pair Programming Goes Wrong / Weird

Since pairing is a relational practice relying on imperfect communication, misunderstandings and slights can emerge.

What could go wrong with pairing? Here are some situations that have bothered me.

  • You come across some code that is difficult to read. Your partner tells you not to refactor the code.

  • Your partner dominates the keyboard for most of the day.

  • Your partner does all the work without involving you.

  • Your partner tells you how horrible is the code that you helped work on the previous day.

  • You discuss a situation and agree to a plan, but then your partner starts doing his / her own thing anyway.

In this section, I list out several strategies for coping with a situation that is getting tense, de-escalating a situation that is tense, and how to re-engage with your partner after a tense situation.

Sustainable Software Development

There are several ways in which Sustainable Software Development has evolved from Extreme Programming (XP)

Sustainable Software Develpoment

  • combines user centered design with agile software development. Product designers apply design thinking and user research to understand what product and features to build. Extreme Programming relies on an onsite customer to tell the developers what to build. This is problematic in two ways. Often an onsite customer is unaviable, impractical, or too expensive to have on a team. Onsite customers are not trained in design thinking and information architecture.
  • reduces the commitment level of the team to the work in progress, the stories that each developer is currently working on. Unlike Extrmee Programming, the team does not commit to an amount of work to be done during a weekly cycle or iteration. There are no iterations in Sustinable Software Development.
  • focuses the technical practices on two main goals
    1. caretaking the code
    2. removing knowledge silos
    Extreme Programming has many practices around improving code quality, but removing knowledge silos is not an explicit goal.
  • replaces predictions with focusing on “what is the minimal that we need to ship?” Planning Extreme Programming utilizes the team’s velocity and backlog to create burn down charts for forecasting the team’s progress. Like Extreme Progamming, the team maintains a shippable product.
  • identifies and removes software development waste. Weekly retros identify wastes and the team removes wastes with action items and chores.

Both Sustainable Software Development and Extreme Programming deal with the tension of dates and features by asking “how do we reduce scope?” and “what should be done first?” In other words, developer happiness and sustainability is achieved by prioritizing the work.

This table compares Sustainable Software Development to Extreme Programming (XP). A more detailed analysis follows after the table.

Sustainable Software Development Extreme Programming 2.0 (XP 1.0 names)
Balanced Team Whole Team (Balanced Team)
Track One Practices
User research Real Customer InvolvementC (On-site customer)
It is impractical to have an onsite customer.
User validation
Road Mapping Quarterly Cycle (Planning Game, Small Releases)
Story writing Stories
Story refining
Project startup
Track Two Practices
Caretaking the Code
Clean code Code and TestsC
Fast test suite and builds Ten-minute Build
Test Driven Development / BDD Test-First Programming
Team code ownership Shared CodeC (Collective ownership)
Continuous refactoring Incremental Design (Simple Design Refactoring)
Integrate frequently Continuous Integration and
Single Code BaseC
branches live a few hours at most
Frequent releases
Each team releases when features are done or at a given cadence
Daily DeploymentC
Acceptance / staging environment (implied)
Build system (implied)
Stop on red builds (implied)
Remove Knowledge silos
Continuous pair programming Pair Programming
Overlapping pair rotations N/A
Boundary Spanning Practices
Backlog grooming N/A
Accepting stories (implied)
Story showcase meeting Weekly Cycle (Iteration Planning Meetings)
Project Management Practices
Core work hours Energized Work (40-hr week)
Stand-up N/A
Retro N/A
Flexible scope contracts Negotiated Scope ContractC
No overtime Energized Work (40-hr week)
Shared Schedule, Be Present Sit Together
Not needed
Agile project management tools (such a Pivotal Tracker)
provide more value than a physical board showing project status.
Informative Workspace
Not needed
there is no commitment to each week
Slack
Not needed for all projects Incremental DeploymentC
slowly replace a legacy systems
Allocations
Move engineers where needed
Shrinking TeamsC
Team ContinuityC
Optional, depends on situation Root-cause analysisC
Pay-per-useC

C are optional XP 2.0 Corollary Practices

For a team transitioning to Extreme Programming, Beck recommends starting with the primary practices. Beck claims that “the primary practices are useful independent of what else you are doing” resulting in an immediate improvement, whereas the corollary practices work well once a team has implemented the primary practices. Extreme programming is not dogmatic about the list of practices. The team chooses which practices to apply.

Once a team has implemented the primary practices, then the team can start implementing the corollary practices. Beck says that it seems “difficult or dangerous to implement [the corollary practices] before completing the preliminary work of the primary practices” [1].


Sustainable Software Development Extreme Programming 2.0 (XP 1.0 names)
Balanced Team pulls together various disciplines into one collaborating team. The team works together to accomplish a common goal. /td> Whole Team (Balanced Team) Whole team is also known as cross-functional teams. Include people on the team with the skills needed for success. Beck points out that a sense of team fulfills an individual’s psychological needs. “We belong. We are in this together. We support each others’ work, growth, and learning.” [1]
Track One Practices
User research includes writing interview guides (questions), interviewing users and synthesizing the results. Real Customer InvolvementC (On-site customer) Make the customer a part of the team. “The point of customer involvement is to reduce wasted effort by putting the people with the needs in direct contact with the people who can fill those needs” [1]. Even if a customer is busy, involving them whenever possible is preferred to not involving them at all. In this situation, the team needs to find alternative ways of validating features with users.
It is impractical to have an onsite customer.
User validation checking whether ideas for the product resonate with users.
Road Mapping the product manager diagrams his or her long term vision for the product. Quarterly Cycle (Planning Game, Small Releases) Each quarter, the team identifies the stories for the next quarter. During a meeting, the team identifies bottlenecks, initiates repairs, plans the feature sets for the quarter, and selects a quarter’s worth of stories.
Story writing literally writing stories that represent potential product features. Stories Stories are “units of customer-visible functionality” [1]. Stories have a name and a description or graphical depiction. Stories track the work to be done.
Story refining generating small, manageable stories from larger, epic stories.
Project startup a special period of intensive user research and limited coding, typically taking four or more weeks at the beginning of a project or during a major shift in direction.
Track Two Practices
Caretaking the Code
Clean code means writing code that is easily understood, easily changed and complies with coding standards the team has adopted. Keeping the code clean requires continuous refactoring. Code craft encourages intention revealing names and discoverable code. Code and TestsC The tests and code are the only permanent artifacts. Everything else (e.g. documentation) is generated from the code. Anything that contributes to what the system does today and what it will do tomorrow is valuable, “everything else is waste” [1].
Fast test suite and builds means that the tests execute as quickly as possible. A fast test suite provides speedy feedback to the team when something breaks. Ten-minute Build Building the code and running the test suite should be automated and short. Longer builds increase the feedback loop.
Test Driven Development / BDD has the engineers incrementally build the software by always writing a failing test, then writing just enough code to get that test to pass. Test-First Programming The developers write tests before writing code. Every production line of code is preceded by a test. Test-first programming addresses several concerns: scope-creep, coupling and cohesion, trust, and rhythm. The practice mitigates against scope-creep; without a clear goal, a programmer can easily write more code than is necessary. The practice improves coupling and cohesion as easy-to-test code often has beneficial design qualities. The practice builds trust in the code as engineers trust tested code more than un-tested code. The practice produces rhythm as it provides a cadence of accomplishing goals.
Team code ownership is the ability for any developer on a team to change any of the team’s code. Shared CodeC (Collective ownership) Everyone on the team can modify any of the system. When engineers sees something broken, they fix it.
Continuous refactoring Incremental Design (Simple Design Refactoring) Build the system incrementally and evolve the design every day. In the past, software teams might often had long design phases that assumed features would not change. When the team’s understanding of the system changes, make incremental changes to the system to align the code’s design with the team’s desired goal. Teams routinely remove duplication.
Integrate frequently Continuous Integration When working on new features, routinely integrate new code into the master code repository. Avoid long-lived branches. Beck suggests un-integrated code live on its own “no more than a couple of hours” [1]. The longer the team waits to integrate, the more unpredictable and expensive integration becomes.
Single Code BaseC Ideally there is only one branch of the code. Multiple branches, (e.g. for releases, or customers) creates extra work. Branches live a few hours at most. branches live a few hours at most
Frequent releases means shipping or deploying the product frequently. For a website, “frequent” might mean several times a day; for enterprise software, “frequent” might mean once a month.
Each team releases when features are done or at a given cadence
Daily DeploymentC Every day put a new version of the code into production.
Acceptance / staging environment means that the product manager accepts a story in another environment that closely resembles the user’s environment. For installed software, this is another machine or cluster. For web development, this is a staging server. (implied)
Build system is for continuous integration or continuous delivery. The build monitor indicates the status of running all the tests with the latest delivered code. (implied)
Stop on red builds means that when a build fails (the build monitor turns red), one pair immediately stops what they are doing and fixes the issue. This is not necessarily the pair that caused the build to fail. Shaming whoever introduced the change to cause the build to break is antithetical to the goals of sustainable development. The occasional failing test is not a big deal, the team handles it and moves on. (implied)
Remove Knowledge silos
Continuous pair programming refers to two engineers co-creating source code and tests on pairing workstation with two displays, two keyboards, two mice, and identical development environments. Other models of pair programming exist, but this setup reinforces the equality of the relationship. Pair Programming Every part of a production system is written by two programmers sitting side by side. The pair works together to solve problems, keep each other on task, learn from each other, and abide by the team’s practices.
Overlapping pair rotations means that, each day, engineers change pairs such that over time, everyone works with everyone else. One member of each pair always retains work in progress from the previous day. N/A
Boundary Spanning Practices
Backlog grooming the product manager routinely re-sequences the stories in the backlog and verifies that stories at the top of the backlog are ready for engineers to start. N/A
Accepting stories after the team delivers a story, the product manager verifies that the delivered work achieves the acceptance criteria. The product manager tries to break the feature with common unexpected input. (implied)
Story showcase meeting a 60 minute weekly meeting to discuss stories that the team will be working on in the immediate future. The product manager describes upcoming stories and the engineers provides feedback on the expected difficulty of each story. The story showcase helps improve the team’s shared understanding of the product. Weekly Cycle (Iteration Planning Meetings) Each week, the team accepts a certain amount of work to accomplish. During a meeting at the beginning of the week, the team reviews progress to date, the customer chooses a week’s worth of stories, and the engineers decompose the stories into tasks and individually estimate each task. “The goal is to have deployable software at the end of the week which everyone can celebrate as progress” [1].
Project Management Practices
Core work hours the team prioritizes working together in the same office during the same hours. Flexibility with the schedule helps with outside of work constraints provided that the team has enough time in common. Energized Work (40-hr week)
Stand-upa 5-10 minute, daily meeting where team members discuss changes that affect the team from the previous day’s work and impediments to the current day’s work. If physically possible, people stand to encourage a short check-in. Usually this meeting is at the start of the day, otherwise it is a daily spin-down meeting.. N/A
Retro a 60 minute weekly meeting in which the team re- flects on the week, celebrates successes, and identifies ways to improve. The ideal retro is a safe environment where a team can discuss any topic constructively. N/A
Flexible scope contracts establish an ongoing relationship between a development organization and a client organization without unnecessary details. It might state that the client owns the code, that new releases are expected at most every two weeks, or the daily rate at which engineers are billed. It typically would not state a fixed scope, schedule or budget. If a schedule or budget is fixed, scope is not and vice versa. Negotiated Scope ContractC< “Write contracts for software development that fix time, costs, and quality but call for an ongoing negotiation of the precise scope of the system.” [1]
No overtime no one is allowed to work overtime except in genuinely exceptional circumstances, such as solving a customer escalation. Energized Work (40-hr week) “Work only as many hours as you can be productive and only as many hours as you can sustain” [1]. Working longer hours may make the team less productive. Team members need to take care of themselves; this includes staying home to rest when sick. The work patterns of the team should be sustainable in the long run.
Shared Schedule, Be Present Sit Together Software development is a collaborative endeavor. Co-locating teams increases collaboration. Distributed teams are still possible, yet “Sit Together predicts that the more face time you have, the more humane and productive the project” [1]
Not needed
Agile project management tools (such a Pivotal Tracker)
provide more value than a physical board showing project status.
Informative Workspace “An interested observer should be able to walk into the team space and get a general idea of how the project is going in fifteen seconds” [1]. The team can modify the workspace to make it more conducive for their work.
Not needed
there is no commitment to each week
Slack During the weekly cycle meeting, include tasks in the plan that the team can drop if the team gets behind. Adding slack to the schedule enables a sustainable work environment.
Not needed for all projects C Switch between a legacy system and a replacement system in incremental steps. Avoid a hard switch over from a legacy system to a new system.
slowly replace a legacy systems
Allocations
Move engineers where needed
Shrinking TeamsC Try to improve team efficiency so that the team can shrink in size. When a team becomes more productive, re-allocated extraneous team members to other teams.
Team ContinuityC “Keep effective teams together” [1]. People are not fungible, and building relationships is part of creating a high performing team. Rotating people through teams helps spread knowledge.
Optional, depends on situation Root-cause analysisC When a defect occurs, determine why it happened. Modify testing strategy to prevent that kind of mistake from happening again. Write an automated system test and an automated unit test then fix the code.
Pay-per-useC Pay-per-use provides alignment between revenue and the features that generate the revenue. “Connecting money flow directly to software development provides accurate, timely information with which to drive improvement” [1].

[1] K. Beck and C. Andres. Extreme Programming Explained: Embrace Change (2nd Edition). Addison-Wesley Professional, 2004.

Software Engineering Grounded Theory Research

If you are using Grounded Theory to conduct research in software engineering, I would love to hear from you. Please contact me at [professor at gmail.com]

If you are starting out on grounded theory research, I would suggest this plan of action:

  1. Determine with variant of Grounded Theory that you want to use. Options include:
    • Classic / Glasserian grounded theory
    • Straussian grounded theory
    • Constuctivist grounded theory
  2. Read Stol’s comparison of the three major variants and recommendations on how to write grounded theory research.
  3. Read my lessons learned paper
  4. Read a few exemplars (e.g. Software Development Waste, Becoming Agile)
  5. If you select Constructivist grounded theory, then read Charmaz’s Constructing Grounded Theory
  6. If you select Glasserian grounded theory, then read Hoda’s Grounded Theory for Geeks and read Glaser’s books in order.

This month I finished my Ph.D. I have used grounded theory for the last three years. I have found the method extremely rewarding. I loved interviewing, participant observation, and analyzing the data. There will be moments of confusion with the data. Embrace the confusion, on the other side tends to rewarding research results.

We Know the Way - Phonetic Lyrics - Moana

We Know the Way – Phonetic Lyrics – Moana

Thank you Opetaia Foa’i for this fantastic song. It is my favorite from the Moana sound track.

I wanted to learn this song, yet the official lyrics are in Samoan, Tokelauan, and English (I think). Since I only speak English, I created these phonetic lyrics.

I slowed the music to 70% normal speed and listened over and over again. Note, we might not have all the phonemes in English.

Ta-toe Ta-na-to-pho-low

Flah-ee-ow-ee-nah

Le’-tua-tsome knee-te-lay

A-ohm-my

Oo-ah-pah el-lay-lu-ah toa-le-eh

Ta-fi-na-fi-nigh

Oh-loh oh-loh tah-kah

Away Away

Ku-ku-en-new-ah

Te mah new lay lay A ta key A

Away Away

Tef-en-new-tim-mall-ee-A

Nah echo hockey lee-ah

Kah-ang nah-A

One of my hobbies is improv, and I play a game where we get our partner to guess multi-syllabic words by acting out each phoneme.

Remote Pair Programming

Developer pair programming

While at Pivotal, I have iterated on remote pair programming while working on two distributed teams. This shares what I have learned about remote pairing.

Best Practices

  1. Periodically check in with your pair about the audio, video, and screen sharing quality. Fix issues promptly. If you are experiencing issues, bring them up. Quickly iterating is the key to improving the remote pairing experience.

  2. Setup a video feed (zoom, appear.in, hangout, slack) that is always on and instant messaging (slack) in the secondary window. Much of communication is nonverbal. Seeing my partner excited or frustrated is super helpful.

  3. Leave the video feed running during breaks. This allows each person to see what is happening at the other end. If my pair is in an office, I might see that my pair is talking to another member of the team.

  4. In the office, leave the microphone open. This allows a remote person to feel connected to the office vibe when their pair is away from the desk. Osmatic communication can still happen.

  5. When a team is distributed between two offices but pairing locally, setup a remote video feed. This allows either office to get the attention of the other office by waving frantically. (Often pairs do not check slack frequently while pairing.)

  6. Use screenhero for screen sharing and zoom for video and audio. The audio in screenhero does not work as well as zoom or appear.in

  7. Have all pairs join the same appear.in channel. Mute everyone but your pair. Whenever you need a team huddle, just enable everyone’s channel. This reduces the friction for team huddles.

Here the team is in one channel with multiple conversations. Note that the team has an extra camera setup to see what is happening in the aisle: Brady Bunch View

  1. Ask for feedback about the development environment. While Pivotal uses workstation-setup for consistent development environments, the screen resolution difference between a laptop and an iMac makes that combination super tricky. I prefer iMac to iMac.

  2. If you are working across timezones, have empathy and be mindful of break times and lunch time.

Equipment

  1. Use a two display system. One screen is shared. The other is for supporting remote pairing.

  2. In the office, use an audio splitter. Have two headsets on the splitter. When another co-workes interrupts you, ask them to join the conversation. Having a second headset makes this easy. At the minimum, use a splitter to reduce friction. I use Scotch 35 Electrical Tape on black Belkin Speaker and Headphone Splitter

If you are using 4-pole headsets, I recommend this 4-pole Headset Splitter:

  1. In the office, use an expensive headset with a boom microphone.

In the office setting, avoid using earbud headsets as they tend to let in too much office noise. If you prefer these, check in with your pair about audio quality.

I updated this page on 10/3/2017 with the reasons behind the recommendations.

Team Code Ownership

Team Code Ownership is “the ability for any developer on a team to change any of the team’s code.”

I’m presenting a paper on Team Code Ownership at EASE 2016 next week.

Read the paper on-line: https://goo.gl/vOeeMn Or download the pdf: https://goo.gl/LXPg36

Summary of Paper

  • My observations clearly indicate that team code ownership is a feeling that can be engendered, not a policy that the team adopts

  • Pivotal developers more acutely feel team code ownership when i) they understand the system context; ii) they have contributed to the code in question; iii) they perceive code quality as high; iv) they believe the product will satisfy user needs; and v) they perceive team cohesion as high.

  • The paper shows that diverse events and trends that can undermine sense of ownership

  • The paper replaces Beck’s definition of Collective Code Ownership (“anyone can change any piece of code in the system at any time”) by introducing the new term Team Code Ownership (“the ability for any developer on a team to change any of the team’s code”)