Every story is an opportunity to refactor – Todd Sedano
See k8s run.
Run k8s run! ™ – Todd Sedano
Title: The illusion of requirements; the requirements myth
“Requirements” is a pre-agile construct based on our false belief that building software was somehow like the construction of buildings.
“Requirements” limits the development team into thinking the end product is fixed and that the team’s goal is to implement a specification.
Agile development teams expect features to change over time. Even with thorough user research and validation, the product will change as soon as we receive user feedback. Even during the implementation of a user story, an emergent conversation on the team might change its acceptance criteria.
In contrast, features are a construct of preferences. Many preferences may stay the same, some will change over time, and most are made up on the spot Lichtenstein and Slovic 2006.
A simple example
Let’s consider a simple and common feature request: “the user logs into the system with a username and password.” If we consider this request to be fixed, we might not consider alternative solutions.
- Some websites support magic links. You don’t type in a password, the system emails you an authentication link.
- Some websites require two-factor authentication.
- Some websites rely on a third-party single sign-on such as Google, Facebook, or Okta.
Several of these solutions also attempt to solve the problem of users creating easy to guess or easy to break passwords.
Treating authentication as a “requirement” misses these opportunities.
How then do we deal with performance, scalability, availability, recoverability?
Pre-agile has the notion of “non-functional requirements” which describe system characteristics or product quality characteristics.
Before agile, I wrote my share of statements like “the web page should load in under 0.5 seconds.” In reflection, my non-functional “requirements” were arbitrary limitations on the system.
Here’s the typical workflow for agile software development. * Conduct user research and validation to determine appropriate feature for the opportunity * Build a minimal implementation of the feature * Verify that the feature matches the opportunity, if not iterate * Improve performance where needed This is a variant of “Make it work, make it right, make it fast.”(http://wiki.c2.com/?PrematureOptimization)
Availability and Recoverability
Many of the system characteristics such as availability can influence Service Level Agreements. Pre-agile treats these as guarantees. If we build the system just right, we can guarantee that we will achieve a particular outcome. Agile treats these as goals. If we want to prove ourselves that the system handles disruptions, then we introduce disruptions much like Netflix Chaos Monkey. We keep iterating on the product and deployment process until we achieve the goal. Adding Chaos Monkey to a system could be considered a “non-functional story” or a chore, since it provides no user facing functionality yet improves the product’s quality.
Avoid pre-agile constructs
It can be hard to remove bias from pre-agile constructs. If you see yourself or your team using the following words, pause and reconsider your options: * specifications, * requirements, * the system shall…
Avoid limiting yourself with pre-agile concepts. Recognize the flexibility of features in building software.
Here’s an academic paper demonstrating the negative impacts of framing potential system functionality as “requirements.”
For more information about the construction of preferences see http://www.cambridge.org/id/academic/subjects/psychology/cognition/construction-preference
Why We Pair
Hi. I have come to love Pair Programming. I enjoy the collaboration, cross-training, and better solutions that emerge while I pair.
Continuous Pair programming with Overlapping Pair Rotations removes knowledge silos to increase teams’ resiliency to team disruption [reference] and builds collective code ownership [reference] as described in Sustainable Software Development.
Pair programming is a complex practice requiring skill to master. Although Pair Programming has been around for 20 years and is well researched, some teams simply drop engineers onto a team expecting them to figure Pair Programming out.
As a practitioner, I am still discovering ways that I can improve the pair programming dynamic.
This article describes 1) several techniques that can improve your pair programming experience, 2) strategies on what to do when things get frustrating or go wrong, and 3) concludes with success stories of converting difficult pair-programming relationships into successful pair-programming relationships.
Like any relationship, such as friendships, family dynamics, dating, or marriage, when pairing works well, it is an amazing experience; when pairing falls apart, it can be frustrating and occasionally super hurtful. While unfortunate dynamics can happen while pairing, I have experienced the joy of finding each other in the rift.
Pairing exposes both our strengths and our weakness. How we handle our weaknesses may have a direct impact on the pairing relationship. One of us might not be comfortable exposing our weaknesses. If we are experiencing imposter syndrome, feeling that we are not qualified for the job we have, we might fear that our partner is going to find out that we are faking it.
I have been on a journey to be a better pair. While at Pivotal over 3.5 years, I have paired with ~39 pivots and ~ 20 client developers. I am grateful to each person who has helped me better understand myself and have helped me assemble and create these techniques. If you paired with me and I did not apply the technique, it may be because I learned it later.
Strategies for Success in Pair Programming
In my software development practice, I rely on several techniques for increasing the probability of success with a pairing relationship.
This article contains a toolbox of techniques. Feel free to pick a few to try out and then revisit the list in a few weeks.
Become invested in your pair’s success is the major take-away. When I am taking care of myself and looking out for my pair, an incredible friendship can form. Consider “what are my pairs strengths? And how do they map to my weaknesses?” and vice-versa. Have I always been invested in my pair’s success? No. My pairs probably have stories on how I could have improved in my pairing. Shifting from “what is in it for me” to “what is in it for us” enables new pairing dynamics.
When Pair Programming Goes Wrong / Weird
Since pairing is a relational practice relying on imperfect communication, misunderstandings and slights can emerge.
What could go wrong with pairing? Here are some situations that have bothered me.
You come across some code that is difficult to read. Your partner tells you not to refactor the code.
Your partner dominates the keyboard for most of the day.
Your partner does all the work without involving you.
Your partner tells you how horrible is the code that you helped work on the previous day.
You discuss a situation and agree to a plan, but then your partner starts doing his / her own thing anyway.
In this section, I list out several strategies for coping with a situation that is getting tense, de-escalating a situation that is tense, and how to re-engage with your partner after a tense situation.
There are several ways in which Sustainable Software Development has evolved from Extreme Programming (XP)
Sustainable Software Development
- combines user centered design with agile software development. Product designers apply design thinking and user research to understand what product and features to build. Extreme Programming relies on an onsite customer to tell the developers what to build. This is problematic in two ways. Often an onsite customer is unavailable, impractical, or too expensive to have on a team. Onsite customers are not trained in design thinking and information architecture.
- reduces the commitment level of the team to the work in progress, the stories that each developer is currently working on. Unlike Extreme Programming, the team does not commit to an amount of work to be done during a weekly cycle or iteration. There are no iterations in Sustainable Software Development.
- focuses the technical practices on two main goals
- caretaking the code
- removing knowledge silos
- replaces predictions with focusing on “what is the minimal that we need to ship?” Planning Extreme Programming utilizes the team’s velocity and backlog to create burn down charts for forecasting the team’s progress. Like Extreme Programming, the team maintains a shippable product.
- identifies and removes software development waste. Weekly retros identify wastes and the team removes wastes with action items and chores.
Both Sustainable Software Development and Extreme Programming deal with the tension of dates and features by asking “how do we reduce scope?” and “what should be done first?” In other words, developer happiness and sustainability is achieved by prioritizing the work.
This table compares Sustainable Software Development to Extreme Programming (XP).
|Sustainable Software Development||Extreme Programming 2.0 (XP 1.0 names)|
|Balanced Team pulls together various disciplines into one collaborating team. The team works together to accomplish a common goal.||Whole Team (Balanced Team) Whole team is also known as cross-functional teams. Include people on the team with the skills needed for success. Beck points out that a sense of team fulfills an individual’s psychological needs. “We belong. We are in this together. We support each others’ work, growth, and learning.” |
|Track One Practices|
|User research includes writing interview guides (questions), interviewing users and synthesizing the results.||Real Customer InvolvementC (On-site customer)
Make the customer a part of the team. “The point of customer involvement is to reduce wasted effort by putting the people with the needs in direct contact with the people who can fill those needs” . Even if a customer is busy, involving them whenever possible is preferred to not involving them at all. In this situation, the team needs to find alternative ways of validating features with users.
It is impractical to have an onsite customer.
|User validation checking whether ideas for the product resonate with users.|
|Road Mapping the product manager diagrams his or her long term vision for the product.Quarterly||Quarterly Cycle (Planning Game, Small Releases) Each quarter, the team identifies the stories for the next quarter. During a meeting, the team identifies bottlenecks, initiates repairs, plans the feature sets for the quarter, and selects a quarter’s worth of stories.|
|Story writing literally writing stories that represent potential product features.||Stories Stories are “units of customer-visible functionality” . Stories have a name and a description or graphical depiction. Stories track the work to be done.|
|Story refining generating small, manageable stories from larger, epic stories.|
|Project startup a special period of intensive user research and limited coding, typically taking four or more weeks at the beginning of a project or during a major shift in direction.|
|Track Two Practices|
|Caretaking the Code|
|Clean code means writing code that is easily understood, easily changed and complies with coding standards the team has adopted. Keeping the code clean requires continuous refactoring. Code craft encourages intention revealing names and discoverable code.||Code and TestsC The tests and code are the only permanent artifacts. Everything else (e.g. documentation) is generated from the code. Anything that contributes to what the system does today and what it will do tomorrow is valuable, “everything else is waste” .|
|Fast test suite and builds means that the tests execute as quickly as possible. A fast test suite provides speedy feedback to the team when something breaks.||Ten-minute Build Building the code and running the test suite should be automated and short. Longer builds increase the feedback loop.|
|Test Driven Development / BDD has the engineers incrementally build the software by always writing a failing test, then writing just enough code to get that test to pass.||Test-First Programming The developers write tests before writing code. Every production line of code is preceded by a test. Test-first programming addresses several concerns: scope-creep, coupling and cohesion, trust, and rhythm. The practice mitigates against scope-creep; without a clear goal, a programmer can easily write more code than is necessary. The practice improves coupling and cohesion as easy-to-test code often has beneficial design qualities. The practice builds trust in the code as engineers trust tested code more than un-tested code. The practice produces rhythm as it provides a cadence of accomplishing goals.|
|Team code ownership is the ability for any developer on a team to change any of the team’s code.||Shared CodeC (Collective ownership)
Everyone on the team can modify any of the system. When engineers sees something broken, they fix it.
|Continuous refactoring||Incremental Design (Simple Design Refactoring) Build the system incrementally and evolve the design every day. In the past, software teams might often had long design phases that assumed features would not change. When the team’s understanding of the system changes, make incremental changes to the system to align the code’s design with the team’s desired goal. Teams routinely remove duplication.|
|Integrate frequently||Continuous Integration
When working on new features, routinely integrate new code into the master code repository. Avoid long-lived branches. Beck suggests un-integrated code live on its own “no more than a couple of hours” . The longer the team waits to integrate, the more unpredictable and expensive integration becomes.
Single Code BaseC Ideally there is only one branch of the code. Multiple branches, (e.g. for releases, or customers) creates extra work. Branches live a few hours at most.
means shipping or deploying the product frequently. For a website, “frequent” might mean several times a day; for enterprise software, “frequent” might mean once a month.
Each team releases when features are done or at a given cadence
|Daily DeploymentC Every day put a new version of the code into production.|
|Acceptance / staging environment means that the product manager accepts a story in another environment that closely resembles the user’s environment. For installed software, this is another machine or cluster. For web development, this is a staging server.||(implied)|
|Build system is for continuous integration or continuous delivery. The build monitor indicates the status of running all the tests with the latest delivered code.||(implied)|
|Stop on red builds means that when a build fails (the build monitor turns red), one pair immediately stops what they are doing and fixes the issue. This is not necessarily the pair that caused the build to fail. Shaming whoever introduced the change to cause the build to break is antithetical to the goals of sustainable development. The occasional failing test is not a big deal, the team handles it and moves on.||(implied)|
|Knowledge pollination are practices the team uses to push and pull knowledge between team members, often just-in-time.||(implied)|
|Remove Knowledge silos|
|Continuous pair programming refers to two engineers co-creating source code and tests on pairing workstation with two displays, two keyboards, two mice, and identical development environments. Other models of pair programming exist, but this setup reinforces the equality of the relationship.||Pair Programming Every part of a production system is written by two programmers sitting side by side. The pair works together to solve problems, keep each other on task, learn from each other, and abide by the team’s practices.|
|Overlapping pair rotations means that, each day, engineers change pairs such that over time, everyone works with everyone else. One member of each pair always retains work in progress from the previous day.||N/A|
|Boundary Spanning Practices|
|Backlog grooming the product manager routinely re-sequences the stories in the backlog and verifies that stories at the top of the backlog are ready for engineers to start.||N/A|
|Accepting stories after the team delivers a story, the product manager verifies that the delivered work achieves the acceptance criteria. The product manager tries to break the feature with common unexpected input.||(implied)|
|Story showcase meeting a 60 minute weekly meeting to discuss stories that the team will be working on in the immediate future. The product manager describes upcoming stories and the engineers provides feedback on the expected difficulty of each story. The story showcase helps improve the team’s shared understanding of the product.||Weekly Cycle (Iteration Planning Meetings) Each week, the team accepts a certain amount of work to accomplish. During a meeting at the beginning of the week, the team reviews progress to date, the customer chooses a week’s worth of stories, and the engineers decompose the stories into tasks and individually estimate each task. “The goal is to have deployable software at the end of the week which everyone can celebrate as progress” .|
|Project Management Practices|
|Core work hours means the team prioritizes working together in the same office during the same hours. Flexibility with the schedule helps with outside of work constraints provided that the team has enough time in common.||Energized Work (40-hr week)|
|Stand-up is a 5-10 minute, daily meeting where team members discuss changes that affect the team from the previous day’s work and impediments to the current day’s work. If physically possible, people stand to encourage a short check-in. Usually this meeting is at the start of the day, otherwise it is a daily spin-down meeting.||Present in http://www.extremeprogramming.org/, but not in the 1999 or 2004 books|
|Retro a 60 minute weekly meeting in which the team re- flects on the week, celebrates successes, and identifies ways to improve. The ideal retro is a safe environment where a team can discuss any topic constructively.||N/A|
|Flexible scope contracts establish an ongoing relationship between a development organization and a client organization without unnecessary details. It might state that the client owns the code, that new releases are expected at most every two weeks, or the daily rate at which engineers are billed. It typically would not state a fixed scope, schedule or budget. If a schedule or budget is fixed, scope is not and vice versa.||Negotiated Scope ContractC< “Write contracts for software development that fix time, costs, and quality but call for an ongoing negotiation of the precise scope of the system.” |
|No overtime means no one is allowed to work overtime except in genuinely exceptional circumstances, such as solving a customer escalation.||Energized Work (40-hr week) “Work only as many hours as you can be productive and only as many hours as you can sustain” . Working longer hours may make the team less productive. Team members need to take care of themselves; this includes staying home to rest when sick. The work patterns of the team should be sustainable in the long run.|
|Core Work Hours, Be Present||Sit Together Software development is a collaborative endeavor. Co-locating teams increases collaboration. Distributed teams are still possible, yet “Sit Together predicts that the more face time you have, the more humane and productive the project” |
Agile project management tools (such a Pivotal Tracker)
provide more value than a physical board showing project status.
|Informative Workspace “An interested observer should be able to walk into the team space and get a general idea of how the project is going in fifteen seconds” . The team can modify the workspace to make it more conducive for their work.|
The team does not commit to accompilshing certain stories each week.
|Slack During the weekly cycle meeting, include tasks in the plan that the team can drop if the team gets behind. Adding slack to the schedule enables a sustainable work environment.|
|Not needed for all projects||Incremental DeploymentC
Switch between a legacy system and a replacement system in incremental steps. Avoid a hard switch over from a legacy system to a new system.
Slowly replace a legacy systems.
Move engineers where needed. Balance business needs with personal goals.
Try to improve team efficiency so that the team can shrink in size. When a team becomes more productive, re-allocated extraneous team members to other teams.
Team ContinuityC “Keep effective teams together” . People are not fungible, and building relationships is part of creating a high performing team. Rotating people through teams helps spread knowledge.
|Optional, depends on situation||Root-cause analysisC
When a defect occurs, determine why it happened. Modify testing strategy to prevent that kind of mistake from happening again. Write an automated system test and an automated unit test then fix the code.
|Pay-per-useC Pay-per-use provides alignment between revenue and the features that generate the revenue. “Connecting money flow directly to software development provides accurate, timely information with which to drive improvement” .|
C are optional XP 2.0 Corollary Practices
For a team transitioning to Extreme Programming, Beck recommends starting with the primary practices. Beck claims that “the primary practices are useful independent of what else you are doing” resulting in an immediate improvement, whereas the corollary practices work well once a team has implemented the primary practices. Extreme programming is not dogmatic about the list of practices. The team chooses which practices to apply.
Once a team has implemented the primary practices, then the team can start implementing the corollary practices. Beck says that it seems “difficult or dangerous to implement [the corollary practices] before completing the preliminary work of the primary practices” .
 K. Beck and C. Andres. Extreme Programming Explained: Embrace Change (2nd Edition). Addison-Wesley Professional, 2004.
If you are using Grounded Theory to conduct research in software engineering, I would love to hear from you. Please contact me at [professor at gmail.com]
If you are starting out on grounded theory research, I would suggest this plan of action:
- Determine with variant of Grounded Theory that you want to use. Options include:
- Classic / Glasserian grounded theory
- Straussian grounded theory
- Constuctivist grounded theory
- Read Stol’s comparison of the three major variants and recommendations on how to write grounded theory research.
- Read my lessons learned paper
- Read a few exemplars (e.g. Software Development Waste, Becoming Agile)
- If you select Constructivist grounded theory, then read Charmaz’s Constructing Grounded Theory
- If you select Glasserian grounded theory, then read Hoda’s Grounded Theory for Geeks and read Glaser’s books in order.
This month I finished my Ph.D. I have used grounded theory for the last three years. I have found the method extremely rewarding. I loved interviewing, participant observation, and analyzing the data. There will be moments of confusion with the data. Embrace the confusion, on the other side tends to rewarding research results.
We Know the Way – Phonetic Lyrics – Moana
Thank you Opetaia Foa’i for this fantastic song. It is my favorite from the Moana sound track.
I wanted to learn this song, yet the official lyrics are in Samoan, Tokelauan, and English (I think). Since I only speak English, I created these phonetic lyrics.
I slowed the music to 70% normal speed and listened over and over again. Note, we might not have all the phonemes in English.
Oo-ah-pah el-lay-lu-ah toa-le-eh
Oh-loh oh-loh tah-kah
Te mah new lay lay A ta key A
Nah echo hockey lee-ah
One of my hobbies is improv, and I play a game where we get our partner to guess multi-syllabic words by acting out each phoneme.