Developer Retrospective

January 22, 2010 at 7:44 pm · Filed under planning

While too much naval gazing detracts from real work, learning from the past is important for improving future outcomes. It’s also fun to look back and see how much we’ve accomplished!

I like to evaluate extended periods of technical work in terms of throughput, quality and direction.

Throughput means getting a lot done. The more quantifiable, complete results, the better.

Quality means the work is real and not hidden. Quality work doesn’t cause more work later by introducing bugs or making it harder to add features. Ideally, past work enables future work so that our progress is like a boulder rolling down hill picking up speed.

Direction means driving the company towards success. Burning down the wrong mountain is no good.

Looking back on the past three months, which is when we released our new Amazon Payments powered add-on, ProcrasDonate has demonstrated success in all three areas. I’m personally proud to have kept Throughput and Quality in balance. I would also like to highlight the excellent job Clay is doing with Direction, especially in response to users, market research and outreach efforts.

On the technology side, we accomplished our goals to:

• Solidify our core product
• Iterate each week
• Respond to rounds of beta testers

Two goals moving forward are to:

1. Continue responding to user feedback and developing the core product
2. Implement selected ideas based on new market research

Pivotal Tracker Data

The data from Pivotal Tracker, which we use to manage technology tasks, provides its own narrow but fun perspective. Ain’t data great!!?!

Below are our release charts from Pivotal Tracker. Every day is spent working towards a specific release. Each release chart plots the number of work left at the close of each day. Work is defined in abstract story points, which I estimate when stories are created. Sometimes I adjust these amounts when a story is completed. I try to be consistent without worrying about precision.

Red and green lines show “expected” progress. On the first day, they show the total number of points in the story. At the deadline they show all tasks completed.

Blue lines show how many points actually remained at the end of that day. They do not reflect stories that are in progress. To burn down a story the whole completed story must be delivered.

Chores do not have points and thus do not show up on these charts. Chores include backend work, developer scripts and testing. Bugs do not have points, either. A breakdown of story types is shown in the next section.

Finally, each iteration is exactly one week, but the period of each release varies. Our first public release using Amazon payments, release 0.3.0, is 3 weeks long. The rest are approximately 1 week each, although things often worked out into one long release and one short release. We didn’t want to be so concerned about deadlines that we didn’t take risks, but we also wanted to get into the habit of meeting deadlines….so we alternated.

Velocity

Velocity is the number of points the team accomplishes each iteration. Our iterations span one week, and our team, in terms of developer work, consists primarily of me.

The black line shows the “running velocity,” which is the average of the past three weeks. The running velocity is used by Pivotal Tracker to automatically partition our prioritized list of stories into iterations.

It’s nice to work so fast that new stories have to roll in from the backlog. It’s motivating to have dipped the velocity so much around New Year’s that our backlog was temporarily chunked into iterations stretching to the summer!

Story Type Breakdown

Features have points. They are customer facing value-adds.

Chores and bugs do not have points. Chores include backend work, developer scripts and testing. Bugs are existing features that are broken.

The following chart stacks each of these types on top of each other.

Retrospective

Let me first mention the New Years Eve dip. I took a couple days off to recharge

The low-point iterations are because of predominantly backend work, such as automatic backups that are restored on a dedicated test machine, and better build scripts, etc.

In the middle is an iteration in which no stories were completed. A very large chore—automatic cross-platform testing—did land in the next iteration, but that only shows up as a single chore.

That large chore was a rabbit hole. The crosstester app I made is totally awesome, but I should have reigned it in to a couple VMs with brief instructions for manual tests. I’ve learned from this and moved on.

What else did I learn?

We became better at meeting deadlines.

Story point estimation and consistency wasn’t as variable as I expected. I don’t remember changing story point estimations all that much, but there are many factors influencing what appears to be a level chart across the weeks (other than the rabbit hole dip).

Investigatory stories are always a little troublesome, since after the investigation they either turn into work or are halted, so you’re never quite sure whether they’re going to be 1 or 3 points….and we don’t start out thinking they’re investigations. Not so many stories are investigations.

I have to remind myself that we became faster at certain implementations. Tasks that gave me pause in November (2 or 3 pointers) are no big deal now (easy 1 pointers). This comes down to familiarity and reusable libraries and scripts.

The first time I had to migrate the extension database I thought it might take all day. I’ve done so many data transformations and database migrations that I’ve ceased putting “updates” and “release prep” on Pivotal Tracker (we never create 0 point stories…just do it!). The first time I wrote tests I also wrote the framework. Some stories get easier over time. Even when additional features are added, or a different kind of test, I can get more done in the same span of time.

Lastly, we averaged completing 7 features (not points), 2 bugs and 1 chore each iteration, with little variation. Keeping bugs and chores low and attacking features mercilessly—the right features, of course—sounds good to me. I was curious about this breakdown and variability when we first started using Pivotal Tracker, so it’s neat to see the results now.

That’s enough navel gazing for now.

I’ve appended the release notes below for anyone interested. Release notes are written for non-technical users, so only features and bugs are covered, and then in pretty general terms. (They show up on the Firefox tab that automatically opens to say “Congratulations! You’ve installed [or updated]…”)

Proudly ProcrasDonating,
Lucy.

Release Notes

0.3.0.html

• ProcrasDonate only tracks the Firefox tab you are currently viewing. It stops tracking as
  soon as you switch to another application or stop using your computer.

0.3.1.html

• Minor usability improvements

0.3.2.html

• New homepage design
• Website classifications added to My Progress page below the gauges
• Proper uninstall added (all data is deleted)
• Add-on build and update process improved. Update download problem (”incorrect hash”) should be fixed.
• Registration process improvements on payments tab

0.3.3.html

• User registration streamlined into just 4 steps
• My Progress page enhancements:
• Classify websites by entering them directly in the Classifications tab
• New Visits tab that shows all of your page visits for the day
• Time tracking halts when computer goes to sleep
  and resumes, if Firefox is the active application, when computer wakes up
• Fixed bug in which flash was not being detected on currently viewed webpage,
  thus the user needed only to be idle for 3 minutes for time tracking to halt.
  With the fix, pages that contain flash halt time tracking once the user
  is idle for 20 minutes. See the development blog post on
  
  How we track website visits for more time tracking details.
• Fixed toolbar icon bug in which times over 24 hours were reported incorrectly.

0.3.4.html

• When upgrades are available,
  users receive upgrade alert once a day when they visit the ProcrasDonate website.

0.3.5.html

• Graphicals!
• Pie chart added to help with choosing donation percentages to multiple charities.
• Trend section added to My Progress. Plots hours per day spent at different websites and overall.
• No visits are recorded while Firefox is in Private Browsing mode. For more information, see
  the FAQ question What is tracked?

.

• If you had trouble with toolbar icons disappearing they should be back, working and shinier than ever.

0.3.6.html

• My Messages tab added.
  Go here to view a helpful welcome message and
  your weekly affirmation reports.
• Top bar inserted above the first ProcrasDonate website visited
  of the day if there are unread messages waiting.
• Hold down My Settings arrows
  for rapid fire incrementing and decrementing.
• Fixed post-install bug requiring additional restart

0.3.7.html

• Play nicely with other Firefox add-ons, or at least don’t pee in the sandbox.
• Enabled announcements from ProcrasDonate and newsletters from charities to be uploaded
  to My Messages, as enabled by your update settings.
• Duplicate messages removed and underlying bug fixed.

0.3.8.html

• Community statistics added to homepage
• Private beta testing preparation: check
• Fixed extension bug in which dollars were being stored as cents. Data stored prior
  to this release will remain unchanged. Only new webpage visits are effected.

0.3.9.html

• Gracefully handle non-http page visits, eg viewing local files
• Fixed bug in My Impact display
• Removed Buy It Like You Mean It! as a pre-selected charity when downloading
  ProcrasDonate with no pre-selected charity selected.

0.4.0.html

• Smoother registration process
• Fixed co-extension bug: 0.4.0 plays nice with everyone, even bullies

0.4.1.html

• Welcome and registration emails sent to users.
• Homepage improvements: added social sharing site links,
  added recent tweets,
  process charity logos without reducing apparent quality,
  enabled video playback when charity logos are clicked,
  popup helper instructions when user clicks download button!
• Registration incentive page clarified thanks to user feedback.

Comments (1)

Story Types

December 5, 2009 at 3:33 am · Filed under planning

This fun chart is from Pivotal Tracker. It’s a stacked line chart of the types of stories we’ve completed each iteration since we began using Pivotal Tracker.

The graph provides a colorful analysis of how we allocated our efforts: squashing bugs; bulking up the code; adding functionality.

Seeing past results is powerful. One naturally follows the trend into the future. What breakdown do we want? Is it good or bad to spend time squashing bugs? I can’t help but feel extra motivated in bringing the red line (bugs) down to 0. I wonder if the black line (chores) is cyclic, and if so with what frequency. Can I raise the blue line (features) consistently? Will all these short term metrics trick us into making a bad decision?

It will be fun to find out.

Note that bugs and chores are always 1 “point” on the chart, since we’ve opted not to assigned points to them. Keeps us driving towards features since that’s the only way to burn down each iteration’s point-based allocation of stories. Although we can’t precisely compare story allocation to time spent, these results look like a reasonable proxy.

I don’t have too much more to say on this. Just wanted to share something colorful while we wait for more data to come in.

Proudly ProcrasDonating,

Lucy.

Comments (2)