Translator

Retrospective on ScrumPad Product/Market Fit

As I am beginning my journey of Lean Startup with our ScrumPad service, I plan to share my experience along the way.

The first stop in the journey is "Product/market fit." For the uninitiated (I am not an expert either, learning from doing), all it means is to prove that there is a large enough of a market for the product or service that I am building. The goals of this stage of a new business development is to identify/define that market with specific details- who are the users, why they like our product, and the business model that would work the best, and a repeatable sales model.

The best way to do this is to look back to understand where we are (since it's been 18 months that we are working on ScrumPad, but did not apply leanstartup model). I'll break the retrospective in two parts- 2008 retrospective, and 2009 retrospective.

Version 1- Free ScrumPad: 2008 Retrospective
What was our hypothesis?
All started with a itch of ours. We looked for a tool that fits our needs. We found two- Basecamp and Version One. But, none was what we were looking for. After trying both, we had decided to build one that we liked. So, since we liked what we had built, we thought there must be more like us who would also like ScrumPad.

How long did we spend to validate this hypothesis?
6 months.

What did we do during this period?
  • We primarily focused on adding functionality (product development, not customer.). We thought more functionality = more satisfaction = more users. 
  • We were releasing incremental version of ScrumPad every two weeks using Agile/Scrum process.
  • We did not do anything on the customer development front. Did not do any marketing. Did not talk to customers for feedback (did not get out of the building). 
  • We completely deferred design thinking that design and usability is secondary to functionality. We thought once we have the functionalities in, we could always add design and improve usability.
  • We did not charge for our service. We thought we need to first build out the full product (not thought about an MVP) before we can start charging.
  • We did put some basic tracking- Google analytics as well as some custom ones (i.e., number of projects, users, and overall project activity index that we defined). It helped us detect some early issues with signup and getting started with ScrumPad. 
What was the results?
We did get a constant stream of SEO traffic from search engines. This traffic resulted in a trickle conversion 1 to 2 every week. Received some unsolicited feedback, primarily from those who liked it. However, when we started charging in early 2009, most of the users from 2008 stopped using the service.

New Visits  Bounce rate  Avg. Time on Site  Pages per visit  Conversions
3,521                 48%                  00:02:50                       2.61                   5%

What did we learn?
Our product is better than the opensource counterparts. However, as a paid service, we did not learn anything about whether we have a market yet as a paid service.

Version 2- Commercial ScrumPad: 2009 Retrospective 
What was our hypothesis?
If we shift focus from functionality to user-interface and usability, there is a market for our service for a $21/user/month with single pricing model (no tiered pricing). I did a pricing analysis of our competitors (I must admit my class on pricing for my MBA from UNC helped).

How long did we spend to validate this hypothesis? 
All of 2009.

What did we do during this period?
  • We changed our logo, updated the UI, and continued to tweak the interface to improve usability. 
  • However, we spent a significant amount of time in refactoring (rework) to fix the sprawling code-base that resulted from the quick piling up of functionality. 
  • We started actively asking for feedback. We in fact incorporated a short exit interview during the cancellation process. 
  • We did not charge for our service. We thought we need to first build out the full product before we can start charging.
  • We still did not do any CPC marketing.
What was the results?
We continued to have a constant stream of SEO traffic from search engines. Even with pricing in place, we saw significant improvements in all 4 metrics. However, we saw a drop in conversion rate. With trial to subscription business model, we finished the year with a few paying customers.
                                                                                                           Conversion
New Visits  Bounce rate  Avg. Time on Site  Pages per visit     Trial   Trial to Paid
   7,578               38%              00:04:23                   3.72                4%          4%

What did we learn?
Since we have customers who are now paying for our service, there exists a market who are willing to pay for our service at the price-point we are offering now. However, we do not know who they are and how to reach them, and why they like our product over our competitors. Also, the metrics suggest that we have room to optimize the conversion rates through A/B testing of landing page and other aspects of the site and the service.

In hindsight, I believe what we achieved in 18 months or so, we could have learned even more in 6 months or less using Lean Startup techniques. More on what's next from here. 

Leave a Reply

 

 

 

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

A Few Thoughts on the Economics of Software Product Development

Some of you probably already get this… some of you might even disagree… but unless you are building software as a hobby… chances are, you building software for money. In other words, someone is paying you to write software for them. Why would someone pay you money to show up and write software? They are [...]

Podcast with IT Kanban

Kevin Ryan at IT Kanban just released a podcast with me. Check it out
Cheers
Mattias

Lean from the Trenches – Managing Large-Scale Projects with Kanban

I’ve published another book! This one’s called “Lean from the Trenches“. It is about how we scaled a 60-person project by combing techniques from Kanban, Scrum, and XP. I chose this title because it really it illustrates how to put Lean principles into practice in a software project, especially the notion of an end-to-end Kanban read more »

Coming to Brazil

Hi Brazil! I’m happy to say that I’ll be visiting you in a few weeks. I’ll be involved in two public events together with Samuel Crescêncio: Feb 10: Public seminar about Lean & Agile (in Florianopolis). More info coming soon. Feb 13-14:  Certified ScrumMaster course in São Paulo. The course will be in English, but Samuel read more »

AgileExams.com Controversy Resolved…Sort Of

Well it turns out the “controversy” about AgileExams turned out to be the biggest of misunderstandings. The Testimonials Were Authentic: Several of AgileExams customers contacted me and revealed the root cause of this confusion is the fussiness of PMI.org’s online … Continue reading