Using Data in Product Development

INTRO

Some days ago (2 June) Tradera hosted a LeanTribe Gathering event in cooperation with Softhouse on the topic of Data driven innovation. I was one of the speakers at the event where I talked about how the product development teams at Tradera use data in the development of Tradera as a marketplace.

The speach was recorded and was in swedish but you can find an english translation of the speach below.

VIDEO

TRANSCRIPT

(slide #1) Introduction
Hello, my name is Sammy-Sebastian and I have a master’s degree in Innovation and Industrial Management from the School of Business, Economics and Law in Gothenburg and I work as an analyst within the product development teams at Tradera which means that I work very closely with developers and product owners.

I will talk about how  Tradera use data in the development of the product according to the Build – Measure – Learn principles of the Lean-Start up methodology and I will try to put it in a larger context in terms of innovation processes.

(slide #2) Innovation?
The theme for today is data driven innovation, but what does it mean? If we break it down into each word; Innovation can be defined as the process of successfully taking a contextually novel idea to market and thereby create and capture value through it. An idea can be manifested as a product idea, service idea, the idea of organizational structure or process or even a novel business model. The market can be both internal within the organization or extrernal (as in interaction with customers).

(slide #3) Data Driven?
What is data driven then? At Tradera there are several different data sources. Interviews with users through phone is one example of a data source, observations in the form of user testing or input from customer service another. Tradera uses a feedback widget to get direct feedback from real users conserning  a specific change. The widget is currently the fastest way for us to interact with users affected by the specific change. We also use experiments such as A / B testing and simpler experiments with mock ups. Surveys are another way of gathering data and quantitative behavioral data that we receive through different tools are additional sources of insight.

As you can see, we have both qualitative and quantitative data sources where the quantitative helps us to answer the questions of what and how while the qualitative helps Tradera to understand why.

(slide#4) The Traditional Process
Ok so if innovation is a process which can be affected by data, how do things look like at Tradera in comparison to a traditional innovation process?

In a traditional process, one can imagine that the ideas go through a funnel and continues through some kind of stage-gate or so-called waterfall process which results in a market release of something that is “done”. Once released, the product or service usually  needs to be available on the market for a while before it generates enough revenue so that profits can be earned. The classic process is visualized by the picture below where one can note the time-to-market and time-to-profit.

original weber real.png

One of the major challenges for a traditional process is that the data  (in terms of market analysis, target group estimation etc.) is often collected at the beginning of the process and thus risks being completely obsolete when the idea is ready for release to the market.

Other risks include a missunderstanding of the customer needs during the initial data gathering or being forced to rely on faith, gut feeling or experience around what customers want when taking decisions during the development

The result could be that one releases a product or service that customers do not want or which contains features that cusomer are not interested in.

Thus, plugging in real data from customers, users, competitors and others in the development process one avoids many pitfalls that are associated with a traditional approach.

(slide#5) The Tradera Process

innovation process Tradera webermodd.png
The image above uses the same “language” as the image of the traditional process but it showes Tradera’s process instead. At Tradera, our user data is part of the development process. Since we test hypotheses on real customers very early in the process, the time-to-market becomes significantly shorter compared to the traditional process and in some cases Tradera can generate revenue during the development process. In addition, Tradera can quickly verify or reject hypotheses early and draw learnings from the collected data for future iterations and experiments.

(slide#6) Using Hypotheses
The hypothesis is fundamental in the data-driven innovation process that is used at Tradera. So, how are the hypotheses derived?

Overall, one can say that the material for ideas and hypotheses come from employee curiosity and the collected data and learnings from previous experiments. Other sources are continous market analysis, brainstorming sessions, analysis of user behaviours, customer support, usability testing, social media etc.

When constructing hypotheses, the entire team is committed and based on the discussions three main points are defined:
• Challenge (s)
• Potential solution (s) (hypotheses)
• Measurement methods (for the various hypotheses)

The different hypotheses are then prioritized into a backlog by the product owner and then the team tries to verify each hypothesis in the easiest possible way.

Regarding metrics, they often need to be customized for each challange or hypothesis but generally the ambition is to reject the hypothesis. It is always good to have a baseline to compare with.

(slide#7) In Practice?
I will try to boiling it all down into a concrete example so that you get a sense of how it all fits together.

Some time ago Tradera worked towards a concrete and overall strategic direction which was to make it easier for the users to find what they were looking for. Tradera already knew that users had requested easier way to filter their search results throuugh and based on a competitor analysis a solution that would simplify the indexing of items was tested. The initial hypothesis was that it would be possible to do technically.

The hypothesis was confirmed quantitatively and the next step was to test the hypothesis “the solution is appreciated by the users“. A prototype was built and then tested internally using live items from the site. The Internal testing gave qualitative data which confirmed the hypothesis. Consequently, Tradera could test the solution on real customers. In order to make the test as simple as possible a specific type of user and item category  was selected where the selection was based on available data on customer behaviour. The hypothesis for the test was  “If we facilitete an easier selection of size, shoe type etc.for our desktop users we will see an increased conversion in that category.” The test showed a significant increase in conversions, thus the hypothesis was verified and a new test was initiated but this time on mobile users.

However, the mobile A/B-test did not show any improvement in conversion which gave reason to initiate user tests which gave great insights and consequently Tradera was able to increase conversionrates for mobile users as well.

FUTURE

A deeper academic analysis of the topics that I talk about will be available in the near future. Please feel free to provide constructive feedback on the transcript or presentation.

REFRENCES

Bessant, J.R. & Tidd, J. (2011). Innovation and entrepreneurship. (2. ed.) Hoboken, N.J.: John Wiley & Sons.

Dodgson, M., Gann, D. & Salter, A. (2008). The management of technological innovation: strategy and practice. ([2. ed.], completely rev. and updated). Oxford: Oxford University Press

Ries, Eric, The Lean Startup: how constant innovation creates radically successful businesses, Portfolio Penguin, London, 2011

Wagner, K. (2007): Characteristics of Leading Innovators. INNO-Views Policy Workshop “Innovation Culture” Eindhoven, December 13 th, 2007 in www.adam-europe.eu/prj/3975/prj/MODULE_Idea_Evaluation_final.pdf

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s