Collection date: Oct 29, 2013
My aim was to gain a better understanding of the amount of customer/audience engagement done by startups, before and during development, as well as exploring the move towards creating “hackable” platforms rather that direct products or services.
- Most are platform style business models (either on purpose to provide early entry to market or because they’re developer driven, most likely both)
- Most are heavily domain expertise driven
- Very little customer or audience investigation is done prior to development
- Most consider customers as marketing reference
- Lean startup methods are used
18 results were collected asking sex, age, platform/product and then open notes taken about the proposition, project history and user engagement activities. These results were collected info from one-on-one coversations (4) with the demonstrators and from watching the pitches presented (14).
No notes were taken on the quality of the demonstrations or pitches, nor results of the day’s voting.
- Strong technical domain expertise,
- Years in dev in other forms or in other products
- Majority male, between 25-34 [Male = 15, Female = 3]
- Many had experiences either running a previous small company or large team
- 40% propositions are inspired by direct experiences, felt by the founders (naturally)
- 40% propositions are an extension of an existing business or activity
- 90% would best be described as a platform offering (contributers and consumers + api’s and/or app facilitiation)
- All were SaaS, cloud or web based solutions
- Specialist technical driven solutions (data scientist, robotics, computer vision, machine learning)
- End users were considered in reference to marketing and commercialisation (as part of the sales story)
- Propositions were rarely articulated in a neat summary sentence, but mainly communicated by attempts to immerse the audience in the situation by sharing the founder’s story of initial frustration (“I was stuck in a cab…”) or imagining an generalised problem (“who here has a mortgage…?”).
- Australia is a suitable customer testbed for global aspirations
- One company mentioned the Lean Startup model during questions from the panel
- During the onstage product pitches, there were very limited mentions of customer engagement (far less than I would have expected). They were referenced as informal or quite specific to a particular customer as part of the development tradeoff. The exception were two medical devices (i. respirator; ii. robot assisted disability) who did extensive end user research.
- During the one-one interviews there were very limited mentions of customer engagement (far less than I would have expected). They were referenced as informal or quite specific to a particular customer as part of the development tradeoff.
- No mentions of formal usability testing except in the two medical device cases mentioned above, which would be required as part of fomalised standards to be met.
- One medical device company engaged the target users and community groups extensively
- One medical device is about to enter clinical trials
- Two devices for elderly health monitoring systems were pitched with a glaring lack consideration for the wearers of the devices or any research into why the past 15 years of attempts have failed.
- Only one demonstrator mentioned “Intuitive interface” with no references to testing.
- Lots of prototyping in the wild, gathering feedback from various informal methods like feedback invitations.
- No mentions of metrix from launched offerings
- No mention of observed testing sessions
- One entertainment offering was live and downloadable during the presentation
- One construction offering overcame a large mental model hurdle using a very simple and clever demo where their project management software was used to build with a small project out of lego.
- One offering has now engaged a UX designer in short bursts to assist the development priorities
- The lack of UX mentions does not mean UX activities aren’t done, but it does show that it’s not considered a worthwhile topic to discuss in either a pitch or a conversation with potential investors or partners.
- People are fixing problems they identify strongly with and then find a market for it (bias).
- No one just wakes up with a good idea and makes it happen, these projects take years to reach the market – there were clear distinctions between the folks who’ve gotten something (anything) out and iterated, and the ones just pitching an idea.
My professional opinion
The offerings were very diverse and each situation has it’s own design problems and approaches.
Generally speaking, MVP’s and prototyping in the wild are valid approaches and I support them, as nothing beats getting your products into the hands of users to stress test the offering.
However I do wonder about the “failure” rate, and associated cost of effort. Without further interviews, I can’t comment.
Notes on reducing failure
Fit for purpose doesn’t equal useful. Failures can be mitigated and both of these can be brought down by at least some minimal UX activities, including:
- proposition development – develop a clear hypothesis
- talking to customers early – find out their real needs, do they match the hypothesis?
- usability testing prior to launch to check for basic failings that may impact adoption despite the offering being fit for purpose. More on this here: uxmyths.com/post/3086989914/ by Zoltán Gócza
Domain expertise can account for the proposition being fit for purpose, and customer feedback direct from the released build can account for feature requests, fixes, bugs, real world use (stress testing).
But what happens if you’re sure it’s right and you’re listening to your customers, and it still fails? Maybe it’s the workflow, or content, information heirachy, discoverability, integration with other services, device design…
The very minimum that can be done to sanity check for these less obvious issues is usability testing with the appropriate customer/audience:
- UX design assistance offers holistic fixes because attending to bugs or feature builds alone will break other workflows
- Talk to customers to find out if fit for purpose is also attending to a need… how many maps apps do you need? Is your differentiator really that compelling?