There's been some discussion among Voice User Interface designers about how much of the VUI design folklore is just that: folklore. It's hard to get speech vendors to spend money to perform carefully controlled experiments and publish the results for the benefit of the community at large; it's even harder to get individual clients to take such a civic-minded approach for their own projects.
I think the speech industry would benefit greatly from a more rigorous, open, and transparent knowledge base of VUI design practices. This would lead to better designs, more user acceptance, and greater project success.
VocaLabs is interested in seeing better design (and more emphasis on testing, of course), so we've decided to step up to the plate and see if we can get the ball rolling. VocaLabs will perform usability surveys for free for any designer or company testing VUI design practices for the benefit of the community as a whole.
Here are the criteria:
- You have to have a well-designed experiment testing a design practice of general interest, not a question relevant to only a particular design.
- You have to commit to analyzing the data and publicizing it to the VUI design community as a whole.
- You have to find someone to host the test application(s) (we can't do that ourselves).
- The raw data will be made available to all interested parties in the VUI design community (peer review and alternate analysis are especially welcome).
- VocaLabs has to be identified as a sponsor of the research in all publication, presentations, etc.
I originally presented this offer to the VUI Designers e-mail list a few weeks ago, in advance of the SpeechTEK conference. There's been some interest from a couple different companies to take us up on this offer. I'll post more details here as things progress.