Vision Critical reviewed

Tim Macer puts a hosted solution for web surveys to the test. Will Vision Critical be a critical part of your operation?

Given that many researchers wishing to do online research are likely to want to develop and run their own panels, it is odd that the Achilles heel of most online survey tools is still in providing decent panel management capabilities. Some get no further than giving you a database, a recruitment survey and some invitation management capabilities, others not even that far. Not so with Vision Critical, a Vancouver-based company that has spent the last three years building a very different online survey tool from the normal fare. Panel management lies at the heart of its Vision Critical web-hosted online survey tool, with not only facilities to fine-tune the selection of respondents from the panel database, but a complete range of tools to aid building the panel, nurturing a panel community and communicating with panel members and keeping it in peak interviewing condition.

Vision Critical is the first foray into technology for veteran Canadian researcher Angus Reid, and former owner of Angus Reid Group, which was acquired by IPSOS in 2000. The benefit of having a researcher shaping the technology is apparent throughout this software. More than any other online research tool I have reviewed to date, this is a product that acknowledges almost equally the needs of researchers, research clients and, with impeccable grace and style, the often overlooked needs of respondents. If you are seeking a more respondent-friendly way of conducting research, then this is certainly one to review.

The software is provided exclusively as a hosted ASP solution – Vision Critical have no plans to produce a version to install in-house. Therefore all of its capabilities are invoked from a web-browser interface. It is clear that Vision Critical is pitching for the corporate user rather than the agency, though the product should also appeal greatly to smaller research companies with no desire to run their own IT. The software comprises five key functional components: panel member administration, sampling, survey authoring tool, survey deployment and two reporting modules: a simple real-time tool, and a more advanced tool, about to emerge from beta testing.

Though it is not released yet, the new Dynamic Reports module module is intriguing – allowing you to use the same tool to run one-off reports for instant viewing, building complex reports as PDF, Word or Excel files, or publishing them to the web in the manner of a data portal for controlled viewing by clients and stakeholders – which will then change dynamically when you release more data into them. The approach here, and throughout the suite, is to not try to be too ‘clever’ in terms of functionality – avoiding options for options’ sake and going for sensible defaults so that it is easy to get started and become more sophisticated over time.

The Panel Plus module is fully released and in widespread use. It contains everything you need to run your own panel without having to recruit an army of admin staff to run it. It works with the sensible assumption that panel members should be able to self-administer their profiles, participation and reward redemption for themselves through the capabilities you can build into the panel member site. Though you can do all of this yourself, if you wish, designing the site and customising it is part of the set-up service that Vision Critical offers.

It also takes pays more than lip-service to the notion of a panel being an online community. For example, it provides the means to edit and publish your own online newsletter for panel members, which could be daily if you wished. It offers a content management system to let you publish findings and reports to your panel members, which is critical when creating ‘professional’ panels. And most interestingly in the context of Web 2.0, it offers the opportunity to blur the edge between researcher-led and participant-led qualitative discussion by integrating online forums or even blogs. It will also interact directly with commonly used Web 2.0 social networking portals.

There is a nice survey authoring tool too, which is well laid out and easy to use, with a very full range of functionality. It will also let you integrate with Java or Flash components, though it is disappointing that this otherwise futuristic tool does not have more in the way of animation and interactivity at present – though these are supposedly in the pipeline.

It is in the connection between panel and surveys that another astonishing difference emerges. The classic organisation for survey tools is to make each survey a discrete set of records, maybe linked to sample or panel indirectly, but with no actual interconnection. Some provide the mean to post back key variables into a respondent’s profile, but only to a limited extent. With Vision Critical, everything is interlinked, with every survey question answered being attached directly to the respondent’s record, and every prior answer being available for re-use. In case this sounds like chaos in the making, with a minestrone soup of a million questions to choose from, the coup de grace is that the variables are still presented as organised by survey, even though it is effectively just a slice through the database. But it means that at any time, all prior responses that a respondent has given are available to you to use for routing, sample selection or quota control – and also at the analysis stage, such as for profiling or comparison. So long as the respondent was asked the question and provided an answer, anything can be cross-referenced by anything.

But isn’t that what panel-based interviewing systems should by like?

The verdict: Vision Critical with Panel Plus
Web-based, hosted solution for high quality web surveys with an integrated, fully featured panel management solution and some support for online qualitative research.

Ease of use – 4.5 out of 5

Cross-platform compatibility – 3 out of 5

Value for money – 3.5 out of 5

Cost
Per month: approx £1,250 for first 5000 panelists, £500 for each additional 10,000, with no cap on number of surveys, interviews or users; add-on modules for analysis, communities etc between £150-450. Allow £7-10,000 on-off for set-up, design and customisation.

Pros

• Complete range of panel management and community building tools

• Sophisticated Web 2.0-like survey capabilities

• Links together all a panellist’s responses to any survey across the database

• Lets in-house researchers or smaller research firms run their own panels

Cons

• Authoring and management is Windows and IE only

• Limited animation support in survey tool

• No enterprise version offered

Further info
www.visioncritical.com

July | 2007