DatStat Illume 4.5 reviewed
A web-based interviewing system is given the once over by Tim Macer. Will it be the answer the tough business questions?
Stand back from research, and you can see that the vehicle of the survey is as arbitrary in defining the boundaries of most research endeavours as is the nation state in defining language, culture and markets. Most of today’s research technology reinforces the silo effect of the survey, with the data collected and isolated from other surveys or other processes that may be taking place in parallel.
DatStat Illume, on the other hand, offers a way to break through these barriers by making it easy to build bridges between surveys, borrow questions or data from one to use in another, or link surveys to other activities taking place simultaneously online.
Not that Illume aggressively confronts you with this radical difference. The software presents very much like other online data collection and analysis products. It still has the concept of questionnaires or projects and these you create online by building up questions in a pleasantly attractive point-and-click authoring environment. There is the typical tree structure on the left, and space on the right to write your questions and answers, select options and tweak their appearance. Furthermore, you can work online, or offline, as you prefer. If you wish to work offline, you simply book your project out to you, so you receive a local copy to work on, which also prevents others from updating it online in the meantime.
Under the bonnet
You could happily switch to Illume and remain unaware of just how subversively permeable the software is underneath, with its rigorous and imaginative application of true database technology and object oriented architecture to the business of market research.
In Illume, each question is considered a self-contained object, as are folders or groups of questions called collections. Any object in the database is accessible, subject to permissions, from any other object. This brings tremendous flexibility. You can add your own objects too, such as a different kind of question, or a resource, such as a data feed from a CRM system or a gateway to panel provider to request sample top-ups on demand and in real time.
An entire survey is also considered an object, and any survey can refer to data from any other survey during interview or analysis, either in aggregate or though respondent linkage, where this can be achieved. This can bring all sorts of benefits to longitudinal research or cohort studies, to panels or as a means of generating new samples.
Creating objects is a task for a programmer, but it does not require a software change from the software manufacturer. Once created, objects can be re-used in all your future work too.
Object of affection
The object-driven architecture means the developers have been able to incorporate much better logic diagnostics into the survey authoring process than is typical. For example, if you update a survey and move a question on which branch logic is dependent, you will get a warning if the logic check now occurs before the question has been asked. It will also check logic to ensure you have not accidentally created any orphaned questions in your survey that will never get asked.
The object approach also helps explain how the software has one of the best library capabilities I have seen in a research package. This can contain questions, answer lists, questionnaire sections or entire model surveys. There is a permissions-based workflow covering who is allowed to submit new repository content, and who may approve it, and even diagnostics on the effects of changes. When you use something from the repository, it remains under the control of the repository. So, for example, a list of car makes can be updated once, and all surveys using that repository item will be refreshed automatically – unless you choose to sever the link and convert the instance in your survey into a local copy.
You ask the questions
This is a highly accomplished interviewing platform, with all the customary built-in options to support everything from the simplest to the most complex survey. If there is a criticism in this area it is that some of the more advanced stunts you might wish to play require some tricky HTML coding within your survey texts – though some might just view this as additional flexibility.
Survey deployment is a doddle, with a highly intuitive survey administration tool that handles samples, invitations and reminders, and allows you to work with other survey modes such as offline mobile interviewing and an intelligent set of real-time response reports.
On the back end is what looks as if it is going to develop into a very promising researcher-driven online reporting and data portal module with the ability to create multi-user enterprise dashboards. However, to build such dashboards or portals at present requires expert help from DatStat. As a result, the tabulation and reporting elements in the package appear not to be as well advanced as the data collection tools.
The software’s underlying dependence on Microsoft technologies also means that, though largely web-based software, survey authors and administrators need to be running Windows. And though all support comes from Seattle, there are vast online support resources, including an efficient and responsive support issue logging and tracking, and excellent documentation.
The client view
Mike Skrapits is vice president of research at Mindwave Research, a full-service research company based in Austin, Texas. Mindwave uses DatStat Illume for the large volumes of online research it carries out, which includes several large and complex tracking studies and numerous international projects. The company switched over to Illume seven months ago, having found it had reached the capacity of another web-browser based survey package.
Mike Skrapits explains: “When we did our evaluation, DatStat offered a little something different and this really had to do with the architecture of the software. Scalability had become an issue for us. The architecture of DatStat Illume means it is highly efficient - and the performance gain we have experienced from that is enormous. We had needed five quad core servers with the previous system, and we still had performance issues. We are now running everything on a single server and enjoy better performance too. It is also very stable: we have experienced no unplanned downtime over the seven months we have been running it.
“We are very pleased with the flexibility of the software when designing questionnaires. I like the look and feel of the software and many of our clients have commented on this and how much they like it. From the perspective of the end user, it is visually appealing and from the perspective of the programmer, it is very flexible.
The system’s modular, object-oriented architecture has also encouraged Mindwave to take advantage of the savings that can be achieved by re-using components and also to develop some novel research methods of its own, as these in turn become components that can used on any project whenever required. “The database architecture allows us to effectively create libraries that we can borrow from,” says Mike, “and that has absolutely been beneficial. We have also created some code of our own to create our own specialised question types. The flexibility is there in the architecture to let you do this.”
DATSTAT ILLUME 4.5
Complete end-to-end web-based interviewing solution with a database and object-based approach to allow greater freedom in working across surveys and incorporating other applications or sources of data into the research process.
Ease of use – 4 out of 5
Cross-platform compatibility – 4 out of 5
Value for money – 4 out of 5
One-off licence fee starts at $25,000 for entry-level user. All licensees pay a variable transaction charge per interview, e.g. 85 cents when purchased in volume. One-off hosted surveys from $5,000.
• Intelligent, intuitive user interface
• Database approach makes it easy to link responses across surveys
• Repository provides an managed library of reusable questions and objects
• Comprehensive SDK (programmer interface) for easy linkage to other applications
• User interface, though web-based, only operates under Windows
• More advanced features require some direct HTML coding
• Online reporting not as well developed
• Support and training from West Coast USA only
April | 2008