Are sample vendors only automating what’s good for them?

Despite the potential benefits of automation for consumer insights, we regularly hear of its failures, particularly with sampling. The problems manifest themselves in worst case scenarios for clients: suppliers letting them down with late or incomplete delivery or, even worse, weird data. They arise because the benefits to sample suppliers don’t always align with those of the clients.

Suppliers are focused primarily on maximising yield while minimising cost, which is another way of saying ‘getting as many revenue-producing completes as possible at the lowest price'. While it is tempting to look at this as self-serving, it is an entirely logical response to the signals that buyers are sending. Yet while clients want improved speed and cost-effectiveness, they also (rightly) expect machines to be more reliable. And above all, they do not expect to have to sacrifice quality.

So when a supplier fails to deliver, or the data is wacky, it means one, or more, of five things is happening.

1. They have not done the hard and tedious work of mapping data.

Deep and detailed mapping of demographic and behavioural data fields, so two systems can speak the same language, is time-consuming, manual and unending. Because of this, companies often limit the attention they pay to this process. The resulting inconsistencies and errors cause ‘leaks’ in the process and can easily reduce feasibility by 10-25%.

2. They have not built programmatic intelligence into their feasibility calculations.

Automating sample operations fundamentally changes the participant experience. Not automating feasibility estimation creates a Frankenstein-style process where the head simply doesn’t match the body. Machine-driven sampling should enable suppliers to process massive amounts of historical data across every imaginable dimension that could impact someone’s propensity to take a survey. Done right, you get rock-solid feasibility estimates.

3. They have not built programmatic intelligence into their field monitoring.

The same ‘Frankenstein’ logic that applies to feasibility applies to monitoring. Automation runs 24 hours per day. By definition, automation that is not aware of field progress will just keep banging away, wasting sample and causing a project to go south fast – far faster than even the best project manager can react to if they have to manually track down and troubleshoot problems.

There is plenty of data to feed predictive algorithms that should allow suppliers to monitor progress to completion and automatically signal problems for review and remediation in real-time.

4. They are abusing respondents.

Yield maximisation will, by definition, result in suppliers mercilessly bouncing respondents from survey to survey in search of a complete. This problem is exacerbated when poor mapping and integration force respondents to provide their age and sex for the 17th time.

Poor experiences fuel disengagement and dropouts and ultimately yield bad data. A recent study reveals that fewer than one in four research participants are satisfied. Yet, as Jeffrey Henning, CEO of Researchscape International, said: "Survey researchers will continue to write questionnaires that don’t reflect best practices, while lamenting that no one pays attention to their research. Panel companies will continue to field long dreadful surveys, because none of them can afford to say no to business, even bad business that burns out their panellists."

Instead of fuelling discontent and dire outcomes, automation should be fuelling a revitalised commitment to sanely and respectfully engaging with respondents by prioritising great experiences and shutting down bad ones.

5. Their fraud detection is inadequate

With automation now a fixture of the digital ecosystem, it has only become easier for maleficent actors to use the same tools for nefarious purposes. While there is no agreed-upon estimate of fraud in the industry, we know it’s a big problem for two reasons. One is that our clients tell us this. The other is that it is a huge problem in the world of programmatic advertising, which is part of the recruitment supply chain. Our ‘industry standard’ defences are inadequate and obsolete.

Automation and artificial intelligence can massively mitigate fraud. With vast troves of historical data on demographics, behaviour, responses, and more, suppliers should be able to maintain a quiet and constant vigil, seeking incongruous and unlikely patterns in the data at a massive scale and silently eliminating the devious and the disengaged.

More often than not, when a supplier that tells you what automation can't do, they are actually telling you what their company has been unable or unwilling to tackle. There is every reason to believe that automation improves speed and cost as well as quality and dependability. If you are not seeing these benefits, it isn’t the fault of automation.

JD Deitch is chief revenue officer at P2Sample