Herd - the hidden truth about who we are

Using {Big] data to reveal stuff about how we are

Interesting piece covered in a lot of mainstream press today by Alberto Aceri of Bristol Uni and team about trends in words used in english Language novels over the last 100 years reveals about us:

" Britain's literature has grown less emotional since the 1960s, but American literature has become more so. Overall, English-language literature has used far fewer emotionally-charged words over time, but American writers have bucked the trend: They've ramped up their use of "mood words" in the past few decades as Brits have grown more stoic." With the exception of "fear" words (as the red line above indicates).

The ebb and flow of sentiment is also clear:

The point being that even really simple analyses of data that's lying about (or observations that can be turned into data) can reveal important changes in our culture

NB THIS may or may not (as the authors and commentary in the Atlantic both observe) reflect how we experience things: e.g. "the socially-conservative mores of Elizabethan England led to an increased demand for writing ''obsessed with romance and sex"" BUT it certainly reflects the culture and the products of that culture. And it's free!

So today's challenge is this: what can you do with the offcuts of data you have lying around to reveal what's going on beneath?

10 years of HERD insights

So it is 10 years ago this week that I first formally presented what I called the HERD hypothesis in written form (see above) at the Market Research Society in Birmingham (for which incidentally I shared the Best New Thinking Prize). Since then - together with a number of brilliant folk - we've managed to evolve and "operationalise" the basic insight at the heart of this paper, with 2 more books, prizes from the nice people at WPP, ESOMAR and Emerald Insight along the way, a host of well-liked articles and some fascinating conversations and experiences with people I'm not sure I'd have met otherwise.

It hasn't all been plain sailing - I have endured any number of pats on the head ("interesting but not really mainstream"), some strange challenges ("OK for kids marketing & poor people - maybe abroad?") and occasionally some hostile responses ("we don't believe that monkey shit round here" being my favourite).

But gradually, over those 10 years things have changed: not least thanks to the explosion of "social media" which has made arguing for the importance of social influence in shaping human (and consumer behaviour) so much easier (praise the Lord for Mr Z for this at least). 

And collaborating with brilliant people - especially Professor Alex Bentley - to turn the analytic techniques developed across the social sciences into practical tool for marketers and decision-makers has been an unexpected but wholly positive pleasure. Back in 2007, we first developed a 4 box map based on patterns to be found in

And now 10 years on, a number of folk have picked up and recycled the work we've been doing (which is exactly how things work and spread), mostly (but not always) attributing sources. So if for example you find someone presenting one of these* over the next few days and weeks,...

 

...you'll know what to think, won't you?  

Yes, the HERD effect is at play...

Thank you all - Alex, Mike O'B, Hugh, Jason, Ray P, Gareth K, Kevin K, Kevin D, Nick K, John K, Alex B, Susan G, Tom E, Audrey, John W, Graeme W, Wendy, Angela, Sair, Mark B, Mark H, John, Fiona, Stephen, Giles, Ben, Geoff, Paul, Liz, Judie, Colin, Chris, Roddy, Peter M, Gemma, David, Bob B, Bob P, Claire, Anne & Merry.

Sorry I can't be at MRS to celebrate the 10 years anniversary but I'm sure it'll be fabulous!

*BTW the version here first developed by Alex & I in 2010 with Anomaly & Sony Europe

 

 

 

 

Related articles

Herd Instinct in the headlines

Nilsson, artists and the HERD

Memories of the future (4) You, me and everyone we know (again)

First Post: The "Herd Effect"

The anti-nudge herd

The Herd on Holiday again

Herd, Spread and Research World

The Letterman Question

Here's a piece I did for the CASRO 2012-13 Journal (based on my keynote at their annual gathering in Scotssdale (page 42ff) based on an idea that the brilliant Grant McCracken gave me when I last had lunch with him:

"TV chat show host David Letterman used to play a game with his audience as he sifted the day’s ephemera, asking this simple question of the things he (and his researchers) found to point and laugh at: is this anything? Or, is it nothing? In other words, is the object, behaviour or utterance significant and suggestive of an important trend? Or, is it merely junk? Should we be paying attention to it or should we smile and nod at contemporary culture’s richness as it sweeps past our noses"

So let's ask ourselves before we start "...what kind of thing is the phenomenon we are studying and seeking to explain [or change]? Is it something shaped by the agency of individuals acting independently of their peers or is it something that’s shaped primarily by social means – by the influence of others other than the individual?"

  What kind of thing is this?

Science, Science Club and the brain

Been pondering recently how and why (and just the surprise of) science becoming a mainstream cultural phenomenon.

Not just the CERN Hadron Collider (much of the tabloid conversation around which was more Sci-fi than science, more crazy physics professors trying to destroy the structure of reality...etc etc)

But also the more general excitement and enthusiasm around science and science-based content - BBC Radio 4's The Infinite Monkey Cage, the irrepressible Prof Brian Cox's Astronomy shows and Dara O Briain's Science club

Some of this is clearly to do with the rise of the geek as a cultural identity, some of it a reaction to the terrible wibbly wobbly irrationality and superstition of the vaccine refusniks, homeopathy proseltysers and the religious fundamentalists.

But let's be honest, little of it has anything to do with actual rational & independent choice. 

Most of this - as yesterday's post about Alex & Mike's NYT piece suggested - is likely shaped by people copying each other, not by them making rational decisions based on the evidence. That said, it's a welcome development given the times.

How strongly this shift in attention to science and scientific work compares with the "personal opinions" on abortion and premature birth survival rates that certain UK ministers have expressed (which are clearly contradicted by the evidence).

And in our own part of the forest, with the over claimed certainties and inappropriate precision of so many vendors of what some call "neurononsense", "neuromania" or just plain "brain porn" (aka Neuromarketing).

How delightful then to watch Prof Uta Frith and other neuroscientists on the latest episode of Dara's show (particualry Frith) with their humour, humility and honesty remind us that we're really on at the beginning of understanding how individual brain activity relates to behaviour.

As my old headmaster would say, "mark and note".

 

The space between

 

Much of the time when we think of populations of individuals - be they political or commercial - we imagine that they are merely an aggregate of those individuals (that's why market research uses the notion of sampling to represent a larger population).

However, mass behaviour is more than the sum of individuals' actions: mass behaviour is more often the result of the interaction of individual agents with each other.

In IHWSH we show how patterns in market-level sales data are fundamentally different when a behaviour is rooted in social rather than independent choices (long tails of popularity distribution vs. short tails of popularity distributions).

Two interesting pieces around today - first, Res Publica's report on the wealth of pre-existing community structures (which is interesting, despite the predictably Trollopian tone of some of their output).

And then the return of the twitter trial and the iamspartacus phenomenon.

Both of these point up the importance of the space between - the space where most stuff really happens

     

Decision making landscape - towards a better map

 

In a recent post about what BE is missing, I referred to this Crawford chart that I feel captures the individualist model clearly: it places individuals - like a Ptolemaic Earth - at the heart of things.

The alternative is not a Copernican one - shifting the focus from one planet/individual to another while retaining the structure of concentric circles around the focal point.

No, a better conceptual map would be more like this: a repeated pattern of Crawford's individual circles or a series of interacting individual balls:

And most importantly of all, a dynamic pattern - one that is changing all the time (which leads to this kind of thing - a dynamic fitness landscape).

So two really big lessons here: 1. most humans live most of their life responding to the ideas, feelings and behaviour of those around them 2. those others are not fixed but in flux because they (too) are responding to those around them.

That's what leads to this kind of real or robot flocking and shoaling behaviour. The smooth running interspersed with sudden and rapid changes in direction are the result of this dynamic interaction.

BE can't begin to describe this level of complexity - which lies at the heart of social worlds -  because of the central failure to get beyond the individualist perspective.

Admittedly, it's hard to conceptualise this dynamic supra-individual alternative - our cognitive gifts seem much better suited to the individualist models - but that doesn't make it wrong to try. And we could do with a bit more trying here, I reckon.

 

Blinded by the technolight

As if by magic: Tom's cartoon explores the underlying technophilia which dogs marekting and policy-making too much. Brain porn or Digital Everything - it's all technophilia, I'm afraid

T'aint what they say

Twice this week, I've sat in a meeting and had exactly the same thought: the researcher here seems to think that what respondents say they do and why they do it is as close to the truth of the matter as damnit!

Fact: it mostly isn't.

So please remember that segmentation is based not on what people actually do and why but on what they tell you they do and why. The respondents are "presenting" as if...

And dear researchers, could you get into the habit of being more explicit, please? "this is what this group of people tended to tell us about what they do but it's probably not quite like that at all..." etc

I'll have what she's having - with John Kearon

DId this little interview a couple of weeks agowith John Kearon (Founder & Chief Juicer) at Brainjuicer, whom I managed to persuade to keep his trousers on...

Enjoy!

How DOES advertising work then?

For adplanners and the like:

In the pre Christmas (holidays to you, Gareth) rush, you might have missed this major U-turn by the lovely folks at Millward Brown research (the people who brought you the "awareness index" a key metric that assumes that advertising achieves its effect primarily by impacting memory - awareness and recall). 

Suddenly - after years of dissing those like my chums at Brainjuicer - who dared to suggest that emotions were the key to advertising effectiveness, the folk we used to know as The Leamington Spa Gang have turned 180 degrees. Apparently, emotion is everything or pretty much everything.

Nice to have you aboard, chaps. Will you be dumping AI scores then?

 

Riot on my own?

Finished an article today for Admap with my chum Alex Bentley, which uses - among other things - the recent riots to illustrate how near the surface our [wrong-headed] assumptions about human behaviour are.

Coincidentally both the Guardian and the BBC have chosen today to kick off public examinations into the causes of the recent riots.

The Guardian have chosen to repeat the format of the famous Detroit study into those disturbances and are embracing a collaboration with the LSE and (as they have learned to do from the MPs expenses) with a broader reader base. (They've also done this fab infographic)

The BBC have opted for a more informal town-hall style live event (video or twitter).

Listening to the latter it's clear how easily the conversation becomes too specific - about the particular things that happened in a particular place at a particular time and the particular causes of it. What gets lost is the underlying mechanics of this temporary outbreak - the copying. Too much already is either route #1 moralising about individuals or route #1A big-abstract-forces- acting-on-individuals kinds of argument; too little is understanding this event in the context of other such events.

Equally - as is all too often the case in market research - the individuals involved (in whatever way) are likely not to be very reliable witnesses, either about themselves or about the larger events. This is as true of the Guardian study as the BBC talking shop.

Fundamentally, only by understanding the behaviour as a social phenomenon (and not one rooted in individauls) can we really get to grips with it and start to understand what we might do differently next time. Think about it: a riot on your own? [sorry Messrs Strummer/Jones]

 

Social Science's to do list

Top ten social-science questions according to the journal Nature (HT Alex)

1. How can we induce people to look after their health?

2. How do societies create effective and resilient institutions, such as governments?

3. How can humanity increase its collective wisdom?

4. How do we reduce the ‘skill gap’ between black and white people in America?

5. How can we aggregate information possessed by individuals to make the best decisions?

6. How can we understand the human capacity to create and articulate knowledge?

7. Why do so many female workers still earn less than male workers?

8. How and why does the ‘social’ become ‘biological’?

9. How can we be robust against ‘black swans’ — rare events that have extreme consequences?

10. Why do social processes, in particular civil violence, either persist over time or suddenly change?

We're working on a number of these issues, directly or indirectly. What about you? Are you increasing the body of knowledge and how? And if not, why not?

The future of Market Research (redux)

Nice piece today by Martin Baillie on a subject that has repeatedly drawn my fire over the last decade or more (I was once called "The Enemy Within" or some such by Research Magazine): the future (or otherwise) for Market Research (and in particular the need to move from set-piece to real-time insight gathering - to "bake-in" continual learning to how we go to market).

While I have great fondness for the industry and its practitioners, it has always struck me as more anchored in the past than the future; feeling more comfortable with past assumptions and practices rather than any unfolding dynamic picture - I think that's what Martin is getting at too.

Equally important are the musings of Ray Poynter (the man behind the NewMR online conference in December). His latest thoughts point to the increasing importance of doing stuff with data (analytics) as opposed to collecting data (trad research):

"I have felt for a long time that the days of asking people what they have done, for example in tracking studies, are numbered. I refer to this aspect of research as bean counting and expect that in the foreseeable future this information will be collected from a combination of loyalty card, credit card, smartphone, CCTV, geo-tagging, RFIDs, and the sorts of wearable devices being designed for exposome research (collectively I think of these as part of the electronic wake, the trail we leave behind us). These changes will be another nail in the coffin of the days of surveys (I have previously blogged that in the future there will be far fewer surveys"

Regular readers will know that I'm a big admirer of my intermittent collaborator John Kearon and his attempts to innovate MR out of its current practices with Predictive Markets, Digividuals [a colloboration with the great Bausola @zeroinfluencer] and all kinds of We-research but if you're thinking about Market Research and its future,just step back 10 years and read this seminal piece by the late, great (and round here anyway) much missed Ginny Valentine Download Repositioning Research

(Sorry, didn't have time to big up Wendy, John or Tom but they also have a firm grasp on what's happening next!)

Change is a comin'....that's for sure but not before time...

Claimed vs. Actual: real and present dangers

HT @joelrubinson

Interesting example of the huge gap between what people claim to do and what folk actually do. Of course, the difference is not always so obvious or so large as it is here but some kind of difference there is likely to be.

Note also how embarrassing it can be when you take market research literally and assume that people know what they do, how they do it and that they can tell market researchers/therapists/themselves so.

Think about it next time your considering a survey or the data it generates.

Free gift: demonstration that the web isn't really built on cats but mice

 

 

Predicting the unpredictable

It's easy to imagine that things are predictable if you could only the right stuff about them. The market research industry's methodological armsrace is predicated on just this thought.

In the music industry, the ability to predict the future has always had real and tangible value (spotting the successful and signing it before it gets to be successful is a sure fire way for labels and managers to make cash).That's why the labels constantly refer to A&R folk as "the guy/girl who discovered/signed X..."

This piece about the UK's Xfactor (whose final was last night) from Hitwise which turned out to be precisely wrong is a case in point: Matt won.[HT @Dancall1]

But it's the thinking behind the approach that's nonsense: we've known for a long time that the way music spreads through a population is not based on the quality of the music itself but on the jumbled interaction of the agents involved (aka random copying). We don't buy/download/vote independently in the real world (online or off-); we do what others are doing - so what assume otherwise?

This distributed form of social learning creates patterns of great volatility that make it genuinely impossible to predict what will win (as in be most popular or most voted for etc). Unless of course, you find yourself (by chance) the winner or runner up (the reason why these kind of competitions do such good business for the labels afterward is that being the winner acts as a shorthand for popularity to everyone else - for at least one release). For most contestents, then, the best bet is probably "horsemeat".

If you really want to improve (but not beat) the odds, then you could do a lot worse than talk to these lovely peeps. Using the power of "We-research".

Chief Culture Officer Bootcamp London

 
Very excited - young Grant McCracken anthropologist extraordinaire and author of among other things Chief Culture Office is in London a week Friday to share the joy of CCO bootcamp. Learn from the man himself - just sign up here. 

I'll hanging around commenting on proceedings and generally getting in the way.  

And repeating the words of last week's Dr Who "Here's something I don't understand - let's go poke it with a stick"

Hope to see you there.

What are social networks for?

 
Pic c/o chinesemedicinenews.com

Been a bit quiet recently - partly because real world (family) concerns seem to have taken over a bit.

But also partly because have been feeling a bit bored by many of the discussions I trip over online.

In particular, there's still a lot of silly stuff around social networks and social influence.

So let's try to get at least this thing really straight:

Social networks are not channels for advertisers or for the adverts/memes you, your clients or any of your so-called "influentials" create, social networks are for all of the people who participate in the network. 

Being a social creature means you spend your life in social networks; being part of a social network gives each individual a number of benefits - shared protection, shared resources and most importantly shared learning. Our ability to learn from each other (the appropriately-named Social Learning) is one of our all-too-mutual species' most characteristic capabilties and the engine by which stuff gets pulled through populations (from technologies to health habits)

(BTW it's almost never the stickiness of your brilliant creation that causes the spread and even less often "social teaching" that most influence-models suggest)

Social networks are not best understood as channels down which folk send things; social networks are webs from which members pull down learning (from each other).

Now how does that change what you're trying to do?

Free gift: The audience and the backchannel

 

Those lovely people at Admap have let HERDfans have have free access to this piece I wrote for them about rethinking the idea of the audience for their relaunch edition!

Longstanding readers will be familiar with much of it, but hope they and the rest of you enjoy/find it useful!

Available until April 2nd!

Neuromarketing: another health warning

Nice quote from Wellcome Trust Director Sir Mark Wolpert on this morning's Today Programme (1hr 33mins plus)

"the toughest challenge of all, which is to understand how we think, how the brain controls behaviour...that is very, very difficult and not entirely clear we can do that now"

UPDATE: Later on Wolpert suggests "we are just scratching the surface"

Word of Mouth: The Eyes Have it

 

pic c/o Sodahead

Listen, I think we need to sort this thing out.

It's not that what people say to each other isn't important in shaping our behaviour.

Nor, indeed that recommendation (or advocacy or whatever you call it) in particular, is completely irrelevant.

It's just that the really important mechanism lies in what other people see, hear and feel going on around them: it's in the eyes and ears of the advocate's peers and not in the words of the advocate or recommendor. It's at the "influenced" end of the telescope and not the "influencer".

Of course, we all use "what other people are saying about x" as a general heuristic to shape our own behaviour and ideas but actually more of the time it's what we observe going on around us - the behaviours and the gestures of our peer - that really counts: what we see and hear and sense.

That's what brands and governments and managers and anyone trying to change behaviour need to think about and focus on. 

And that holds for Human and Timelord behaviour alike.

Tools from evolutionary science for understanding Human Behaviour

 

Very excited about the session I'm running at the Market Research Society Conference at the end of March:

I've got Dr. Alex Bentley and Professor Mike O'Brien confirmed for a session exploring how they have applied ideas and techniques from evolutionary science to the study of human behaviour. 

If you thought all evolution had to offer the study of contemporary human behaviour was Geoffrey Miller & Robin Wight's Peacock's Tailery (that most of what we do, we do to signal our reproductive fitness), then think again. Cultural evolution is now a highly developed multi-disciplined, many-headed school that has much more to say to us - not least, it uses hard data, not speculation to do its thinking. 

Now as we have these guys in London for a few days - would anyone be interested a longer, more detailed session to explore these ideas further and to share their techniques?

Nigel Hollis and JFK

 

Picked up something from the twitterstream this morning that seems to have got a few chums excited.

Nigel Hollis - chief adresearch guru at Millward Brown - has been musing on the relative roles of Mass & Social Media and whether on

Here's the tricky bit of his post

"All of this leads me to believe that social media is no way for a mass market brand to disrupt the status quo. It is simply an additional means to communicate and engage your loyal customers"

Well, sort of.

On the one hand, it's true that i. few mass market brands have (YET!) entirely switched their marketing budgets to Social Media and ii. as Faris points out the whole SM game tends to deliver slower changes in outcome than most heroic marketers would like (it's no easy quick fix for sure).

On the other hand, I suspect what lies behind Nigel's comments is this:

He still sees Mass Media as tools to do something to/send messages to/persuade etc audience members and Social Media as just a smaller scale and geekier version of the same kind of thing.

The point being (again) that SM is not really "media" in the sense that he thinks Mass Media are (nor indeed is mass media really as Neil et al have been pointing out): it's a different kind of thing.

And that different kind of thing seeks to create (or rather, stimulate or catylise) change in an entirely different way, by open up and humanising the business.

To paraphrase JFK, it's not what Social Media can do for you but what it can do to you and your business.

Memories of the future (4) You, me and everyone we know (again)

  

I really enjoy the debates that people have (with me and with each other) around the keynotes that I'm asked to give. I remember with particular fondness being told back halfway through the decade by one rather puffed-up marketing exec (as I recall, from the Blue Monster!) that all this social influence stuff is interesting but not that important as ultimately it is always an individual who does the buying - why bother particularly if technology was going to help us find and target them precisely?

2009 was the year that that this finally got to sound silly to most ears (not just signed up members of the HERD club).

And - no coincidence - it was also the year that 3 really important books got published on the science behind all this:

First, Sarah Blaffer Hrdy's extraordinary Mothers and Others which suggests that the origins of our incredible social nature lies in the shared child-rearing habits (alloparenting) of our ancestors, rather than the more masculine "machiavelli" hypothesis (i.e. in order to gain social power and status).

[Watch her at the RSA here]

Secon, John Caccioppa's fantastic social neuroscience study of Loneliness which suggests that loneliness is an aversive adaptation - like pain - which if not acted on has real physical effects on our all-too-social selves.

[See John at the RSA here]

And finally, Nicholas Christakis and James Fowler's brilliant Connected which pulls together a really insightful review of what connectedness means - at the heart of which is their analysis of the famous Framingham longitudinal health study (not just a large sample but one which records social connectedness not just health indicators)

[Here they are at (hey hey!) Microsoft labs...reminding us that online social networks are a very recent version of much older phenomena]

I'm not sure I can pick a winner for you out of these three (you should read all of them) but I know that work about the science of social (even work of this quality and coherence) would not have found the mainstream audience it now commands just a few years ago.

What cheers me is that in many spheres of human life - politics, medicine , social policy, management & of course, marketing - it's no longer a question of maybe for the whole "HERD" Social hypothesis; it's a given (even if we're still only just beginning to work through the implications Download BrainJuicer Paper - Me-to-We Research).

There's one thing you can count on going forward - there's no going back. It's us - you,me & everyone we know.

I won't say that I told you so, but...

Memories of the future (3) Rethinking the human

 
Regular readers will know that a lot of the drive behind my work and my writing over the last decade has been the flood of insights about human beings & human behaviour that - over the last 10 years - the behavioural and cognitive sciences have unleashed. In fact, you might say, reading and making sense of the science has shaped

Far from being a marginal - oddball, even - pursuit, this is fast becoming mainstream and will continue to grow apace this year.

From the IPA's Behavioural Economics Initiative to COI's Common Good Research on Behaviour Change the newly-published RSA Social Brain Project - all of which I've found myself involved in shaping.

Many of our most strongly-held common-sense ideas about humans and human behaviour are challenged by the work aggregated and synthesised in these kind of proejcts: like (as we HERDies like to remind folk) the Anglo-Saxon assumption that we are a species of self-determining individuals or the related one that thought shapes behaviour...

Of course, this is not just a liberation (and a step towards universal enlightenment) but it also offers the potential for the kind of one-eyed pick-and-mix applications of science that previously characterised things like "neuromarketing".

Memories of the future (2) living off the exhaust

 
One of the ideas that I've bumped into again and again this last year in different forms is that the exhaust data is where the juice lies.

For example for too long, traditional market researchers have indulged themselves in a methodological arms race which presumes that new data is almost always the answer, rather than find new ways to analyse the datasets they already have.

For users and vendors alike the truth is the same - rather than look at existing datasets afresh or rethink how we manipulate them to see what we can glean, most of us instinctively rush to collect new - fresh - stuff.  Which is a shame because - given the right kind of analytic curiousity - we can all learn a lot from examining the "free stuff" that's all around us, using the kind of  [See for example the paper John Kearon and I wrote this summer Download BrainJuicer Paper - Me-to-We Research]

Also, if you're going to be at MRS this March, I'm pulling together a bunch of smart social scientists to share their [Darwin-based] analytic tools for understanding mass behaviour. Download Jan 8pp prog_low res-1 And - if there's enough interest - we'll organise something separate around the time to explore the subject in detail.

Equally, I've been struck how people like David Bausola and The Kaiser are experimenting with putting the "hosepipe" of exhaust data that e.g. Social Media creates to good use in for creative and entertainment purposes. And - as will be revealed in due course - for other purposes, too...

Think about it in terms of sustainability: rather than rush to new data or fancy tools, what can you do with the exhaust? With the stuff that everyone else (including you & me) throws away?