I continue to be vexed by internet gremlins. I couldn’t post this comment as a comment so I’m taking Eric’s advice and am posting it as a new post. Sorry, folks.

Anyway, this is a response to Debbie Pascoe’s 16 Aug 08 11:22am comment.

Yes, owners not tagging all the pages desired, intended or necessary. We share that evil, I guess.

I didn’t know so many possible errors could creep in as far as javascript goes. It makes me happy that one of our original design decisions to was make things as simple as possible on the client’s side and that all the heavy lifting would be done on our side. Our side we can control pretty well. The client? Not so much so.

Thank you for describing what is meant by calibration (”automatically scanning the site and uncovering these conditions so they can be corrected”). This is something our technology does during the running of reports (uncovering invalidating conditions).

Calibration should not be a core competency of WA vendors? Yet WA vendors should partner with people who have that as a core competency. Hmm…

So if they should partner with such individuals, they must in some part be assuming responsibility for proper calibration, correct?

I read your Response to Michael Wexler’s Post re:What Web Analytics is Missing and Michael Wexler’s What Web Analytics is Missing… (with any luck I left a comment there). I asked some NextStagers about this. The discussion was very interesting. The core issue is something we’ve seen repeatedly in labs; people making first order estimates based on inadequate data because they failed to figure out the basic parameters of their experimental systems.

After a lot of conversation it pretty much came down to “Yes, calibration should be something offered by vendors” and there was a caveat that plays to your partnering theory, that such a service should be included in the fee structure.

The fact that this topic has risen to the “conversation topic” level is an indication that it happens enough to be recognized in the web analytics community. Of course, one of the joys of our society is that we think contracts assign and absolve responsibilities (except probably with very large clients).

Lucky us, huh? Sometime when we’re together, remind me to tell you what one of our early employees, a fellow from Australia, thinks about contracts in the US.

The amusing piece of all this is (to me) that (I thought) web analytics was all about accountability.

Glad you enjoyed the thumb comment. I only wish it weren’t true so often.

(This post is a response to Jim Novo’s 12 Jul 08 9:40am comment. I’m posting it here as I’m including some images and I need to post rather than comment in order for images to show up. You can read the original comment here)

Thanks, Jim. Good to be engaged.

Yes, I’m familiar with Recency. We use some variants of the standard concept in our blog research and tools. I’ll agree that Recency is a pervasive human behavioral model. Do I think Recency is the best link between our worlds? That would take a few more discussions and I’m happy to learn if it is. Note that I’m qualifying my agreement because I think the devil might be in the details on this one (something I hinted at in my comment to your http://blog.jimnovo.com/2008/06/22/peak-engagement/ post.

Recency

As you point out, the Recency model is very simple and I definitely agree that one can do better. My understanding is that it’s very useful as a general metric and its usefulness decreases as one digs down and explores why and how recency occurs. Some good papers on this are Customer value modelling: Synthesis and extension proposals (Journal of Targeting, Measurement and Analysis for Marketing, Volume 11, Number 2, 1 September 2002 , pp. 124-147(24)), How to develop new approaches to RFM segmentation (Journal of Targeting, Measurement and Analysis for Marketing, Volume 13, Number 1, 1 September 2004, pp. 50-60(11) and Metacognition and learning about primacy and recency effects in free recall: The utilization of intrinsic and extrinsic cues when making judgments of learning (Memory & Cognition, Volume 36, Number 2, March 2008, pp. 429-437(9)). The list is extremely extensive and I’m happy that a great deal of it either correlates, substantiates or validates NextStage’s research. For example, I’ve talked at eMetrics and elsewhere about getting visitors to do things online simply by placing action items on pages in a way that makes volition near mandatory. Primacy and Recency Effects on Clicking Behavior (Journal of Computer-Mediated Communication, Volume 11, Number 2, January 2006, pp. 522-535(14)) is completely independent of NextStage research and does just that (I wrote about priming in my BizMediaScience blog along with related subjects).

For example, recency is highly age dependant with the strongest cues being in early adulthood (barring social impetus effects). This is great for people marketing to that demographic and the results shouldn’t be thought as characteristic for other groups (see Temporal distribution of favourite books, movies, and records: Differential encoding and re-sampling (Memory, Volume 15, Number 7, October 2007, pp. 755-767(13)), Aging and contextual binding: Modeling recency and lag recency effects with the temporal context model (Psychonomic Bulletin & Review, Volume 13, Number 3, June 2006, pp. 439-445(7)) or Age-related differences in advertising: Recall and persuasion (Journal of Targeting, Measurement and Analysis for Marketing, Volume 13, Number 1, 1 September 2004 , pp. 7-20(14))). What I haven’t seen in the literature so far is anyone using PCP methods to augment recency analysis (and I’m thrilled for someone to point this juncture out to me).

Simplicity

As far as a given model’s simplicity being its greatest advocate for use…Have you seen Responding to “Visitor Engagement: Time for a reality check?”, specifically the part about not thinking that a simple metric is the best metric? In a lot of ways the quest for simple web metrics makes me think of the evolution of software itself. Originally not easy at all, then tools came along that made it easier and now it’s a often a matter of point and click simplicity. But god forbid you attempt to get under the covers of the tool that’s giving you that point and click simplicity! The http://blog.jimnovo.com/2008/06/22/peak-engagement/ post you offered at the end of your comment and the comments in that post indicate (to me) that simplicity wasn’t a concern when developing those protocols (and I’m happy to be told otherwise).

I think this talks directly to your “…the pure simplicity of the model is tremendously appealing, especially when faced with the challenge of trying to get people to analyze anything at all.” The goal should be to provide simple to use metrics that demonstrate actionable results (and yes, part of that is understanding and developing metrics that aren’t actionable at present or aren’t financially successful (a nod to your honesty, Good Sir!)) because in the end (I believe) people don’t care about what is being measured or how, they only want to to know how to get the effects they want.

I also recognize that I tend to resist Recency as a “primary screener” because I’ve seen the model fail more often than I’ve seen it succeed. This is probably due to NextStage making use of recency models to analyze how people process time (as noted in Responding to Christopher Berry’s “A Vexing Problem, Part 4″ Post, Part 2). People are “…are most strongly influenced by our most recent experiences…” so (in my world) recency is really indicating how much of a challenge a given population is having getting things into and out of memory in order to act upon them because people reference “recent” by markers their brains create in memory (you don’t want me to include citations, do you? I already got flack once today from somebody looking over my shoulder at this and cracking up at the citations included thus far).

<ASIDE>
I learned that editors of an online I use to write for had an office pool based on how many links I’d include in my columns. It seems being able to document one’s discovery path isn’t as required as it use to be. Sigh.
</ASIDE>

This discussion interweaves with my Responding to Jim Novo’s 19 July 08 9:33am comment, I think, because one of our reports, Return Visitor Ratio, reports on how many visitors believe they’ll be returning. The advanced version of the report includes a time calibration (similar to our Loyalty report) so perhaps I should share them in more detail.

Visitor Return Ratio, Loyalty, Credibility and Believability

The figure on the right is one of our basic Visitor Return Ratio charts taken on 25 Aug 08. It indicates that about 78% of the people to this site will return (that’s the black) and about 22% won’t (that’s the red). The little yellow dot on the left of the chart is an indication of how strongly the last visit is influencing the decision to return (basically not at all).

How come this success or failure of this visit isn’t influencing the decision to return? Because the visitors are loyal and a single experience (unless extreme) isn’t going to influence them that much (this hearkens to the definition of loyalty given in Responding to Jim Novo’s 19 July 08 9:33am comment).

The question then is “How loyal are visitors?” This is answered in the chart on the right. They are loyal and not outrageously so (the black bar is on the positive side of “0″ and not a lot). The red dot on the “0″ point indicates that although loyal, it won’t take much for people to defect to a competitor. This is one way of recognizing when people are loyal due to something they believe or something they accept as credible.

Believability comes from a deeper place in the psyche than credibility. In a PersonalLifeMedia post to Moxie Insights SVP of Customer Insights, Diana Middleton I explained that believability and credibility work quite differently in human consciousness (and I’ll include a link in the comments if the post hasn’t gone live yet).

“Credibility deals with facts and Believability deals with emotions and desires. Literally we’re dealing with the differences between scientific thinking (Credibility) and anecdotal thinking (Believability). Again, this is something I wrote about in Responding to Christopher Berry’s “A Vexing Problem, Part 4″ Post, Part 1, My Easter Eggs Critiqued, Defining Engagement (Again? Oh, Lordy!) and Exploring the Holes in Flawed Logic and Responding to Geertz, Papadakis and others 5 Feb 08 comments.

Credibility is easily measured; check references, check sources, interview/survey other individuals who participated or witnessed, etc. Litigation specialists and trial attorneys know the value of a credible witness (and “expert” witness) is (can you guess?) negligible. (This is something I mentioned during a dinner discussion arranged by Susan Bratton in Chicago and no, there is no link to it. Darn, huh?)”

Loyalty due to belief is often demonstrated by that red dot on the Loyalty chart being way over to the right, well beyond the end of the black bar. What this particular chart is showing is that the majority of visitors find the information presented credible and not understandable (any guesses as to which site that chart is from?).

The true challenge comes when one wants to answer “When will a visitor return to a site?” People don’t conceive of time as in hours or days or some such. At the DC eMetrics 07 I sat with two friends in the bar and put them through an exercise that demonstrated just how difficult it was for people to “think” of something much beyond 2-3 hours in the future or past, and that just thinking of doing something (actually thinking of doing it, not just saying “I’m going to do …”) much beyond a day in the future can exhaust most people’s cognitive resources. This is because the majority of people surround themselves with a wall of “now-ness” and devote their attention to what’s going on around them now (evolutionary installed neural wiring). This is why “planning for the future” is one of those wonderful things that so few people do truly well (I love Sir John Harvey-Jones’ “Planning is an unnatural process; it is much more fun to do something. The nicest thing about not planning is that failure comes as a complete surprise, rather than being proceeded by a period of worry and depression.”)

The best approximation we use at present is that we have lots of understanding of what the majority of people recognize as “now” so we can guess when they’ll return. I always emphasize to clients that it’s a) a guess and b) doesn’t take into account the elements of everyday life that prevent people from doing something they planned to do but wasn’t something critical to their survival. What we’re really reporting is when visitors believe they’ll return (note, not “hope”, “think” or even “plan”. These imply conscious processes and while they’re reportable they’re not truly useful. I can “plan” (cognitive) on doing something but the conscious act (behavior) won’t take place in reality until I non-consciously believe I have a reason (motivation. Hint to readers; motivations stem from the most primitive desires, the earliest wiring, and are usually the strongest drivers to action. This is why law enforcement always looks for “motivations” for a crime) to do it, hence the importance of the {C,B/e,M} matrix).

<ASIDE>
Guestimating when someone will do something is based on determining their intender status. NextStage’s solution to this was developed 3-4 years ago and is based on Husserlian Conceptual Mechanics. You can find references to either in any of Priming the Conversion Pump with Color, The First Sale is the Next Page: an NSE Marketing Paper or Empathy and consciousness.
</ASIDE>

Intender Status

So anyway, this is what a typical Intender Status report looks like. What it’s showing is that the majority of people coming to this site believe they’ll return the same day (red bar), within the next 24 hours (green bar) or so far down the road they really can’t comprehend it (the white bar, labeled “> 6 months). Automotive manufacturers, governments, vacation properties and such, basically any organization with long decision cycles tend to be interested in the blue (1-3 months) and white bars, everybody else is interested in the red, green and yellow (1-3 days).

<ASIDE>
I appreciate that many people may be looking at these charts and wondering “That’s useful how?” First, please remember that NextStage came into being to prove the technology in our patent worked hence many charts are based on being able to predict, cause and propogate behavior. We’re changing our focus at present (scary, huh?). Second, yes, these charts do quickly lead to suggestions and action paths for improving marketing results. Just takes a little training (surprise!).
</ASIDE>

Tying this kind of data to a Recency metric would be interesting, I think. It would probably point to a way of being able to cause someone to return to a site at a precalculated point in time (and I’m already thinking of looking into certain sociology fields as I’m sure the research has been done there). Do you happen to have some recency models from small to large data sets you (or anybody) can share with me?

I think and am not sure that we’re back to >trans-temporal reafference, something I first entered into the discussion in a comment to you in a previous post.

<RAMBLE>
It’s funny (to me, anyways). You mention that “…you can improve the probabilities by adding other information — past products purchased, number of customer service calls, etc. — but the primary screener is still the Recency effect.” I suppose (and this isn’t meant as a criticism, only as my mind pulling stuff together) that Recency as you’ve defined it as an excellent “big hammer” metric. i’ll recognize and honor that, in fact.

My studies, though…I’ve seen and NextStage has agonizing amounts of data on how different types of “noise” — social networks, fair-exchange, … god, the list is quite long — affect people’s behaviors.

I think and am not sure that this discussion leads to a simpler way to deal with what you call “dis-engagement”, recognizing it sooner (or at least through a variety of measurements. I’m a strong believer in triangulating for accuracy. If you look at some of my math you’d see that I often dodecalate and have been known to go much higher to improve accuracy and surety) and alerting stakeholders of it.
</RAMBLE>

‘Recency with Modifier’, Pain-Pleasure, Negative and “0″ Acceptance

I would agree with your “…this ‘Recency with Modifier’ situation as quite similar to the Pain-Pleasure, Accepting-Rejecting model…” et al. I disagree that ‘Marketing will have no effect on people with “negative acceptance”.’

I think one of the places marketing must focus its attention is on consumers with negative acceptance because the strength of an individual’s or group’s negative acceptance indicates how much they’re willing to act upon it (word of mouth, networking, etc).

Let me start by making sure our definitions are similar. Rejection (negative acceptance) is not the opposite of acceptance. The opposite of rejection and acceptance is indifference. This is similar to “What is the opposite of love?” The opposite of love is apathy. Apathy is also the opposite of hate. Both love and hate can be very strong emotions that direct a wide breadth of our neural resources towards individuals, groups, social systems, clan affiliations, so on and so forth. The opposite of a strong emotion is a lack of emotion. The height that one can love will always be equaled by the depth one can hate (see Where You Should Stick Your Ad and Why).

Similarly, the height that one can accept something will always be equaled by the depth one can reject it. Marketing will have its toughest job with people who are at the “0″ point. Something needs to happen to shake them from their indifference. Current and historical research indicates this is where social marketing methods truly shine. Also, these methods are so culturally specific as to allow near surgical precision in getting messages out (here the citation list is agonizingly long. Let me know if you’re interested and I’ll post a bibliography of others’ and NextStage’s research on BizMediaScience. Some quick reads would be the first section of Reading Virtual Minds, my posts on PersonalLifeMedia or my Social Network or Word of Mouth BizMediaScience archives).

“…acceptance is not static…”

You write “…acceptance is not static, it decays over time. And that is why the timing is so important, you have you have to get in front of decaying acceptance (dis-engagement) and act before the customer slides into negative acceptance. Otherwise, it often becomes too expensive from a Marketing perspective to reverse the acceptance. There is probably psych literature on this idea of ‘resistance to reversing acceptance’ though I am not aware of it. When I’ve seen this effect, it often looks physical to me, as in ‘a body in motion tends to stay in motion’. This is why I often refer to ‘Customer Momentum’, and it manifests in the parabolic nature of the graphs in the examples I provided.”

Well stated and well said. The reasons for rejection/negative acceptance are fairly well understood, easily recognized, so on and so forth. It does take some effort to turn rejection/negative acceptance into acceptance because the neural mechanism is more like a switch than a slide (your “too expensive” statement). Moving from the “0″ point to either side is a slide in neuropathy and this is why it’s a different problem to solve (and also why social marketing works so well in getting people’s attention but not so well at keeping people’s attention). The best description I can offer for the switch from rejection to acceptance is that marketers need to overcome a cognitive inertia in the prospects’ minds (your “Customer Momentum” and I’m so flattered you used a term so similar to my own, momentum and inertia). This cognitive inertia/customer momentum occurs because people tend to think they have an “idea” of why they accept or reject something and most of the time acceptance and rejection are based on emotional, or “belief” constructs (and this gets us back to my discussions with Chris Berry (isn’t it wonderful how inter-related all these things are? I love it). Marketing is powerless to turn a rejection into an acceptance unless it addresses the emotional cause of the rejection first (yes, you are correct, the literature is rich on this subject. Let me know if you want a research bibliography and I’ll make one available on BizMediaScience).

Obviously we’re in strong agreement although our language might differ. The one flag on the play I’ll offer is to your “…acceptance is not static, it decays over time.” I believe I understand what you mean. My rephrasing would be “action based on acceptance decays over time.” I offer that because once something is “accepted” it tends to stay there unless there’s lots of effort to dislodge it.

For example (and parents, tell your children to leave the room during the next paragraph), most of us never gave up an acceptance of “Santa Claus”. How we demonstrate (act on) that acceptance has changed, perhaps, and rarely the acceptance of the concept. We went from opening presents to giving them, from sitting on the elf’s knee to being the elf. The spectrums involved require us to accept “Santa Claus” throughout, merely to act upon that acceptance differently (okay, kids back in the room).

Yes, strong agreement that these effects occur everywhere. They’re part of human nature/psyche/wiring. If they didn’t occur I’d be concerned. It would be evidence of pod people (here I write of “Invasion of the Body Snatchers”, not people with iPods and such… and is there much of a difference sometimes?)

Yes, again strong agreement with your “…the more control a customer has over the interactions, the smoother (and thus more predictable) the response functions are.” This goes directly to my SNCR presentations about the history of technology being a demonstration of placing power in people’s hands. Love your examples, by the way, and especially the “Friction” comments. That will bear some investigating.

I did read your http://blog.jimnovo.com/2008/06/22/peak-engagement/. Many thanks for directing me there. I posted some thoughts there, as well. I’m seeing much overlap in what we do although not quite in how we do it; complimentary, corollary, congruence, … My two cents, anyway.

Howdy All,
this post is a response to Visitor Engagement: Time for a reality check? posted by Matt Belkin, 14 July 08 on Industry Insights The Omniture Blog. A friend sent me the link and asked for my thoughts. I read the post, formed my thoughts, asked some folks to review what I wrote, did a little editing and went to post it today. I clicked to post my comment and my screen went blank (quite common for me) hence I’m posting my comment here and on Responding to “Visitor Engagement: Time for a reality check?”. Susan Bratton calls me a Righteous Luddite. Not sure about the righteous part, definitely a Luddite.

Anyway, here’s what I posted as a comment to “Visitor Engagement: Time for a reality check?”

Hello,

If I read the paragraphs under the The Basic Premise of Measurement heading correctly the logical outcome is that the only measurements being made (or provide value) are those that are subject to intervention (”improve”).

I have to admit that confuses me. I agree that lots of things are measured with the goal of acting upon the results of the measurement. I also recognize that lots of measurements are made simply to know what they are and with no realistic hopes of being able to modifying what’s being measured. I did look in several texts (started with Horst De la Croix’s Psychological Measurement and Prediction, went to Sabins’ Remote Sensing: Principles and Interpretation (I’ve often thought this discipline most closely demonstrates concepts and principles that would be valuable to people doing web analytics. I mean, consider Sabins’ opening statement to the 2nd edition, “Remote Sensing is broadly defined as collecting and interpreting information about a target without being in physical contact with the object.” Admittedly website visitors aren’t the subject of the book and the theoretical constructs often apply, I think), skipped over to Fukunaga’s Introduction to Statistical Pattern Recognition with lots of side trips into other fields) to find something similar to “The basic premise of measurement is that you want to measure something so you can improve it, if necessary.”

The best I could come up with was that the basic premise of measurement is to determine if something is measurable. I think the reframe is valid and leads to some necessary concerns.

Example: Some factor, A, is recognized and known to exist within a system (a website). A measurement is taken. Modifications are made to the system and another measurement is taken. But factor A hasn’t changed. Perhaps something else has changed and not factor A, the recognized and known factor of concern.

So we recognize that whatever is being modified based on the above measurement has no effect on factor A hence whatever is being measured is not factor A nor does it contribute substantially to factor A.

Yet still we want to understand factor A and be able to effect it at will. Other measurements are made and only of things we know we can improve. Nothing in our existing measurement framework seems to affect factor A.

Shall we continue? Shall we pursue other metrics? Shall we investigate until we can produce a “this = that” metric that has statistical validity?

“The basic premise of measurement is that you want to measure something so you can improve it, if necessary.” Thus the answer is no, we neither pursue nor investigate.

Nothing in our existing repertoire successfully produces a “measurement so you can change it” tool for factor A therefore we can conclude any of a) factor A doesn’t exist (even though it is recognized and known), b) it exists and is unmeasurable, c) the existing toolset does not include a tool with the ability to take a measurement of factor A such that changes can be made to the system that affect factor A, …

However, the fact that what is measurable (and actionable based on those measurements) is changing is quite true. The horizon of metrics meeting the initial conditions of measurable and actionable is, like the observable universe’s, growing every moment.

And I freely note that I’m a researcher, not a web analyst. I did ask one of our researchers, someone with a long history of lab work, and they offered that the basic premise of measurement is to determine the fundamental characteristics of what is measured.

“It’s pretty simple — collect data, analyze, improve. I love this because of its simplicity and objectivity.” But if the only data being collected and analyzed is based on factors we know we can change then there is no objectivity in the measurement or method involved. The only factors being analyzed are those recognized as changeable in the existing paradigm (the toolset).

Let me offer an example that demonstrates moving from measurable but unactionable to measurable and actionable: We’ve been intentionally collecting meteorological data for about 250 years. There was nothing in that toolset that could affect climate until recently. We also developed the ability to collect meteorological data going back far into prehistory (I can provide references to paleoclimatology if necessary. A quick search of Science provided some 1012 references going back to 1900).

Thus things were measurable. Without being able to measure them we could never have learned a) that they did indeed exist, b) that they were affecting us or c) how to affect them. Thus does our observable (web analytics) universe grow from measurable but unactionable to measurable and actionable.

“…you introduce a level of abstraction on the data. You “dumb it down” introducing bias and subjectivity.” I agree with this to a point and offer Communicating Science to Business and Vice Versa for your reading pleasure. I also recognize that all web analytics solutions offer training and certification on their toolsets.

Are these trainings and certifications offered to insure that all abstraction, bias and subjectivity are removed when a consultant or in-house analyst uses a given toolset? I want to take part in a web analytics tool class where the word “interpret” isn’t used.

“So what happens when you start combining metrics into uber-formulas …? That model breaks, because you introduce a level of abstraction on the data.” This also confuses me. Given a formula that has several elements and each element is known to be valid and measurable then the sum of the elements is also valid and measurable. This is basic mathematical logic (distributivity); if x is true and y is true and conservation of units holds then (x + y) is true. This principle (not by name) is mentioned in this post itself when the subject is RFM.

“First, what kind of return frequency is “often” - two visits? Four? Six? That’s subjective. What is “important” content? The home page? An article? A support document? Subjective again. And what is a “long” time on site — 5 minutes, 10 minutes? Perhaps “long” means any visit that exceeds the average for the site that week?” This logic reminds me of one of my favorite jokes:

---

A man wants to know what 2 + 2 is so he goes to a mathematician and asks, “Professor, what is 2+2?” to which the mathematician replies, “How may decimal points do you want it out to?” He then finds a psychiatrist and asks, “Doctor, what is 2+2″ to which the psychiatrist replies, “Interesting. How did your mother treat you when you were two?” He then goes to an economist and asks, “Sir, what is 2+2?” and the economist answers, “What do you want it to be, my boy?”

---

I do agree that the types of values you reference need to be defined, qualified and quantified (and my experience is that they usually are by the business model of the organization requesting the metrics to be applied) before being used in any measurement and resultant decision process. This is something I believe Nathan Janitz states in his comment(s). I also believe that to suggest subjectivity is not part of the business world — even when it comes to something as definite as well-defined measurements — is naive. To that point, suggesting that “often”, “important” and “long” implies subjectivity without recognizing that the conversation drawn from was not meant to have any rigor is amusing, hence my offering of the above joke. This goes back to the reference to RFM. If the underlying (or “principle”) metrics are valid then the uber-metric is valid. And I also recognize that my paradigm might not be appropriate here. I do think the core concept of measurement (of anything) is scientific in nature. Ms. Debbie Pascoe writes in her Continuing the Discussion with Joseph Carrabis “Anyone who has ever used a tire gauge or a tape measure has employed a scientific calibration method.” Hence I offer Paul Davies’ “A scientific claim is taken seriously only if it can be tested by others in a disinterested (not uninterested) way.”The true test of any measurement is that it can be duplicated when attempted under similar situations with similarly calibrated tools. That concept, I believe, is where true simplicity and objectivity will demonstrate themselves first. After that is achieved, companies that make measurement tools will find a way to make the measurements personally relevant to consumers (see Framing Science). This, I think, hearkens back to Ms. Pascoe’s comments and thoughts.Jim Novo offers “Simple to explain to just about anyone…” and I worry if our decisions on usefulness are going to be based on simplicity of explanation. If so, then everyone who doesn’t understand solid-state physics, magneto-interference pattern imaging, magneto-optical coding, lasers, satellite telemetry, … oh, heck, let’s just go to basic electricity… throw away your digital cameras, computers, pdas, smartphones, stop watching TV, move out of your homes completely, never again use cars, buses, boats, trains or planes, …

But (!!!) if the decisions on usefulness are to be based on simplicity of use? That’s a completely different story. My experience is that most people don’t care (or even care to know) how complex something is “under the covers”, they just want to know that “when I do this, that happens” and even more accurately “when I do this, I get what I want” (this is the history of technology concept I often share (see The Long Tail, Part 1, Rocks, Hammers, Competition and How People Get Left Behind or VerizonWireless’ 20 year plan)).

Bravo to some and not all of what Steve Jackson adds. In the end, any metric that provides value will survive, even if that value is simply that something is being measured.

I also appreciate Nathan Janitz statements. His statement “…you also can’t catch a mouse without looking at the right information….all of the right information” is, I think, exactly to my point.

Allow me to quote Joe Tragert, Director, Market Development, EBSCC Publishing, ‘The question isn’t ‘Which mouse trap works better?’, it’s ‘Did we kill the mouse?’” You can have all the metrics in the world available to you. And if your goal is still unachieved? Or if none of the available metrics address your goal? Then it is time to either give up your goal or create new and valid methods of measurement that help you achieve that goal.

Hello again,
some of my comments can’t be published as comments in their original threads because I include images, hence I’m publishing some responses to readers’ comments as additional posts.
Sorry for the confusion.
Imagine what it’s like on my end.
—
I picked up the thread of this conversation at Back into the fray comes Joseph! and am planning on getting more involved in this blog again simply because some folks took the time to comment, therefore I owe them the honor of responding to their comments.

Yes, I know…I’m just like that. Anyway, here I’m picking things up at Dr. Geertz’s February 5th, 2008 at 11:20 am comment.

Enjoy.

The Good Dr. Geertz, 5 Feb 08, 11:20am

Since Dr. Geertz published this comment he and I have had some rousing and wonderful phone and Skype conversations, one of which I extracted and published in My Easter Eggs Critiqued. Dr. Geertz was also kind enough to read through NextStage’s patent (several times. Somebody buy that man a beer!) and lend his expertise to our internal discussions. It’s wonderful when one finally learns the identity of one’s advocates.

Dr. Geertz offers that I was suffering from some consternation in my remarks above. I thank him for the thought and suggest that I was biting my tongue until I knew we had the patent more than anything else.

And also and for the record, I do not object to anybody else’s definition of engagement and their use of that definition as a description of some metric or tool they’ve developed, I only recognize that their definition is different from the one NextStage uses. I also believe that NextStage’s definition is more realistic and calculable when it comes to determining future acts by a given (visitor) population across the broadest possible number of platforms throughout time.

Theo Papadakis, 5 Feb 08, 2:43pm

Hello,

I read through your post on Avinash’s blog a few times and applied both simplified logical calculus and Alexsander&Dunmel’s Logical Calculus of Consciousness to it. I think there’s a flaw in your original proposition. You write “If x is engaged with y, x is related to y.” This translates to something like “If y = f(x) then (x,y) are in some space S” which is true. Prior to that you write that what you’re defining is a one way relationship and that’s not valid. This inconsistency demonstrates itself later on in a logical analysis of the steps involved in the hypothetical demonstration you offer.

You write “two kinds of engagement with an object, positive and negative, by which I meant that someone can be engaged with an object.”

Hmm…I think I explained this in Back into the fray comes Joseph!, Online Engagement: What Exactly Is It?, Meet Online Engagement’s Little Friend, Satisfaction and elsewhere. I’ll also be following these up with an AllBusiness.com post (it’ll go live on 21 Jul 08, 6amET, and is called “The Money Is Where Engagement Meets Satisfaction Online” for those who’ll go looking).

Engagement is the same because the same parts of the brain are active. You can be engaged by pain or pleasure. Your response to the pain or pleasure will (probably) be positive or negative and the parts of your brain telling you “Pay attention!” — those parts that dictate whether or not you’re engaged — don’t really care about your response, only that your attention is focused sufficiently such that the response can be effectively and adequately acted upon.

You then write “Engagement is not itself a psychological state but involves a mixture of rational beliefs and psychological states…”. I’ll admit to having some challenges with this. I think my definitions of “psychological state”, “rational” and “beliefs” are inadequate to your use of those terms. Beliefs, for examples, stem from a part of the {C,B/e,M} matrix that doesn’t rely on rationality.

<ASIDE>

I’ve written about the {C,B/e,M} matrix in several places such as Guest Blogger Joseph Carrabis Answers Dave Evans, CEO of Digital Voodoo’s Question About Male Executives Wielding Social Media Influence on Par with Female Executives, Responding to Christopher Berry’s Vexing Problem, Part 3 post, Responding to Christopher Berry’s “A Vexing Problem, Part 4″ Post, Part 2, my comment to this post and several other places. I can also offer a bibliography of how the {C,B/e,M} is derived for those with an interest.

</ASIDE>

A simpler version of the {C,B/e,M} matrix is described in Reading Virtual Minds and deals with the different ego states of Core, Identity and Personality. Where ever they come from, beliefs may be informed by data but they are not bound to rational thinking. They are, as described in my exchanges with Christopher Berry, based on anecdotal thinking and subject to all the pitfalls inherent therein. You have a good primer on this material in Schacter and Scarry’s Memory, Brain, and Belief

Likewise “rational” is not an adjective I’d use with “beliefs”. “Psychological states” I recognize as “a mental condition in which the qualities of a state are relatively constant even though the state itself may be dynamic”. Some worthy albeit diverse reads on this subject include Tomasello’s The Cultural Origins of Human Cognition, Gazzaniga’s Mind Matters: How the Mind&Brain Interact to Create Our Conscious Lives and Allison’s Minds in Many Pieces (sorry, couldn’t find those last two online).

You also write about the “degree of engagement” and tie your definition to “psychological investment/involvement”. I’m quite sure I don’t understand the use of these terms in these contexts. This is not to indicate they are invalid, only that I don’t understand them.

Lack of understanding, however, is much the central problem that needs to be addressed in these discussions. Any metric is meaningless unless the language of what is being measured is specific. It might make good business sense that some group has a proprietary definition but when that definition no longer applies the business suffers. It’s probably better (opinion warning) that a common definition be used and that businesses work at being more accurate in providing a metric based on that common definition (just as a point, NextStage uses common definitions that anybody can use. It’s our methods and technology that are proprietary, not our definitions).

If I understand your hypothesis correctly, you suggest that your definition of engagement can’t be measured, only inferred. I don’t think I’ll argue that point, only question it’s utility. Breaking down the logic I come up with

a) Hypothesize some phenomena

b) Define the phenomena such that it can’t be measured using standard and readily accessible tools

c) Take measurements using standard and readily accessible tools

d) Develop a formula that takes the measurements from the above

e) Claim that what is measured, via the formula, is the phenomena.

There are some logic holes here (to me, anyway) and I won’t go through them in detail. I will offer that there must be an unbroken chain of physical connection between how something is measured and what something is in order for that measurement — hence the resulting metric — to be valid.

I’m happy to continue this discussion if there’s interest. I do want to point out that the lack of precision in both definition and logic will probably be impedimentory. Do also note that I don’t recognize problems with “measuring degree of engagement” based on the definitions I use and the measurement methods applied (the three “real questions” in this thread’s original post).

Eric Peterson, 5 Feb 08, 8:28pm

“Save asking every person who comes to a web site “are you engaged?” (which I would assert is A) impractical and B) just as imprecise as my calculation, if not more so!), how would you propose we ground truth engagement and test the hypothesis?”

I can’t speak for Dr. Geertz and this is where that nap-of-the-earth flying thing comes in, I guess. I’ve often described Evolution Technology (ET) as doing exactly that, asking every person to a website “Are you …?” and then responding via whatever business rules are in place. Hence this is not impractical. Is it imprecise? That depends on how much precision you’d like. In more tests than I care to remember NextStage’s ET averaged 83% accuracy predicting outcomes (what people would do, when they would do it, etc.).

Re “satisfaction”: Yes, agreed. This is something I mentioned during one session at eMetrics SF 08 and also in Back into the fray comes Joseph! and Meet Online Engagement’s Little Friend, Satisfaction. I mentioned a future AllBusiness.com column post on this subject I’ll share some highlights here.

First and for explanatory purposes, I’ll shade in the figure in Back into the fray comes Joseph! so that it looks like the one on the right.

<ASIDE>

Those shaded areas actually do have meaning. People may have heard me mention that ET makes decisions and offers suggestions based on solid probabilities. The combination of those shaded areas (and their extensions) creates a bell shape. Imagine that bell filled with metal. Now you have a solid bell. Imagine every atom in that solid bell representing a probability that something will or won’t occur, that a visitor will do this or that, will respond one way or another, will think this way or some other way. The accuracy of the prediction — the likelihood or probability that something will occur — is based on the atom’s height within the bell and its distance from the bell’s surface. Things closest to the center of the bell are more likely to occur than things at its edge.

Thus something midway up the edge of the bell is less likely to occur than something at the bottom of the bell but dead center.

Hey, this is my world. I’ve learned to live in it and I’m not claiming anybody else has to or should.

</ASIDE>

What happens when you map those quadrants onto a standard Engagement chart is something like on the right. The money (if you will) is in the periodicity. The periodicity can depend on several things, most of which are business dependent. What the periodicity gives you (via linking to some standard (at least what I think are standard) web analytics values) is a near surgical ability to recreate optimal satisfaction-engagement periods at will.

Thus ends the 5 Feb 08 comments

Again, whoosh! At least I got a column out of this. Next time I’ll start with Dr. Geertz’s 6 Feb 08, 11:56am comment.

And thanks for everyone’s patience.

Proving that Serendipity is doing it’s job, I’ve had in my mind that it’s time to return to these thoughts and several people contacted me to find out if I was going to return to this blog.

Okay. Into the deep end first.

My time away has been due to busyness. Perhaps some readers have heard, NextStage Received its first patent on its Evolution Technology. For years we’ve been intentionally below the radar, now we seem to be becoming a recognizable object rapidly approaching from the far horizon. Now that we’ve left nap-of-the-earth flying I’m able to discuss things more openly, me thinks, hence some of my responses now and in the future.

Are the visitors happy?

One of the things I did while I was away was talk with a few people (about 100 so far) about what I’ll call The Purpose of Web Analytics. I did this research because of something I wrote in this thread above, “…all these analytics are worthless unless they create happy, satisfied visitors, yes?”

I’ve talked with upper management in education, politics, at national telecoms, financial institutions, transportation, recreation, … a pretty diverse group. Most of them were involved in marketing products or services or some other form of gaining marketshare. None of them were web analysts or involved in web analytics except that they received reports and were expected to act upon them. None of them were particularly happy about being made accountable to a system that (they believed) wasn’t measuring … and here’s where the challenges really made themselves known.

What was being measured? Lots of money was being spent and lots of people were being told that the measurements mattered and as one fellow explained, for the amount of money they were spending they expected some consistency.

“What do you mean by consistency?” I asked.

He pretty much didn’t know. He and those with him said lots of things and it could be distilled to a general dissatisfaction that there wasn’t a single model that they could consistently use and derive actionable meaning from. The dissatisfaction grew geometrically when the discussion got into executives making decisions based on sales presentations rather than a given product’s specific informational abilities.

At one point I leaned towards a speaker and quietly said, “Remember, Joseph friend,” and everybody laughed because the tension in the room was broken.

I reference these anecdotes because one of my original hopes for this platform was an increase in understanding and acceptance of some mutual goals regardless of discipline or tool platform.

In the end, doesn’t it all come down to “…all these analytics are worthless unless they create happy, satisfied visitors…?”

If I can’t act on it, it doesn’t exist

The next item I wish to thread into this discussion comes from an online conversation I had with Critical Mass’s Christopher Berry about why web analytics seems to be a harder sell in Canada than in the US. You can follow my side of the conversation in Canadian Based Business Differences — Responding to June Li, Christopher Berry and Jacques Warren, Responding to Christopher Berry’s Vexing Problem, Part 3 post, The Language of Web Analytics - The Hard(er) Sell in Canada, Responding to Christopher Berry’s “A Vexing Problem, Part 4″ Post, Part 1, Responding to Christopher Berry’s “A Vexing Problem, Part 4″ Post, Part 2 and Communicating Science to Business and Vice Versa and links are provided to Christopher Berry’s side on the conversation in those posts. I’ll invite people to pay particular attention to Communicating Science to Business and Vice Versa because (and as Mr. Berry noted) the summation is what counts, “Business is different. Business (me thinks) tends to be more ‘Tell me how to use this’ hence most business proposals and reports start with Christopher Berry’s nuggets then go into explanations.”

My research is convincing me that (what I recognize as traditional) web analytics is going to be losing its authoritative power in the coming years. I think web analytics (and yes, this does go back to my original hopes for this blog) will evolve (just as anything will if it is going to survive in a given changing environment). What will it do and look like? I have some ideas, of course. Just ideas at present, though. More things to research before putting down on paper (or in a blog) at present.

This does tie into my comment re Avinash Kaushik’s “…we shouldn’t use ill defined engagement metrics as a proxy for something solid like a sale.” I’ve been an oft-times unwilling father-confessor to businesses frustrated by ill-defined metrics of any kind and wanting something that is justifiable a) financially, b) scientifically, c) arithmetically (forget mathematically) and d) produces some kind of “do A, get B”, “this-equals-that” link between action and outcome.

The comment I love about this is “If I can’t act on it then it doesn’t exist”, ie, it’s noise, a distraction at best and something best ignored. This was a wonderful statement used in a business practices discussion.

I’d really enjoy being involved in a web understandability/measurement/future usability discussion that has as its theme “If I can’t act on it then it doesn’t exist.”

“To measure and analyze on and offline behavior and then try to predict who to market to by figuring out what they think is not doable with one tool or one metric.”

I responded earlier to this comment. People who attended either the Toronto ‘08 or SF ‘08 eMetrics conferences are probably well aware by now that NextStage has patented a technology that can determine how someone is thinking through any programmable device. I won’t go deeper into the topic here except to offer a comment I posted on Jim Novo’s blog about the {C,B/e,M} matrix and its use in marketing and analytics.

Picking up where I left off with Jim Novo’s comments in this thread…

I finally had an opportunity to read Jim Novo’s Measuring Engagement and its related Framework for Engagement posts. I truly enjoy Jim’s writing style and the points he makes.

I especially enjoy and appreciate his referencing Relationship Marketing because it places people center stage. Understand people and you can both understand and predict what they’ll do. Watch only what people have done and you can only understand their actions in a specific historical context, you can only predict what they’ll do when the confluence of events that led to their original actions repeats itself. Exactly (and don’t hold your breath). Relationship marketing works at the question “…all these analytics are worthless unless they create happy, satisfied visitors, yes?”

Jim writes “The challenge with this model - and probably why it isn’t more widely known - has been the data, it’s a very analysis-intensive model…”. Yes. Agreed. If Jim (or others familiar with these concepts) is reading (or perhaps at the next conference we meet at), I think this is where being able to substitute cognitive heuristic models makes sense (see Liberation and Heuristics or Responding to Christopher Berry’s “A Vexing Problem, Part 4″ Post, Part 1. I’ve also written elsewhere that I often wonder why more businesses don’t make use of cognitive heuristic models).

For example, I’ve recently been applying heuristic models to helping adult second language learners increase their language acquisition abilities. That’s a traditionally very tough nut to crack and (so far, anyway) I’ve been able to isolate neural activity that tends to make adult language acquisition challenging. Example 2, using heuristic models in the above grew out of learning which heuristic models are used (non-consciously, of course) by which personality types in their decision making processes. This non-conscious heuristic model selection process is being integrated into NextStage’s Rich Personae. These and some other areas of my studies are intensely data-rich models that can be reasonably simplified via cognitive heuristics.

I also strongly like your concept of dis-engagement, although I tend to use a methodology that incorporates “satisfaction” into the scaling system (see Meet Online Engagement’s Little Friend, Satisfaction. I shared that the complete form of this during a discussion at the SF ‘08 eMetrics. It looks something like the figure on the right.

Some definitions to help in understanding; the x-axis is Engagement and is a measure of the amount of pleasure or pain an activity is giving you. If something is giving you either pleasure or pain to any degree your attention is focused on it, hence you are engaged by it according to the definitions documented in Attention, Engagement and Trust: The Internet Trinity and Websites. The y-axis is Satisfaction and is a measure of acceptance and rejection of some internal state and/or external event.

I believe what you are referencing as “dis-engagement” is what we recognize as the slide from high acceptance to “0″ acceptance. Note that this is not rejection (as rejection is an active negation of acceptance) it is a lack of acceptance. I appreciate that the difference might be subtle and I believe that difference is significant. Rejection — the active negation of acceptance — can be thought of as someone pushing something away. Zero-acceptance is the point where one can “take it or leave it” and the internal state and/or external event does not have any value assigned to it, hence doesn’t register strongly in the mind/brain.

Mapping this figure to real world experience, you always want visitors/consumer/etc to be in the first quadrant (where the green curve is). People are both positively engaged (they like what’s going on) and positively satisfied (they accept it gladly). Depending on what you’re selling you may or may not want people in those other quadrants. The second quadrant (bottom yellow curve) indicates someone focusing on painful experiences or information, the fourth quadrant (top yellow curve) indicates someone who finds pleasure in painful experiences or information. The third quadrant (red curve) is where visitors/consumers/etc often end up and marketers/businesses don’t want them to be — the former are actively psychologically and physically moving themselves away from a business/product/service.

I’ll offer that the above is also a reasonable representation of your:
1. Define / Measure Engagement – any way you want to, as appropriate for your business; whatever activity or combinations of activity you feel appropriate
2. Measure dis-Engagement – the absence of Engagement, as in the visitor / customer stopped doing whatever it is you define as Engagement for your business model

I think where the image above (and the math behind it) adds real value is with your “3. Take some kind of Marketing or Service action to slow or reverse the dis-Engagement with dis-Engaging folks” because it provides enough information to know how, exactly, visitors/etc are “right now” interacting with your marketing information.

I also agree whole-heartedly with your statements about predicting “dis-engagement”, etc. I would love to see the data you used in your example and apply it to the above. I’m willing to bet that satisfaction/acceptance was the real driver (and I won’t get into the depths of group satisfaction/acceptance states here (really, Joseph? You’re going to leave something out? Whatever for?)). I did get a kick out of your graph of email response rates falling over time. It was very similar to the results we found in our research on how to design an effective email newsletter. Bravo! I always love it when our findings match others’. Gives me hope we’re doing something right.

<ASIDE>
For what it’s worth, much of the rest of what you’ve written in your post is so close to what we learned in our email newsletter research that the overlap is astounding. Not surprising, I guess, as you’re listing an email-based experiment. It would be interesting to learn what else the rules we discovered pertain to. Let me know if would like to explore this.
</ASIDE>

You also list an implication about sending different messages to different segments. Yes, agreed. I believe the above allows for much more targeted and action-driven messaging (based on much of what I’ve shared above).

Perhaps, in the end, we’ve derived nothing more than a simplified mathematical model (complete with suggestions for better outcomes) of Relationship Marketing?

Whoosh!

Took me two days to put the above together folks. Sorry for the delay. More to follow. Soon.

Promise.

This post picks up where I left off on Starting the discussion: Attention, Engagement, Authority, Influence, … and is a follow on to Eric’s two parter (Measuring Engagement Online: The Next Stage and Measuring Online Engagement: Step One). Eric references me as “Mr. Carrabis” somewhere in there. Oh, he must think I’m ancient to call me “Mr.” I will be going through the rest of the comments people made to my original post and commenting on them there. It just takes me time (nobody noticed, I’m sure) because I tend to think things over a bit before responding to them.

And FYI, I’ll be posting elements of the equation’s derivation on BizMediaScience. I doubt anybody is sitting on the edge of their seat waiting to see what I integrate and if I use Tau or Gamma functions, so me thinks that’s best. I will be posting my thought process here, though, and welcome comment, suggestions and so on.

Form Follows Function

Right now the definitions of Engagement that I’m working with are Eric’s (Engagement is an estimate of the degree and depth of visitor interaction on the site against a clearly defined set of goals.), the one we use at NextStage (Engagement is the demonstration of Attention via psychomotor activity that serves to focus an individual’s Attention.) and the dictionary’s (participation, involvement, involution).

I’d love to hear anybody else’s definitions as I’ll work to incorporate them provided they adhere to the philosophy proposed in Starting the discussion: Attention, Engagement, Authority, Influence, …:

1. What do you mean when you use the words “engagement”, “attention”, and “trust” online?
2. Can you repeatedly measure what you mean by them so that there’s a reasonable surety that what you’re measuring is what you mean by the terms you’ve used?
3. Can you make these measurements through a commonly used web-enabled device?

Both Eric’s and NextStage’s definitions pass muster on item 1 above as both are declaratory statements. It’s also worth noting that both definitions contain the meanings of participation, involvement and involution in them as conceptualized in Eric Peterson’s Engagement Project and the Engagement Equation, Part 1. So far, we’re doing real good.

Item 3 above is a definer to our variable set — we are only allowed to use measurements that can be made through a commonly used web enabled device. Note that a “definer” isn’t something that sets limits. A limit would be something like “You can only use screen size and color depth”. The phrasing of item 3 is intentional. As technologies change so will the measurements that can be made through a “commonly used web enabled device”.

Item 3 is really our “looking forward” element. When web analytics started (and when NextStage started, for that matter) web enabled phones, pdas, iPhones, Smartphones, …, were unknown. Fortunately I’ve never let something’s lack of current existence deter me from anticipating it (if you’ve read our patent, you have an idea).

Item 3 covers our “extendable” and “extensable” concerns. It allows us to create a mathematical form that can alter its requirements as new capabilities and technologies come on line without sacrificing its ability to calculate and return real (business) value.

Likewise, with Items 1 and 3 covered, all that remains is to isolate a set of easily captured variables that satisfy the definitions in item 1. The equation — the mathematical form I’m using — allows for new variables and interfaces as they arrive. In other words, the final equation will be good now and way, way down the road. Again, this is something I learned from writing up NextStage’s patents; be as far thinking as possible.

There are other concerns and considerations that (by my nature) I’m putting into the mathematical form. For example, accuracy. Accuracy is a function of target size, not mathematical rigor. Accuracy of 10% with three variables active can quickly rise to 90% accuracy with as few as four or five variables active. Let me give you a “marketing” example. You’re selling to a) 53yo b) white c) males and you’re capturing 10% of that market. But what if you’re selling to a) 53 yo b) white c) males in d) NH with who e) are business travelers? Ah, well, now perhaps you’re capturing 90% of the market.

Some people aren’t aware that the opposite can also be true; it’s possible to achieve (for example) 90% accuracy with three variables and dwindle it to 10% when more variables are present. Imagine a bullseye style dartboard. You can get lots of darts in the yellow and good for you; that’s high accuracy. Then again, there are only five colors you can hit (five variables in the equation).

Now imagine a more traditional dartboard with a very small center area and lots of other areas indicating different values and multipliers. Both types of dartboards are circles, yet add or change a few variables and accuracy as a percentage of dead-centers is shot to heck.

Thus the mathematical form should allow for various degrees of accuracy based on what variables one is able to measure. This means that careful definition of the initial and possible variable sets is critical to the success of the metric as well as its accuracy.

Am I boring you yet?

Without driving you to drink, I chose to make the equation solvable by using three spaces, A, I and V. A is the set of all possible solutions of “engagement” (note that I’m not specifying “online”). I is the set of all possible interfaces that will ever exist (note that I’m not limiting this to “online” or even “machine”) and V is the set of all possible measurements that can be collected for a given interface I (note that I’m not limiting this to real-valued measurements). From here we can go through about two pages of finalized equations and whole lots more to get there.

Or I can show you the next to the last form in the derivation. The image above is important for lots of reasons. First, it’s really nothing more than a mathematical shorthand for Eric’s definition. But wait, there’s more!

It’s also a mathematical shorthand for NextStage’s definition. Also for the dictionary’s definition.

Are you excited yet? I love this stuff! And it gets better!

It’s also a mathematical shorthand for Eric’s definition blended with NextStage’s. Or for any group of definitions that follow Items 1, 2 and 3 above. If you can define the term (1) and can measure it repeatedly through a standard, commonly used interface whatever that may be now or in the future (3) then you get the formula above and it satisfies (2).

Want to know the best part? The formula above is also a mathematical form of SQL statements. You have some data hanging around somewhere and you want to create something you call “engagement” and you want to interrogate the data to create it? No probs. The formula above does the job. It doesn’t matter the size of the query result set nor the initial parameters.

Let me give you an example

Say you have some huge data set of N records and you’re really only interested in some portion of those records. That’s “N - n”. And you want to know how many visitors in this “N-n” group were “engaged” as you define it. That’s “h_i(v_i)” and is also where Eric’s definition, NextStage’s, Eric’s and NextStage’s, Eric’s with five parts of NextStage’s, two parts of somebody else’s, three parts of another person’s, …, goes. Now say of those “N - n” records you want some sequencing (maybe most “engaged” to least “engaged”, maybe something else). That’s “x_i - t_i“. And you want everything else constant, like geographic location and income level and who knows what else. That’s “D[h_k(v_k)]”. Not only can you hold things like geography, income, etc., constant, you can also hold other definitions of “engagement” constant while plucking out the one form you want for a given application or deliverable. And you only want “engagement” for a given interface (web as opposed to mall kiosk, etc). That’s “V” or “I” or “A”. The above basically reads as

SELECT “h_i(v_i)” AS “ENGAGEMENT”
FROM “N - n”
WHERE “D[h_k(v_k)]” = somevalue
AND “D[h_j(v_j)]” = someothervalue
AND …
GROUP BY “x_i - t_i”
ORDER BY “V” (or “I” or even “A”)

The “But”

The only caveat to the above is that each solution to A and I and V requires the interface to be monitoring a thinking being. I won’t even suggest “person”, but (there’s the “but”) you have to be evaluating “engagement” with something we can most easily define (but not limit ourselves to) as “a visitor”. This “but” has some teeth (and a good thing, too!). This means the only v_i available — the measurements and metrics you use — have to be something “a visitor” would do, which means they have to be recordable by a standard interface (item 3 above).

Okay. I’m done for today. It’s taken me about two weeks to come up with a derivation (that’s all the pages I’m not showing you) that works for any definition of engagement on any interface that has any metrics that fits items 1-3 above. Soon I’ll start showing how Eric’s solution fits and what inferences can be drawn from it.

Okay. Something controversial to start.

The only problem is that to me what I’m offering isn’t controversial. It deals with measures and measuring.

Measuring what?

Well, when you put some Flash object on a page. What can you measure? I’m not a web analyst so to me the answers are obvious; measure the psychomotive and psychobehavioral cues that visitors are demonstrating. These and other elements are what make up the Cognitive, Behavioral/effective, Motivational matrix or “CB/eM”. The CB/eM tells you things like age, gender, buying styles, best branding strategies, impact ratios, touch factors, education level, income level, etc.

I understand that not everybody finds these things fascinating. Anthropologists, behavioral and cognitive psychologists, psycholinguists, sociologists, behavioral etymologists, …, those kinds of people go nuts over this kind of stuff.

Some of the stuff listed above has to do with things like attention, engagement, authority, influence, …

This is where it gets a little … umm … interesting. I see words like the above used a lot in web and web based “behavioral” analytics. This is a mystery to me. Much in the same way that an anthropologist and a microbiologist use the term “culture” to mean two very different things, I think the way web analysts and web-based behavioral analysts use the terms attention, engagement, authority, influence, … to mean two sets of very different things. I’ve often commented and written that behavioral tracking as defined by the industry doesn’t track human behaviors at all. Not as I understand them, anyway.

Okay, so what do I mean by these things? To recycle content from Attention, Engagement and Trust: The Internet Trinity and Websites:

• Attention is a behavior that demonstrates specific neural activity is taking place.
• Engagement is the demonstration of Attention via psychomotor activity that serves to focus an individual’s Attention.
• Trust is what the consumer — well informed or not — gives the site (or whatever is asking for the consumer’s Attention) when their Engagement is rewarded with useful, relevant and meaningful information.

I can go into authority (something fellow SNCR member John Cass caused me to explore and which I’ll be publishing about soon) and influence. I know how to measure what I mean by these things. But the definition I use don’t come from the web world even though what I mean by them can be measured through any number of commonly used web-enabled devices.

And while I’m not sure, I don’t think my definitions are those used in web analytics and web-based behavioral analytics. What I can offer is that my definitions — and this is my opinion here — are more closely aligned to what is generally understood in the literature (in the disciplines I mentioned above) than what is meant by web analysts and web-based behavioral analysts.

I’m not equating “close alignment with literature” with “more valid”, merely offering that different paradigms can offer more understanding than any single paradigm alone. But right now I think I’ve gone on enough. I came here to learn. I’d really much rather hear what others think, understand what they measure and what value they assign to it.

So for me the real questions are:

1. What do you mean when you use the words “engagement”, “attention”, and “trust” online?
2. Can you repeatedly measure what you mean by them so that there’s a reasonable surety that what you’re measuring is what you mean by the terms you’ve used?
3. Can you make these measurements through a commonly used web-enabled device?

To push the conversation along, here are some external links that are worth reading:

• Attention, Engagement and Trust: The Internet Trinity and Websites
• Get the attention you’re already paying for
• Defining Attention on Websites & Blogs
• Focusing Your Customer’s Attention
• Expertise - Who Decides?

Remember, this whole blog is about having a conversation. Do you have these same questions? Do you agree with the definitions I propose or do you have different definitions? And most importantly, how do you answer the three questions I posed above?