Eric Peterson added the following ...

Joseph,

You and I may end up campaigning for some type of navel gazing award, at least until we can convince the rest of the world to jump in and join the conversation … but I’ll have a whack at your conversation.

I think what you’ve pointed out is the profound difference between the way the so called “real” world works and the way stuff works online. I have little doubt that “Engagement (your caps) is the demonstration of Attention via psychomotor activity that serves to focus the individual’s attention”, not that I’m entirely sure what that means ().

The problem I have, and you perhaps lead me to this place, arises when you ask your three question about the practical nature of measuring attention, engagement, and trust, especially your second question, “Can you repeatedly measure what you mean … so that there’s a reasonable surety that what you’re measuring is what you mean by the terms you’ve used?”

Now I’ve written a great deal about measuring engagement online, having gone so far as to publish an open framework for measuring what I call “engagement” on the Internet. And I perhaps am looking at this question harder than the average person, but I’m not sure that I can cross the border between the soft-and-squishy stuff we measure online and the soft-and-squishy stuff between our ears. But I think that’s what you’re asking, isn’t it?

Assuming the ability to generate a “reasonable surety” about the value of my engagement metric is, well, impossible. Just as impossible as knowing whether a high bounce rate means your page sucks, or knowing whether a high conversion rate means you have a well-designed path. In the online realm what we’re often looking for is, um, proxies for the truth.

It’s like Sterne stumbled into at Emetrics awhile back (you were there, I saw you!), “It’s not precise, but it’s true.”

No truer words have ever been spoken — reasonable surety assumes some level of precision that simply doesn’t exist in the online world. Sucks, but it’s true. And so we go looking for reasonable proxies instead — metrics that are useful if not perfect that get the job done.

The criticism of my engagement framework (not that you’re criticizing, but some have) is that it’s too complicated. Kind of like consulting a neuroscientist, shannonist, semioticist or semanticist when all you really need is a page view count. But given the profound difference between the way the online world seems to work and the way things happen in “First Life” (sorry Marshall!) is it any wonder some of us are looking for more robust measures?

So I have little doubt your measures of Engagement (again, your caps) are valid and right and useful, but these are early times my friend and some of us will have to walk before we can run.

I wonder if anyone else is reading this blog?

Joseph Carrabis added the following ...

Howdy,
An example of “engagement” would be two people talking together as follows: Person A nods in agreement every time person B makes a point. That nod (for most people) is a non-conscious psychomotor activity that demonstrates person A’s attention is focused on what person B is saying, hence person A is engaged.

The definition you supplied, Engagement is an estimate of the degree and depth of visitor interaction on the site against a clearly defined set of goals, can be thought of as similar (I think they’re similar, anyway) and a definition of “visitor interaction” needs to be supplied for similarity to be complete. There are some definitions to the equation you provide in the same post and I’m not comfortable with them, perhaps because I don’t recognize anything that indicates the visitor’s internal states during the navigation (and notice how carefully I’m staying away from a discussion of Turing machines here).

I believe your definition is fine for what I read it as and my understanding of it can be greatly in error. I also recognize that what the two of us find interesting to measure is probably vastly different.

It comes down to a belief I have (and have shared with others) that all the charts and all the graphs mean nothing if they don’t explain how to get to the next step, what to expect once you’ve taken that next step and provide a better than average indication of where to go after you’ve gotten there.

• The quest of analysis is accurate prediction, me thinks. This is scientific inquiry applied to engineering, for example. We’ve observed enough of A to know when they’ll nod at B.
• This gives B the ability to prepare for A’s nods and direct A to where B wants A to be.

The increasing microsegmentation and diversity of audiences being able to accurately predict A’s behavior is becoming paramount (an opinion. Not enough data to state that empirically, yet).

So where does your “engagement” and my “engagement” overlap? And before that, do they? And is there any gain (ROI) in their overlap? Or are they simply measuring two completely incongruous things (the mathematician in me kick’s in and shouts “We can solve for that!”)?

My goal in this blog (in case nobody noticed) and part of my work this year will be to determine how these two areas of interest overlap, when, where and why.

Joseph Carrabis added the following ...

Forgot to mention that the type of engineering described is “Modality engineering”

Jim Novo added the following ...

As background, you might want to read this:

http://blackbeak.conversionchronicles.com/2008/01/29/measuring-online-engagement-re-visited-and-introducing-the-rean-model/#comment-6761

Joseph, I can appreciate the need to fit this all into published frameworks because it leads to continuity between discplines.

The problem I have with Attention - Engagement - Trust model, which makes sense and is supported by the academic literature, is the concept of Trust is a bit problematic, since a person can Trust a company completely and never do business with them again.

As a Marketing person trying to optimize a system, I’m not interested in spending money on people who are never going to do business with me again - even if they Trust me. The reverse is also true. Personally, I do business with lots of companies I don’t Trust at all based on their past behavior, but they are the only alternative for various reasons.

In other words, Trust doesn’t directly address “dis-engagement” and the allocation of spend optimally. It sometimes gets close though, because the creation of “dis-Trust” often will lead to dis-engagement, so I might be able to live with it.

But I get the sense that what we’re really talking about is what happens after Trust, what happens when “Engagement” ends? Because it is the dis-engagement that is most important, it’s the process through which a relationship moves from relevant to irrelevant.

To get a better handle on this issue, perhaps you could comment on this scenario:

You have a friend you used to be close to, and all of a sudden you realize they have not contacted you in awhile. You try contacting them, but they don’t respond. After a couple of tries at this, you decide not to try contacting them again. Pretty soon, you’re not friends anymore.

What are the stages / attributes of a dissolving frienship from the literature? What are the possible outcomes *after* the Trust stage has been attained? Perhaps that would help…

Eric Peterson added the following ...

Joseph,

If I’m translating correctly, your “nod” is my “click” and so as long as the click’s keep coming, the person is paying attention, and is thusly engaged. Correct?

I guess the problem I have with that, if I’m hearing you correctly, is that some people have an uncanny ability to nod at exactly the right point in a conversation without paying any attention at all. You could, for example, nod to your wife while thinking about, oh, playing the guitar. To your wife you might appear to be paying attention, but the little bubble above your head would show you picking-and-grinning … (as it were)

The purported complexity of my calculation accounts for that. Maybe the visitor is clicking but not paying attention, maybe they’re staying on the site, but again not paying attention. But are they interacting? Are they coming back to the site? Are they subscribing? Etc.

I think if they’re not engaged, the answer to those questions will be “no” and thusly their calculated metric would be lower.

Trust me, I know that people struggle with this calculation — some because it’s too complex, some because their application doesn’t make the calculation, and some for reasons I simply cannot determine. And while I may not think of human behavior in the same complex terms you do, I think that:

1) Simple metrics like “session duration”, “recency” and “bounce rate” ALONE are not robust enough to begin to approximate something as complex as “engagement” (or “Engagement” in your words.)

2) Given the direction our (my, I’ll let you decide if you want to be one of us or not LOL!) industry is heading, I’m absolutely convinced that if we’re afraid to model and make more complex calculations, well, eventually we’ll pay the price.

I am very close to gathering data that will help us see if our two definitions are the same or not. Your thoughts are helping me to shape that research so obviously I appreciate your insights, even if only you, Jim Novo, and I are paying any attention

Jim Novo: THANKS for contributing to the conversation! I’ll let JC respond to your question first but I love the analogy.

Jeff Chasin added the following ...

Other people ARE paying attention! Please continue!?!

Attention, Engagement, Authority, Influence are too complicated for simple, sound-bite solutions - people (like me) who are new to this field rarely get to listen in on conversations like this where the concepts and issues are explored and discussed so thoughtfully - so please continue!

Steve Jackson added the following ...

I’ve been using a model for some time called REAN (reach, engage, activate and nurture) which is a planning tactic designed to help build KPIs across what the WAA calls dimensions. The dimensions Reach, Engage Activate and Nurture are designed to model the Customer LifeCycle and allow us to build KPIs which measure the various stages. Jim, Eric, Theo and I discussed this in some depth;

http://blackbeak.conversionchronicles.com/2008/01/29/measuring-online-engagement-re-visited-and-introducing-the-rean-model/

I’ve already stated that I think Eric’s formula is a comprehensive KPI because it’s the first metric I’ve seen published which has an element in every stage of the REAN lifecycle.

I think what you are discussing with Attention, Engagement and Trust can actually be measured using KPIs and by applying the same planning process. When Jim talks about “dis-engagement” it’s a great example of how the REAN model can help.

REA the basic touchpoints and objectives of any campaign (Reach the audience, interact - engage with them and convert or activate them).

Nurture is about getting them to re-engage and re-activate. One way to do that is to as Jim suggests measure dis-engagement. Let me illustrate with a real example of a client campaign;

One lack of activity KPI metric we came up with for a cross channel campaign was “Inactive registrant index”.

The goal of the campaign is to get mobile phone users activated and sending MMS’s to each other. The resulting MMS lift was the goal of the campaign. The idea being that if users know how to send MMS messages they will start sending pics, and sending pics means an uplift in bandwidth usage which means an uplift in revenue for the mobile operator.

Basically the visitor dropping into the inactive registrant index is reached, (engages and activates) registers at either an offline event, a mobile website or the website but then does nothing after a certain time period. The action around this lack of activity was simply to contact the person with a good incentive to re-activate them. The incentive was to give the inactive user free coupons which the user could use to send MMS messages to his/her friends via the mobile website.

The point is that before we introduced the REAN model they had a lot of jumbled, unclear ideas about how to measure all of the various activities both on and offline.

A week later after we’d put the plan to them it became very clear and the CRM guy loved the KPI and has engaged us again for other projects around his customer database.

I think that by doing what Jim suggested measuring disengagement KPIs, as well as engagement KPIs, defined by the REAN lifecycle model or something similar and backing it all up with qualitative data you will get as close as you can to the truth about who your best prospects are and how to act on the data.

That after all is the point.

Regards the complexity of all of this. I personally don’t find it complex. I took one look at Erics formula and got it - I was like - Yep, Scoring formula, been there done that, never wrote it down though. It took me a while longer to analyze it, but once I had I was impressed with the KPI.

The reason this is so complex isn’t really the fact that geeks like me can get it, it’s the fact that clients typically have a lot of problems putting all of this together. It’s hard (as someone once said).

To measure and analyze on and offline behavior and then try to predict who to market to by figuring out what they think is not doable with one tool or one metric. Even Visual Site which I think is the most powerful web analytics tool won’t tell you what people think. CRM doesn’t either. Qualitative data will to a point but you have to then know how to combine all of the data you have.

Most clients just ask questions like “tell me what my visitors are doing”.

This is why REAN can help because it simplifies things in the eyes of the client. Have a look at the post on my blog and take a look at the REAN diagram I’ve recently added. It’s a bit small but hopefully you can get the idea.

Eric Peterson added the following ...

Jeff: Thanks for your feedback, I wasn’t sure if anyone else was paying attention LOL!

But let me ask you this: Since you’re new to the field, what expectation do you have of web analytics? Do you expect only simple metrics like bounce rate and conversion rate? Or are you looking for something more robust that has the potential to ground your work in real human experience?

I’d love to hear your thoughts.

Jeff Chasin added the following ...

Man, and here I thought I could get away with being a fly on the wall.

OK, well, my first expectation is finding an analyst job. After that, I would say that my expectations have been evolving. I remember the first time I looked at Google Analytics and thought (like a newbie) “look, we got hits”!

Obviously, more robust metrics/KPI’s that lead to customer insight and enable site owners to provide content that’s more relevant and more specific to visitor’s needs and wants would be far better than bounce rate and conversion rate. In more traditional areas of business analysis (manufacturing, process improvement, financial analysis, etc.) the big target now is predictive analytics. Tools like SAS, SPSS, MicroStrategy, Cognos, etc. allow analysts to make fairly accurate future predictions based on deep analysis of existing data. I expect (hope?) that web analytics tools or methods will evolve to become as useful for predicting future behavior online as say SAS is for predicting credit portfolio default rates. (possibly a very bad example given current market conditions:)

I have no idea how the “fuzziness” of web data effects the potential for analysis that can be conducted with it though? I always read and hear that “web data is different because it’s much less precise”, but then at the same time, I read that trend movements are “good enough” to be predictive. At this stage, I have no idea if that’s accurate. Any thoughts?

A Framework for Engagement - Background » Marketing Productivity Blog » Blog Archive added the following ...

[…] topic that started here and fragmented into a bunch of chunks and related topics, including here, here, and here.  I have also had a lot of one-on-one correspondence in and around these topics through […]

Jim Novo added the following ...

@Steve - that’s a fantastic example, exactly what I was talking about. You can’t say someone is Engaged just because they “do something”, there has to be a “continue” or they are formerly Engaged. Did you test an optimal reactivation trigger point? In other words, how long would you allow them to be in the process of dis-Engagement before you tried to reactivate, and was there a an optimal time frame in terms of response? Because typically, their likelihood to reactivate would decrease as time goes by…did you see any of that?

@Jeff - given your background, try this:

http://blog.jimnovo.com/2007/04/25/measuring-engagement/

I trust you will find some of it familiar ground…

Joseph James Geertz added the following ...

I don’t think Joseph is suggesting in anyway that a ‘click’ is a ‘nod’ is engagement. I think the issue for Joseph is that everybody wants to be able to show ‘engagement’ or show ‘trust’. However, wanting to show it and being able to show it are two very different beasts.

If we had a tool that could only count the number of legs on an animal and somebody wanted a device that could identify dogs, we couldn’t rationally say that our tool was a dog identifier by arguing that dogs have four legs so when we count four legs we have a dog. Nor would the statement, “It’s not precise, but it’s true” be rational. That statement seems equivalent to “We don’t know any better, so it’s true”, but lack of knowledge is not a seed for truth, it is a seed for research and investigation. You can use clicks and call it engagement, you can call site penetration or visit duration engagment, you can call repeat visits trust, but that doesn’t make any of it true. At best, you have a hypothesis that requires testing.

With respect to Steve’s statement, there is a tool more powerful than Visual Site that will tell you what people think and how they think. It was developed by Joseph Carrabis, one of the above authors. And my guess is you can sense a bit of his consternation by reading his contribution to this blog. He has put years and years into determining how people think online and others put days into the effort and propose to achieve the same results under a ‘this is as close as I know how to get’ standard.

There is a world of difference between developing a tool that measures engagement and alleging an amalgamation of simple web metrics can be used to compute engagement.

Theo Papadakis added the following ...

I hope im not too late in this discussion.

I have tried to elucidate the concept of ‘engagement’ in a guest post in Avinash’s blog. I think that it is compatible with your definition. It attempts to refine the concept by suggesting that there are two dimensions according to which a typology of engagement can be created, and that web analytics can only measure one of them:

• two kinds of engagement with an object, positive and negative, by which I meant that someone can be engaged with an object. Engagement is not itself a psychological state but involves a mixture of rational beliefs and psychological states, such as — in the case of positive engagement — sympathy, love, pleasure, pride, happiness, gratitude, empathy, affection etc.

• degree of engagement: The degree of engagement lies on a continuum that ranges from low engagement, namely, the psychological state of apathy, to high engagement. An engaged customer is a customer with an above average psychological investment/involvement with his or her object of relatedness.

What I argued in the aforementioned post is that kind of engagement, cannot be derived using web analytics alone. Only degree can. ‘Kind’ can however be inferred and hypotheses on what can be measured, degree of engagement, can be formulated.

If any of this sounds interesting I would really like to hear what you think.

If you want to have a look at the aforementioned post take a look here:
ww.kaushik.net/avinash/2008/01/measuring-online-engagement-what-role-does-web-analytics-play.html

If you are interested I would be more than happy to discuss what many of us feel are the problems with measuring degree of engagement.

Joseph Carrabis added the following ...

Boy, I go away for a little vacation and people start posting.

Bear with me and I’ll respond in the order things appeared, going back to Eric’s 29 Jan 08 comments. I responded earlier and, going back over the communications since then, perhaps I should clarify things a bit.

One of my favorite anecdotes is about the anthropologist and the microbiologist having lunch. The microbiologist checks her watch and exclaims, “My goodness! I have to go kill a culture.” The anthropologist has a heart attack.

This anecdote is about two different disciplines using the same word to mean two very different things and the confusion that ensues.

I share this anecdote because I’m not sure folks realize that NextStage has been monitoring and reporting on visitor’s levels of attention, engagement, trust, etc., etc., since we started in 2001. And yes, I do mean attention, engagement, etc., as written in Attention, Engagement and Trust: The Internet Trinity and Websites, Defining Attention on Websites & Blogs, in the 7 day blog arc starting with NextStage
Evolution’s Evolution Technology, Web Analytics, Behavioral Analytics
and Marketing Analytics Reports for the BizMediaScience Blog, 7 day
Cycle, Part 1: Are Visitors Getting Good Value? and in several dozen papers written since I started studying these things back in 1991 (1987 if you want to stretch things a bit).

So when I write “Engagement is the demonstration of Attention via psychomotor activity that serves to focus the individual’s attention” and “Can you repeatedly measure what you mean by them so that there’s a reasonable surety that what you’re measuring is what you mean by the terms you’ve used?” I recognize that the intersection of those two statements is “Can you repeatedly measure engagement as the demonstration of psychomotor activity that focuses the individual’s attention so that there’s a reasonable surety that what is measured is what is meant?” I answer yes, we’ve been doing it for years.

However, is what NextStage measures and reports on as “engagement” what Eric means by “engagement”? Probably not and the differences are what I wish to understand (I’m getting closer thanks to a conversation I had over the weekend with Stephane Hamel. Thanks, Stephane). I can look at the reports we generate from sites we’re monitoring and tell the owner “Visitors on this page were interested in this but this is what got their attention. They did/didn’t click on it for these reasons. This is what you need to do to make them click/stop them from clicking on it. They were completely engaged with this information on this page and completely disengaged with this other information on this other page. Here’s what you need to do to get them to act/respond. …”

(As an aside, another point of confusion for me is the increasing interest in “Time-On-Site” and “Time-On-Page”. These are metrics we’ve been strongly monitoring since 1999 when I started testing my theories on real eCommerce sites. Causing people to spend less time and more time have distinct benefits based on the site owner’s goals. This, also, is heavily documented in a variety of publications.)

Next up: Does some level of precision exist in the online world? I think it does and I’ll leave that for my next post.

Eric Peterson added the following ...

@JJG: There is no possible way to disagree with your comment “there is a world of difference between developing a tool that measures engagement and alleging an amalgamation of simple web metrics can be used to compute engagement.” Unfortunately most of us are forced to work with what we have and so my alleged amalgamation is what I’m left with (and trust me, I know I’m lucky to be running Visual Site because many people can’t even make this simple calculation!)

Your comment seems to speak to the need to “ground truth” engagement: Save asking every person who comes to a web site “are you engaged?” (which I would assert is A) impractical and B) just as imprecise as my calculation, if not more so!), how would you propose we ground truth engagement and test the hypothesis?

I ask not to be obtuse (which I may be) but rather to get you to help me think-out what is a pretty complex problem. I appreciate your comment (as does JC I’m sure) and look forward to your response.

@Theo: I absolutely agree with your “degree of engagement” idea and am glad to modify future posts on the subject to reflect that my calculation approximates degrees of engagement, not an absolute measure (not that I’m sure anybody thought I was talking about an absolute but one never knows!)

But I guess I look at your positive and negative engagement differently (and commented this at Steve’s blog I think.) I think about “Kind” more in the context of satisfaction — mostly because we already have good and established measures of satisfaction from vendors like ForeSee and OpinionLab and I kinda hate to make stuff up when it’s ** not ** necessary.

I think that you can have “positively” and “negatively” satisfied visitors, and both types can be engaged to different degrees. So you have a matrix with CS on one axis and engagement on the other (sounds like something Novo would draw, huh?) and you need to respond to each type of visitor state in a different way.

Thoughts?

Either way I have to say I’m really thankful to all of you who have engaged in this discussion! Far exceeding my expectations for the blog already and this only the first discussion — excellent! And Theo, I’ll be in the UK in March/April so perhaps we can debate our differences over a pint, huh?

Joseph James Geertz added the following ...

Eric,

I would agree the problem is complex. To prove the hypothesis, which is “conduct x is indicative of/shows engagement,” I think you need to be able to independently find a way to determine when someone is engaged.

Behavioralists can watch a person and identify when they are engaged. I’d bet Joseph could suggest someone. Short of having someone observe individuals, there may be physical data (e.g., heart rate) that could be tracked. Point is, find a way to reliably identify when someone is engaged independent of your web activity analysis.

Once you have a means for independently showing engagement, begin a study. Find a college campus and enlist 50 to 100 students (age, gender, and cultural diversity would be desireable). Give them a list of ten sites to visit over a set period (e.g., two hours) to view the sites, asking them to use the entire time period. Monitor them using your means for independently showing engagement, monitoring them against time as opposed to watching which site they are on. Then match the data and see if the metrics you believe indicate engagement properly correlate with the independent showing of engagement. If not, why not, and are there other web metrics that do?

There are probably other ways, better ways, to scientifically prove your hypothesis, but this is my first thought. I don’t think you are obtuse and I think if you put your mind to it and the time and energy, you could probably develop a better way to test your hypothesis. I worry though that you’d rather expend your energy selling your hypothesis as truth as opposed to proving it, which I do regard as lazy (a fault of which I am not immune). We are only forced to work with what we have because we have not worked to develop something better.

I also am not trying to pick on you or single you out. I regard you as educated, thoughtful, and well-intentioned. I regard your position as one shared by the majority. But I think there are a lot of web analytics and predictive analytics tools that are based on bad science practices and fail to achieve promised results, but sell because consumers want ’something’ and will buy into a promise. Over a hundred years ago we’d apply leeches to a person’s body to cure disease and we did so because we didn’t have anything more effective and we needed ’something’. Right now this industry is fraught with leech applications supported by bad science.

Eric Peterson added the following ...

@JJG: Excellent points, all, and I should say I hardly feel picked on or singled out! I’m happy that you’re willing to engage Joseph and I in this conversation (rather I assume you’re engaged, you tell me! LOL!)

The college campus idea is one that sounds good on paper but I suppose I’m looking for something in between my calculation and an artificial study like the one you propose. I guess I believe that with all the tools at our disposal there is/should be some intermediate step that would allow us to gather validating/invalidating data that would more directly tie to the calculation. (And yeah, I’m working on that …)

That said, yeah, I evangelize for the calculation in public but in my experience having the conversation and actually listening to people’s response is a big part of how we’ll prove (or disprove) this thing works. I certainly don’t take any credit for it, but we’ve seen some pretty interesting conversation at my blog, Avinash’s blog, Steve’s blog, Novo’s blog, etc. about what all this means recently and all of that knowledge goes back into the equation.

One question: What would you offer as a “best” example of web analytics that relies on good science practices? Measures of customer satisfaction and the work that Larry Freed and the nice folks at ForeSee Results have done with the ACSI springs to mind but I’m trying to think of a study that has correlated web analytics to real visitor behavior (Joseph, feel free to jump in here anytime …)

The use of leeches is a great example, but I suspect that in situations where the alternative was death that it made sense to try something. Not that anyone is going to die if they can’t measure engagement … LOL!

In the end, if I’m known for having prompted some really smart people to think about what engagement means in the online world and how that measure can be taken then I’m happy. I guess that’s what I think the moniker “thought leader” is about — not being right necessarily but having a strong point of view (and being willing to share it.)

I very much appreciate your insights and feedback!

Theo Papadakis added the following ...

JJG: I disagree with you because I think that we can safely hypothesise that a click is indicative of engagement.

According to Joseph C: “Person A nods in agreement every time person B makes a point. That nod is a non-conscious psychomotor activity that demonstrates person A’s attention is focused on what person B is saying, hence person A is engaged.”

Let me take this analogy further.

Person A types a keyphrase in a search engine. He reads one of the results. The result I see as person B (the creator of that page) making a point. The result read by person A, I see as a point person B is making, a point to which person B pays attention to and with which person B engages. I think that the act of reading involves not simply paying attention to what is being read but also a certain degree of consciousness.

Person A clicks on that particular result. I see that as a nod. In fact I see that click as a stronger indicator of engagement than a nod, because I believe that a click is a conscious act rather than simply non-conscious psychomotor activity. (because I think that the act of reading a particular search result before choosing to click on it must involve consciousness).

Person B does not click on that particular result. Even that I see as a nod, though from a web analytics this cannot be measured. Choosing to ignore something someone has heard/read I think also involves consciousness.

So up to now we have A saying something (typing in the keyphrase in a search engine) and B saying something back (the search result), which A reads and to which he responds (by choosing to stop paying attention to it i.e. ignore it or choosing to continue paying attention to it). Now let’s suppose that A clicks on the result he’s just read. The webpage that comes up I construe as B speaking again. And so on.

I would only agree that A’s behaviour does not show engagement if I was told that A is blindfolded and has no other way of interacting with his/her PC i.e. he is blindfolded when he types on the keyboard , and clicks blindly with his mouse on the screen. Of course I don’t know if some users actually do this. But I think I can safely hypothesise that most do not and hence, that their interaction with B, is conscious, and is an indicator of engagement.

Please tell me what you think.

Joseph Carrabis added the following ...

(Picking up where I left off or close to…)

I’m not sure and I believe your thought that the border between what is measured and what is between our ears can’t be crossed perhaps exists due to a false assumption that what’s between our ears can’t be measured.

Indeed, yes, it can. I can go into details and believe, you wouldn’t want me to.

Also there’s the reasonable surety thing. Unless a metric provides reasonable surety of repeatedly measuring cause and effect, I’m not sure I’d be comfortable calling what is measured a “metric”, at least not one I’d be comfortable having clients put money on.

Proxies for the truth: this is (I hope) a standard of understanding in any scientific measurement. Measurements don’t provide truth, they provide pointers to the truth. My favorite example of this is the deep cave experiments where they fill million gallon drums with chlorine and line the sides of the drums with photocells. Should (Should!) a stray neutrino collide with a chlorine atom a spark occurs. This is a very rare occurrence and one of the only ways we have of proving some incredible cosmological and quantum mechanical questions.

But when you read what cosmologists and quantum mechanicists claim from a highly random occurrence! And this, I ask, is suppose to be “truth”?

• Does a high bounce rate mean your page sucks? Depends on the completely measurable squishy stuff between the visitor’s ears and what’s on the page.
• Does having a high conversion rate mean you have a well designed path? See the above. In both cases, it is one data point among several and I always work to exhaust all the possibilities and all the data before making a statement or a suggestion.
• Does a reasonable surety assume a level that simply doesn’t exist in the online world? Depends what you’re measuring and how it’s being measured, me thinks. The purpose of this blog is to cause discussion, my desire is for a meeting of minds, a glorious accident, and I’m not going to say I have “the” answer, nor the “truth”.

However, I do believe that, knowing we’re working with proxies for the truth, we should either find the closest proxies or learn how to combine proxies from different disciplines so that the new generation or new class of tools does indeed bring us closer to that truth we know exists (shades of “Faith in things unseen…”).

Am I criticizing your engagement framework? Not at all. I recognize that it is from a paradigm with which I’m unfamiliar therefore I default to one of my favorite responses when someone asks me what I think of another researcher’s methodologies, “They do it differently than I was taught to do it.”

I also believe working to find similarities and overlaps in our paradigms is a manifestly worthwhile exercise (more work, yeeha!).

“Kind of like consulting a neuroscientist, shannonist, semioticist or semanticist when all you really need is a page view count.” Ohh. Ouch. I think. Walking and Running. But never in traffic. Especially on websites.

Joseph Carrabis added the following ...

(responding to Jim Novo)

Howdy,

I realize quickly that one of my hopes for taking part in this discussion is being realized. Your suggestion to read http://blackbeak.conversionchronicles.com/2008/01/29/measuring-online-engagement-re-visited-and-introducing-the-rean-model/#comment-6761 was quite enlightening. From that post and elements in this thread (still going through it, folks) I form the belief that what I define as “engagement” is what must occur neuronically before what is being described as “engagement” in these posts can occur. Fascinating.

I must ponder this…semantic differences can be mathematically bridged (something our Language Engines are designed to do, so I’m a little familiar with how to do it).
Brad Berens gets a kick out of asking me questions because it’s fun to watch me go into a fugue state developing solutions.

Hmm…yes, I can understand how the bridge should be constructed. The model isn’t complex, merely rich in hierarchies…

[allow me to continue thinking on this while I respond to something else you wrote…]

I did not detect any resistance to the idea that if we are going to measure Engagement, that implies a relationship, which implies “value over time”, which drives towards the idea that “likelihood to continue” is what differentiates measuring Engagement from measuring Activity.

So Activity defines Value:

You can have Best Customers or Endorsers or whatever you want to call them for the environment you are in - your highest value participants. At the other end of Activity / Value, you can have Low Value Customers or Detractors or whatever you want to call them. Their Activity defines their Value.

I accept (for the present) what I believe falls from the above: “Engagement” is a determination of an individual’s likelihood to continue exchanging value with another individual over time.

Just so we’re all clear on how I actually frame this; “Engagement” is a determination of entity A’s willingness to continue exchanging value with entity B over time. The reason for the reframe from “individual” to “entity” is to allow for non-biologics (such as websites) to be on the sides of the equation. Allowing for this change in construct gives me the ability to access some neuromathematics dealing with how humans conceptualize identity structures with which I’m already familiar.

The reason for the reframe from “likelihood” to “willingness” allows me to include a metric NextStage has used since its early days; Loyalty (please remember that what I mean is probably not what web analysts mean by the same word. See Usability Studies 101: Brand Loyalty). In many ways I sense that your use of “engagement” has elements of trans-temporal reafference in it (Branding in Online Video, Branding and Online Ad Placement). Trans-temporal reafference! My god, that would be a beautiful construct!

Oh, James, I love you! You’re making me think! (not a pretty thing. Ask my wife. Much worse than my dancing)

Somebody tell Jim Sterne we have another eMetrics presentation, “How to get visitors Engaged - Moving Loyalty from Their Minds to Your Wallet!” Or maybe Lars Johansson. He has a conference coming up, too, yes?

(Okay. I must calm down. Slow breaths. That would be a kicker presentation, though. A kind of “Here’s what has to happen in a visitor’s brain in order for them to produce value over time. Here’s what you have to do to your website in order for those things to happen in the visitor’s brain”)

I must adieu for a while and allow these thoughts to gel.

Joseph James Geertz added the following ...

Theo,

I am not saying a click cannot be indicative of engagement, but a click does not equal engagement.

As an example, there are times when I am driving home from work that I am engaged with the road, with the act of driving. There are other times that I am engaged with my thoughts on the day, with the radio, or with a passenger and disengaged by the road, the driving. When I become disengaged from the road, I do not go careening into other vehicles or flying off the road, but I continue driving in a semi-conscious/non-conscious state.

This is particularly true on my commute, where most of the view I know like the back of my hand. If my wife calls and asks where I am, I often need to re-engage the road and look for a sign (literal or figurative) to identify my location.

Same is true with some web pages. Some web pages I know like the back of my hand and, if there is a specific page I want to see, I may click-through/navigate through three different pages to get to that page. None of those three pages engaged me, despite my clicking. I knew where my links were and moved forward. Those initial pages could be written in Spanish and not only would it not slow me down, I probably wouldn’t notice. Similarly, once I get to that page, there may be nothing to click, so I read and move on to another site. In that experience, from the moment I am engaged with the site until I stop, I don’t click at all.

I went to the Kinko’s site close to Christmas to order playing cards with a picture I could put on the cards. I clicked through that site forever trying to find the page I was looking for. If clicks measure engagement, Kinko’s might think they have a real page turner. The truth is, I was lost and confused and would have only clicked a couple of times if the site were more easily navigated. Is that a message to web designers? Hide the good stuff and unsuspecting travelers will become deeply engaged by your site. Engagement is inversely proprotional to navigatability (warning: lexicography in action).

Again, I am not saying clicking the mouse is signficant of nothing, I am just saying engagement is a complex problem and the suggestion clicking=engagement is an errant oversimplified solution.

Joseph James Geertz added the following ...

Joseph,

The love is (awkwardly) appreciated. A pleasure to fugue you (whoops, more awkward).

Joseph Carrabis added the following ...

(still responding to Jim Novo’s 31 Jan 08 10:59am entry…I think that’s the one, anyway)

I love your comment “…the need to fit this all into published frameworks because it leads to continuity between disciplines.” Yes. Perhaps I’m seeking the GUT (Grand Unified Theory) of Analytics. Wouldn’t it be grand (no pun intended) though, to find some underlying principles that guide, shape and form these many disciplines?

Or perhaps I’m attempting to create a Double Benioff Zone to understand the web…

Someday at some conference we’ll all meet in the bar and, if you’d like, I’ll bore you to death with an explanation of trans-temporal reafference, how and why it’s probably relevant to any rich content (Flash, audio, video, …). I was going to explain things at a high level here and the high level explanation got to be several pages long, so to heck with that.

I’m pretty sure that trans-temporal reafference is meaningful in a discussion of web pages that are intended to keep a visitor engaged (as I define it) for some period of time longer than a “typical” web page is intended to keep a visitor engaged. Meaning if a web page delivers some content such that the visitor doesn’t need to refresh or move to some other page for a recognizably longer period of time than they spend on most web pages (let’s say they watch a video download that takes 30m of time and a typical page takes 3 seconds to scan and act upon, thus y >> x | x Ä 0 for calculation purposes).

Anyway…

I’ll accept your statement “You have various levels of Engagement, a continuum from Highly Engaged to Formerly Engaged,…” This makes great sense to me.

“…which is a prediction of Value in the Future.” Hmm…I suppose, as I’ve stated my understanding thus far. I would need to run some tests to confirm my suspicions.

“So you can have Endorsers who are Formerly Engaged, you can have Low Value customers who are Highly Engaged - and buckets in between. To optimize the system, you would use a different Marketing strategy for each segment, which is the whole idea behind Relationship Marketing. You could then step up to REAN to implement Relationship Marketing based on the Customer LifeCycle.” You are defining tic marks on a scaling system, yes? Accepted. Different marketing strategies for different segments? Definitely yes.

REAN? Haven’t gotten that far in my readings yet. Sorry. Guess I’ll close for now and add more once I’ve studied a bit more. - Joseph

Jeff Chasin added the following ...

Have any of you seen this?

“The Great Engagement Debate ‘08″

The promo site is here: http://videoegg.com/events

The pdf with the deets is here: http://snipurl.com/engagepdf

If you’re in NYC on the 20th, thought you might be interested?

Debbie Pascoe added the following ...

Responding to JJG comments:
“…I think there are a lot of web analytics and predictive analytics tools that are based on bad science practices and fail to achieve promised results, but sell because consumers want ’something’ and will buy into a promise. Over a hundred years ago we’d apply leeches to a person’s body to cure disease and we did so because we didn’t have anything more effective and we needed ’something’. Right now this industry is fraught with leech applications supported by bad science.”

I love this as it gets right to the heart of the issue. From day one of the web, people have been obsessed with “who’s looking” - from counters, to multiple counters, to path analysis, A/B testing, personalization, etc., the focus has been on trying to gain insight into the visitor and how to more effectively manipulate them to take the desired actions - to be engaged.

And until now, people have not stopped to ask themselves if the data is any good to begin with. The good news is that they are starting to. Web analytics implementations are not simple and foolproof. Companies that have made the move to page tagging tend to be at the outer edge of the curve with large, volatile, complex websites, lots of user-contributors, multiple platforms, sometimes dealing in multiple global geographies.

The subject of engagement via the website is underpinned by the notion that the data is accurate, complete and reliable, and that no site quality and compliance issues are infecting the data. This is not the case. Implementations are fraught with issues. Without verifying the accuracy of the measurements tools, and the quality of the sites they are measuring, people can’t rely on the data with any degree of confidence. Implementing automated validation of the measurement tools will move organizations to a more scientific approach and away from the leeches.

I have written on this subject several times and invite your review and comments.

Web Analytics Shootout Highlights the Importance of a Quality Controlled Environment - the most important thing Eric Enge discussed didn’t have to do with the findings, it had to do with the method.
http://weblog.maxamine.com/2007/10/08/web-analytics-shootout-highlights-the-importance-of-a-quality-controlled-environment/

Myth No. 1 - There Aren’t Enough Web Analysts - five “myth-understandings” are clouding the ability of organizations to deal with data quality issues.
http://weblog.maxamine.com/2007/10/19/web-analytics-myth-number-1-there-arent-enough-web-analysts/

Web Analytics 2.0 - Defining the New Discipline - a further discussion of Avinash’s “Multiplicity” concept
http://weblog.maxamine.com/2007/11/08/web-analytics-20-defining-the-new-discipline/

Top Four Characteristics of the Optimal Web Team - today, web management is fragmented throughout organizations; this has to change for organizations to fully capitalize on the value of their web investment.
http://weblog.maxamine.com/2008/02/13/top-four-characteristics-of-the-optimal-web-team/

Joseph Carrabis added the following ...

(still responding to Jim Novo’s 31 Jan 08 10:59am entry…I went off and studied up on things that appeared relevant to the REAN model he was referencing)

I did not know (my apologies, all) that web analytics made use of geometric models (something REAN seems to be). I won’t discuss how the R, E, A and N dimension metrics are collected. Each dimension seems to be some kind of reduction of several other dimensions, though. I’d love to know if there’s some documented formulae for these. My concern is that the problem will be the same western medicine faces when working to understand jun-chen-zuo-shi.

Don’t know if I asked this here or not before…are folks measuring what matters or are they measuring what they can measure and saying it matters? Maybe I need to change the focus of my questions from “what is it called?” to “what is being measured?” This was briefly addressed in the exchange on proxies for the truth. It’s kind of like claiming something is a best practice because it’s been around for a while. Best practice in 1400AD was to sail close to the shore because the edge of the world lay in the other direction.

I’ve been trained to learn an application, distill its principles to extract the theory that makes the application work, then use the theory to develop multiple applications.

That offered, let me restate the question from “What can we measure?” to “What do we want to achieve?”

For example, not “We want to measure engagement” to “We want people to come back to our site.”

Yes, this is a move back to First Principles. One of my mentors, a brilliant mathematician, taught me to always make my mistakes at the beginning of a solution. Errors at the end were usually too difficult to find because you were caught up in the logic inertia. At the beginning of a solution you hadn’t really developed enough of an idea of what the solution looks like to be caught up in the need to see it to its end.

So, with your indulgence and a desire to better understand the task at hand, What do we want to achieve?

(I hope people don’t think I’m out where the buses don’t go when they read my comments. I assure folks, my interests are the same as others — creating reliable metrics. I also hope that my coming from such a different paradigm and with such different trainings will be recognized as opportunities rather than something to be quickly dismissed. One thing I really like about the REAN model is its recognition (as I read it, anyway) that different elements are influencing different things being measured and creating a “collective” measurement. My experience is that measuring isolated processes causes the measurers to ignore the diversity of elements required to understand the situation being observed)

more to follow…

Joseph Carrabis added the following ...

(still responding to Jim Novo’s 31 Jan 08 10:59am entry…now on to “…the concept of Trust is a bit problematic, …”)

I disagree with the statement that “…the concept of Trust is a bit problematic…”, both as something measurable online and as a concept. We demonstrated years ago that a measurement of the non-conscious message “You can trust us to help you” was directly connected to success in the market place.

I think (and am not sure) that what you’re describing is “trust in the marketspace”, something recognized in neuroeconomics and economics in general (R. S. Burt, Bandwidth and Echo: Trust, Information, and Gossip in Social Networks. in Networks and Markets: Contributions from Economics and Sociology, edited by Alessandra Casella and James E. Rauch. New York: Russell Sage Foundation, (2001) 30, Social Decision-Making: Insights from Game Theory and Neuroscience. Although not directly related, an equally good piece is Watts and Dodds “Influentials, Networks, and Public Opinion Formation” in the Journal of Consumer Research Dec 2007 v34
also Watts’ “The Collective Dynamics of Belief”) and closely tied to what’s called “fair-exchange“.

I think (and am not sure) that another facet of what you’re describing is something that occurs in language and not in the mind. The mind — without extensive training — can’t conceive of negatives, therefore what (most) humans do is think of a positive and then negate it. In other words, it’s not that someone “doesn’t trust” another person or organization, it’s that they trust them to do something unpleasurable. The degree to which person A “distrusts” entity B is really a measure of how much person A trusts entity B to do hurt or harm to person A.

Taken from this perspective, it’s not that you trust or don’t trust some company, it’s your confidence in your ability to manage the pain or pleasure you believe you’ll receive as part of your exchange with them.

<Ramble>
(no doubt you’re thinking “you’re signaling a ramble? You mean, the rest of these aren’t rambles?”)
“manage pleasure” is almost an oxymoron, don’t you think? Reminds me of Commander Cody and the Lost Planet Airmen’s “‘Too much fun’? That’s news to me. ‘Too much fun’? There must be a whole lot of things that I’ve never done. I’ve never had too much fun.”
</Ramble>

This may seem like semantics to you and I believe it isn’t. The brain-mind is an amazing place. It’s well worth exploring and I hope that is my value-add in this discussion.

(more to follow…picking up with “…what happens after Trust, what happens when ‘Engagement’ ends?”)

Joseph Carrabis added the following ...

there’s suppose to be an image in the above, no idea what happened to it. In the paragraph dealing with negation not occurring in the mind, the image should be http://www.nextstagevolution.com/images/p2ptrustslider.jpg
- Joseph

Jim Novo added the following ...

“it’s not that you trust or don’t trust some company, it’s your confidence in your ability to manage the pain or pleasure you believe you’ll receive as part of your exchange with them”

Yes, I’m sill following your “rambles” and I am with you on the idea above. In fact, isn’t this continuum related to the previously discussed continuum of engagement / process of dis-engagement? They must interact in some way, e.g.

An “event” occurs, let’s say “bad” which in terms of data could be a call center resolution code, a negative comment, a returned purchase.

Post-event, on the Trust continuum, you slide toward lowered confidence in ability to handle the pain.

You dis-engage to some level, to compensate for this lowered confidence - what you are trying to do is reach an “equalibrium” state of some kind with respect to the company or relationship. If you are less confident in being able to handle the pain, you want the likelihood of the pain occurring to decrease.

By dis-engaging, your Potential Value to the company is lower than it previously was, the likelihood you will re-engage is lowered, and the company has to “do something” to restore this Potential Value or dis-engagement will proceed and you will be lost as a customer.

In a way, this describes the “lost friend” process I asked you to comment on earlier; as the confidence in being able to handle the pain decreases (Trust), the avoidance of that pain (dis-engagement) increases, to the point where neither friend sees and benefit in reaching out - Potential Value of Relationship dies.

Joseph, even if you and I are the only ones on this thread I think this is worth beating into the ground until we get some context within established thought.

Web Analytics Demystified » Blog Archive » What is the future of web analytics? added the following ...

[…] get there. In the last two months we’ve had excellent conversations started by the likes of Joseph Carrabis (NextStageEvolution), Rene Dechamps Otamendi (OX2), and most recently Mr. Ian Thomas of Microsoft […]

Joseph Carrabis added the following ...

(still responding to Jim Novo’s 31 Jan 08 10:59am entry…now on to “…what happens after Trust, what happens when ‘Engagement’ ends?”)

<Ramble>
Two things from readings read to better understand these comments:
1) Sometimes I wonder if the semantic similarities are Batesian or Mullerian in nature.
2) What if a real use of the REAN model demonstrates that what we want to measure can’t be measured the way we want to measure it? There’s a need to determine ahead of time how much credence each lens of the REAN model gets and by its definition (my reading of it, anyway) each lens requires equal weight.

A comment from someone following these posts:
I’ve been a little busy (so haven’t been paying lots of attention) and it seems only Mr. Novo is still following this thread. A reader of this thread talked with me on the phone, asking if this discussion had its run as it seemed (to the reader and I agree) that nothing has been resolved, not even a “we agree to disagree”. I’m concerned that this discussion and the sprouting of others is doing more to create barriers between the specialities represented than offer ways to bring them together.

One thought:
Can evidence inform the debate? A friend told me that I have to start showing graphs and charts of the results I talk about when I do presentations so that people will believe me. This was an interesting request, I thought. Graphs and charts aren’t “believable” to me, they merely represent the outcomes of experimental systems I accept as valid. This is one of the challenges I have with web analytics making claims to engagement, etc. Great charts and nice formulae and they don’t come from anything that measures what I recognize as engagement, etc.

As noted above, I get busy and may not be devoting the time to this blog that some would prefer. My apologies. But my busyness often produces some fascinating results. The other discussions in this blog have merit, I’m sure. But if the function is to perfect the visitor experience (that is the goal isn’t it? I mean, all these analytics are worthless unless they create happy, satisfied visitors, yes? Just my thought, anyway) then there has to be a way to blend something such as the following:
I’ve been going through journals looking for information that backs up some research NextStage did that deals with page load times. This became of interest when I was a WAA member as there seemed to be some discussions around page load times being a possible concern. NextStage has always measured page load times (and remember, what NextStage calls “page load time” may not be what others define as “page load time”). Anyway, one of the things we learned was that (given some standard page load time T), some visitors prefer slower page loads (t > T), some faster page loads (t is where NextStage comes in)) and I think something worth focusing on as part of the discussion, should any be interested (and willing to put up with my slow response rates. Guess that means you’d appreciate slower load times if you correspond with me…)

Let me provide an example; I’m currently developing a tool for a client that will allow the user to determine if a blogger believes themselves or not. We’re already measuring if blog readers believe the blogger or not (again, this goes back to a question posed to me by SNCR Sr. Research Fellow John Cass). In both cases, though, the actual discussion — the content of the blog — is pretty irrelevant. What matters much more is the blog reader’s perception of the author’s reliability and credibility and the author’s self-perception of their own reliability and credibility. This isn’t something that I believe can be measured by clicks and such alone. That’s my guess and I’m thrilled to be corrected.

Okay, enough rambling. Now to get back to responding to Jim Novo…
</Ramble>

I knew there was a reason it took me so long to get back to this…the question of what happens when “engagement” ends. This was actually something that fell out of our email newsletter research and will be published this Friday, 29 Feb 08 (I think), in IMediaConnection; how people demonstrate an “irrelevancy” response to a newsletter (or much of anything else in the online world, it seems).

The scenario you describe re a friend and losing contact; this is something we call “Determining if the door is open or closed” and has a strong history of research and applications in social work. It is strongly tied to culture, belief, education, training, …, and pretty much can be summed up with “How does someone define friendship?”

(Interesting side note: most people fail at sales because they fail to recognize the difference between “relationship” and “friendship”, ie, that “friendship” is a one type of relationship. Their boundaries are unclear hence they have challenges in sales relationships. I just finished working on a project with a LeadGen/DemandGen firm and one of the ways we boosted productivity was to train individuals about creating boundaries between types of relationships.)

My belief is that the concept of “Is the door open or closed?” exists in sales based on my experience (note that I’m not a salesperson, never have been, don’t wish to be. My experience is observational only).

You ask “What are the stages/attributes of a dissolving friendship from the literature?”

Excellent question! Thanks for asking! The answer is wrapped up in that culture, belief, education, training, etc., stuff I mentioned above. I’ll use myself as an example; one of NextStage’s early execs often commented to me that he couldn’t understand why so many people agreed to work with and for me based solely on a handshake. I was remarkably ignorant of business at the time (still am, pretty much) and replied (very honestly and ignorantly) that perhaps because I spent my childhood milking cows I had a good grip, hence a good handshake.
What it really comes down to is concepts of “friendship” and “trust”. I know how to demonstrate “friendship” and “trust” to people and people echo that back. One way of doing so is by agreeing to work with and for me based on a “handshake”.

An important element of this needs to be mentioned; People who trust and are friendly with themselves will demonstrate trust and friendship to others. We have another term for this when we do trainings, presentations and talk with clients, “If I am a thief then you must steal”. People who don’t demonstrate trust and friendship are people who will not work in your or their own best interest. A good friend and early adopter/trainer of NextStage’s technology is a native Australian. He spent several years here in the US then moved back to Australia. I was one of the first people he told about his decision to move back. His reason is one I’ve heard from several people from other countries who deal with US businesses. He said, “In the US, the whole purpose of a contract is to decide how and when the mutual f???ing will begin.” A painful statement, yes, and one definitely worth studying.

Note that none of the above deals with “trust” per se because the “trust” doesn’t change value, it is what is “trusted” that does. A lost friendship isn’t a diminution of trust so much as a trust that the relationship has changed.

Let me ask a question re your “Pretty soon, you’re not friends anymore.” What does one do if the friend unmet for several years is seen in the mall? Or knocks on your door? I am curious because the ability to “stay in touch” is very recent historically, only since the advent of high-speed personal communications. Forgive the vanity of quoting from Reading Virtual Minds:
Business often demands that we “stay in touch” or “be in touch” 365×24x7. At the same time, they implicitly demand that we stay in touch or be in touch by not touching or interacting with a human at all. How often are you invited to go to a company’s website while you’re waiting for a human to respond to your support call? How many of you have direct deposit and do most if not all of your banking on line? How many of you would be upset if ATMs vanished and you — Gasp! — had to wait for a human teller to handle your transactions? How many of you spend your day listening to your personal music player regardless of whether you’re in an office surrounded by co-workers or one of many people walking a crowded street ? And how many of you would be upset if someone told you you couldn’t listen, that it was distracting you from something they deemed more important, like work, or that truck rushing down upon you which you can’t hear because your earphones cancel out all other noise?
The truth is we as a society in this “modern” world are allowed the solace of others less and less even though the increased pressure this same society and modern world places upon us demands it more and more.
And be aware of how language was used in that last paragraph to make you feel something you might not have otherwise; modern is in quotes, separating it from the rest of the sentence, allowed, solace, increased pressure and places upon are all metaphors of physical contact and distance. The message hidden yet strongly suggested in the above? “Oh, these devices, we use them at our peril! Be Aware! Watch the Skies!” and all that.

“Staying in touch” was unthought of in 1900 except for the very rich. Go back to 1800 and “staying in touch” meant (maybe) receiving a hand-written letter when the ships docked, maybe once or twice a year. Prior to that the norm was that people who left the village were gone forever — ahh, but not forgotten and still friends.

I remember my paternal grandfather’s return to the village of his birth in Italy after an absence of some 60 years. People were hugging and kissing him, opening their homes to him, brining out their best wines. He was still their friend. Just as he was still “trusted” to do harm to those who had harmed him before he emigrated to the US 60 years earlier (Grandpa often quoted Archliochus).

(Anybody wonder what, in my mind, qualifies as a <Ramble></Ramble> and what doesn’t?)

I’m also not sure that the concepts of “trust” and “friendship” can be used in this discussion because an unequal friendship can also involve concepts of “betrayal”. Betrayal happens in business, yes, and I’ll ask that we limit this to B2B and B2C situations. Businesses may want consumers to think the business is a “friend” and that strategy has (I think) shifted to concepts of branding and identity-branding (where individuals are concerned) because consumers are experiencing new voices thanks to the internet and new voices equate to new power.

Consumers may feel “betrayed” by a business, that their “friendship” has been devalued in the relationship and that they can “trust” the business to cause them pain or harm. But consider the response — Comcast and other businesses radically change their business methods and practices because consumers have voices they never had before. A business would gladly suffer a class action suit rather than a blog visited and commented on by several tens of thousands of visitors a day or week. The former can be negotiated. The latter can destroy.

Applying this to web measurement. Yes, it is possible to directly measure if people find marketing material trustworthy or not, hence worthy of “friendship” in the sense that we as a species tend to accept a higher degree of familiarity with those we trust to cause us pleasure than those we trust to cause us pain. Similarly, it is possible to directly measure that such people are engaged, even how long after their web (in this case) session ends they will remain “engaged” (using the definition I’m most familiar with). Note that I also recognize that engagement crosses several channels, not just the web. People act as a response to (usually) several touches. I think (and am not sure) that web analytics is aware of this.

<Ramble>
Yes, I may go off and fall silent for a bit. Isn’t it fun when I come back, though?
</Ramble>

And my next entry will respond to Eric’s “If I’m translating you correct, …” comment.

Thanks for your patience. - Joseph

Joseph Carrabis added the following ...

Now onto Eric’s “If I’m translating you correct, …” comment.

Well…I’m not sure I’d agree that my nod is your click. Actually I’m quite sure I wouldn’t agree. Consider the following example.

You’re at an eMetrics Summit. You’re talking with some folks mid-afternoon about what you’ll all do for dinner after the last session. Everybody is nodding, agreeing with the plans. But things change, other people come and go, and when you finally count heads at the restaurant you discover some of the nodders aren’t there and people who were never involved in the discussion are.

Your click is counting the people who show up at the restaurant. My nod is everybody talking, some showing up and others not. I chose to explore those nods because the degree, angle, direction, inflection, …, of the nods tells me long before people show up at the restaurant who will and won’t show up. This is — I think — one of the fundamental differences between what we measure and analyze and what I understand of web analytics. One of the fallouts from understanding the nods (if you will) is that you can determine if someone is paying attention as I define it and thus engaged as I define it.

Going back to the conversation about dinner. Consider the person who’s nodding while looking around the hotel lobby. Do you have their attention? Some and not all, and exactly how much depends on the individual’s cognitive, behavioral/effective and motivational (C, B/e, M) matrix (Headlines That Attract Attention, Adding sound to your brand website (I touched on this one at the DC eMetrics ‘07 Summit), Intelligent Website Design: Expand Your Market (Page 2 of 5), AllBusiness.com’s Chris Bjorklund interviews viral marketing expert Joseph Carrabis, founder of NextStage Evolution, Part 5, KBar’s Findings: Political Correctness in the Guise of a Sandwich, Finale, Notes from UML’s Strategic Management Class - Saroeung, 3 Seconds Applies to Video, too, Technology and Buying Patterns) and any environmentally available information that more accurately targets and activates their {C, B/e, M} matrix. More to the point, are they engaged by you or with you?

Now that’s an interesting question. Only to the point that whatever else their devoting their {C, B/e, M} resources to isn’t activated by environmentally available information. This is a polite way of saying “No, they’re not”.

Next up, Eric’s “I guess the problem I have with that…”. Now I must prepare dinner…

Joseph Carrabis added the following ...

(interesting note: what I wrote above is a very obvious example of my {C, B/e, M} matrix) - Joseph

Joseph James Geertz added the following ...

Random thought on attention/engagement/trust

LinkedIn has apparently changed the look and feel of their home page. I used to be able to go to the home page, move to links that were in known locations - locations with which I had become comfortable - and retrieve the information I wanted efficiently. This morning, I was stumbling about on the site searching for the information I sought, clicking more links than I would normally, spending more time on the site than I would normally, but becoming increasingly discouraged that I could not find everything I wanted.

All of us have had similar experiences when web sites we frequent change the look and feel of their home page, so I don’t feel the need to babble on too much further. Here’s my thought:

A proper web analysis of my interaction this morning would reveal (and feel free to disagree) a high degree of attention devoted to the site, a high degree of engagement, and a feeling of distrust. The distrust would stem from: 1)changing faces toward me (two-faced would be an appropriate colloquialism); 2) the fact that I frequented the site in exchange for certain information - the underpinning of an informal social contract - and the site is no longer making good on the promised exchange; and 3) the site has signaled to me that I cannot trust it to provide desired information going forward in return for my patronage.

I don’t regard the high degree of attention and engagement as a positive for LinkedIn (again, feel free to disagree) because it directly correlates to a negative experience. Which current web analytics tools would let analytics consumers (aka web site designers/providers) both that statistically attention and engagement are high and those high marks correlate with a bad experience?

Jim Novo added the following ...

Joseph C,

I get the feeling we need to have this conversation in person - the whole asynchronous thing just creates too many possible answers to any reply! It’s like have 15 different conversations at the same time…are you going to be at eMetrics Toronto?

Perhaps we should start a different thread on this, but I have always wondered about the implication of having 3000 friends on Facebook or MySpace - doesn’t that mean the value of each relationship is so small as to be close to zero? And doesn’t that mean these social apps never were about “connections”, they are about ego?

Me Broadcasting Inc.?

Joseph G,

This is why I don’t think you can talk about Engagement without addressing dis-Engagement. I agree the “onsite engagement” piece from web analytics can look remarkably similar when actual experience is either positive or negative. The same holds true for “Duration”, for example, where long Duration can either be positive or negative.

But, if you fail to come back to the site as often or at all after the onsite engagement, this dis-engagement from the site clarifies the previous visit Engagement as negative, or the relationship “complete”.

As I have said many times, you could go to a site, pull all the levers and push all the buttons, and appear very Engaged. At the end of that visit, you could decide the experience sucked and never come back again.

The folks that measure Engagement without a “time since” component are in for a very rude awakening some day when they wake up and “all of a sudden” have very Engaged visitors but a lot less of them. On this day it will be too late to do anything about it, as opposed to tracking “time since”, becoming aware of the dis-Engagement problem, and addressing it proactively.

Joseph James Geertz added the following ...

Jim,

“Time since” can be a useful component, but it comes with its own problems. First, this is the Internet. If a metric requires 30-60 days to be meaningful, it isn’t useful. Second, I’m not sure a metric is required that lets people know their traffic has diminished because of a negative experience. Once traffic diminishes, the experience of visitors is intuitively recognized as negative. At that point, all you can do is explain to a client how the good engagement metric relates to a loss of business. Third, some sites will not get return visits regardless of experience. I bought a Nordic Track treadmill from their site. I visited once, found the product I wanted, made my purchase and haven’t needed them since. Engaged, positive experience, and no positive ‘time since’ value.

If anyone hasn’t caught on, I’m a fan of a Carrabis’s. I’ve been a fan since I first saw him present on his technology in 2002. I get the newsletter, I visit the blogs, I got the t-shirt and drink the kool-aid. I like the technology because it delves more deeply into individual interaction between users and web sites. I use the mouse/keyboard/input device differently when I am engaged. I use the input device differently when I am positively engaged than when I am negatively engaged. I use the input device differently than someone twice my age or half my age.

Think about a brick and mortar car dealership. You can look at the number of visitors you get each week against the number of sales made to get an understanding of how effective the sales force is (conversions). You can look measure how long each visitor stays at the dealership (duration). You can look at the number of test drives (site penetration). You can look at how many customers come back to buy another car (time since). But that information only tells you so much. Maybe the data comes back good in spite of the sales force. Maybe the data comes back bad in spite of outstanding work by the sales team (maybe trying to open up a used Chevy Nova dealership was a bad business venture).

You don’t really know if the consumer experience with the sales force is positive without observing. Are the customers laughing with the salesmen (engagement)? Are the customers offering to shake the hand of the salespersons when they leave (trust)? Are the customers yelling (betrayal)? Are they nodding as they listen to the salesperson (acceptance)? These one-word parenthetical summaries lack adequate depth, but the point is that you learn more about a consumer’s experience by observing their interaction as opposed to the effects of that interaction.

More specifically, how long and how often am I pausing between keystrokes (standard deviation is probably more important than the mean). When I am on one of your web pages and I am moving the mouse without trying to change pages, what does the mouse movement mean? Am I scrolling up and down the same page repeatedly (confusion)? Am I slowly moving the mouse back and forth horizontally (reading, engagement)? Am I zipping over the page (scanning)? I am throwing darts at the wall a bit on this and probably borrowing from some comments I have gleaned from Carrabis, but the point is that if you really want to appreciate a consumer’s (or consumers’) interaction with the site, you have to know how to observe it in a meaningful way. I don’t think any combination of duration, penetration, or ‘time since’ is enough to get you there. I don’t know that Carrabis’s solution is THE solution, but I think analyzing and appreciating input device behavior is a prerequisite for THE solution.

Joseph Carrabis added the following ...

Howdy,

First, I didn’t forget the comments here. I got a little involved in some projects, and probably some of you know by know that I’m working with Eric on The Engagement Project (see Measuring Engagement Online: The Next Stage, Measuring Online Engagement: Step One and Continuing the discussion: Attention, Engagement, Authority, Influence for more on this).

The reason I stopped commenting here for a bit is because I stopped my comment stream at Eric’s “The purported complexity of my calculation…” and, as I had never looked at Eric’s calculation and he was entering it into the discussion, I decided I needed to study his calculation in order to understand the statements being made.

Anyway, the end result of that can be followed in different threads, some here, some in Eric’s blogs, some in other TheFutureOf posts and also on BizmediaScience (see Eric Peterson’s Engagement Project and the Engagement Equation, Part 1 and Eric Peterson’s Engagement Project and the Engagement Equation, Part 1 - Responding to WindKiller’s Comments for starters).

So, picking up with Eric’s “The purported complexity of my calculation accounts for that. Maybe the visitor is clicking but not paying attention, maybe they’re staying on the site, but again not paying attention. But are they interacting? Are they coming back to the site? Are they subscribing? Etc.”

One of the things that came out of working with Eric on his calculation was a quantitative recognition that our definitions of “engagement” were different because we were framing our definitions on different interfaces. Eric’s frame of reference is the browser and within that frame his definition works well. My definition’s frame of reference is the brain-mind and within that frame of reference NextStage’s definition works well. NextStage’s analytics are using scripting and related tools to measure what’s happening in a visitor’s psyche, Eric is using scripting and related tools to measure what’s happening at the browser.

Another thing that falls from this hearkens back to my original thoughts in this thread about A Meeting of Minds. The work we’ve done on understanding these different frames has produced a method for merging and mixing Eric’s definition, NextStage’s definition, anybody’s definition in ways that produce actionable results.

Yeah for our team, huh?

Another yeah for our team is that Eric’s calculation is no longer complex and is highly simplified. One more yeah for our team is that we can do things like use one reference frame to determine value in others (borrowing from physics a bit). NextStage has (at this point) about 15 years of online data and some 70,000 data points that it uses when making calculations about such things as attention, engagement, authority, trust, etc. It’ll be interesting to learn how this type of analytics merges with more traditional analytics environments (one of my goals this year).

So, is the visitor clicking but not paying attention? From both a psycho-cognitive and web analytics perspective, now we’ll know. Are they staying on the site but not paying attention? Ditto. Are they interacting? We’re able to not only say yes or no but also give good approximations of how much of their cognitive resources they’re devoting to your website (are they navigating the site while talking on the phone? Are they talking to someone sitting beside them while sharing something about the site? Are they navigating your site because someone suggested it to them? How important is your information to them? Answering questions like this is quite simple now).

More to the point, in answer to Eric’s “I think if they’re not engaged, the answer to those questions will be ‘no’ and thusly their calculated metric would be lower.” is a determination of just how engaged visitors en masse and individually are. This, in turn, means we can bring some more sophisticated mathematical tools forward to answer some questions such as:

1) How engaged does someone have to be before they convert?

2) How much attention does someone have to devote to a site during a visit in order to guarantee a return visit?

(and most significantly)

3) What can be changed/edited/modified/updated to make these things happen faster/sooner/quicker/better?

Eric also comments that people struggle with the calculation for various reasons. Part of the process we went through was to make the calculation more universal. The form shown in Continuing the discussion: Attention, Engagement, Authority, Influence is like general relativity. It’s meant to work under all conditions. Eric’s calculation (with some simple modifications) is like special relativity. It’s meant to work with metrics available through a standard and commonly used web interface and for his definition of engagement. To the question, “Is Eric’s definition valid?” Let me first offer that the economic value of a metric is directly proportional to:

1) the information value of what the metric reports on
2) as that information value is defined by some group with an interest in it and
3) that same group’s ability to change environmental factors so that
4) the metric changes report value (not information value) in direct and obvious response to that same group’s intentional changes in environmental factors.

In this context (and I admit it borrows from a study of language and semantics a bit), Eric’s definition is valid if consumers find value in it. The same is true for NextStage’s definition and everyone else’s.

And let me now share something that’s most important. Stating that engagement doesn’t exist as a metric is also valid if a certain group of consumers find no value in any existing definitions of engagement. It’s a question of reference frames and if no value exists for engagement in some reference frame, so be it. “General Relativity” will accept that and return (duh!) a value of “0″ for that reference frame.

Are simple metrics like “session duration”, “recency”, …, alone enough to define engagement? As I offer in Continuing the discussion: Attention, Engagement, Authority, Influence:

“Accuracy is a function of target size, not mathematical rigor. Accuracy of 10% with three variables active can quickly rise to 90% accuracy with as few as four or five variables active. Let me give you a “marketing” example. You’re selling to a) 53yo b) white c) males and you’re capturing 10% of that market. But what if you’re selling to a) 53 yo b) white c) males in d) NH with who e) are business travelers? Ah, well, now perhaps you’re capturing 90% of the market.

Some people aren’t aware that the opposite can also be true; it’s possible to achieve (for example) 90% accuracy with three variables and dwindle it to 10% when more variables are present. Imagine a bullseye style dartboard. You can get lots of darts in the yellow and good for you; that’s high accuracy. Then again, there are only five colors you can hit (five variables in the equation).

Now imagine a more traditional dartboard with a very small center area and lots of other areas indicating different values and multipliers. Both types of dartboards are circles, yet add or change a few variables and accuracy as a percentage of dead-centers is shot to heck.”

So, are simple metrics enough to define engagement? Depends how accurately you want to define it. If you want to use “session duration” to measure engagement then just call it “session duration” and keep things in their simplest form (the KISS philosophy). You have high session duration and want to call it engagement? Then great! The majority of your visitors are engaged. Are the majority of your visitors doing something useful? To themselves or to you? No? Then they are not engaged in an economically useful way (and I’m using “economics” in the NextStage sense of exchange, not a simple money concept). Eric’s definition concludes with the concept of business goals. If simple metrics fulfill your business goals then you’re good. If they don’t, join the discussion.

(more to follow, picking up with Eric’s “…eventually we’ll pay the price.” statement.

Joseph Carrabis added the following ...

Now picking up with Eric’s “Given the direction our (…) industry is heading, I’m absolutely convinced that if we’re afraid to model and make more complex calculations, well, eventually we’ll pay the price.”

I have to admit, sometimes my time delay in getting to things can amuse me. Eric states in the above that, unless we make more complex calculations for our metrics models and Steve Jackson suggests on Continuing the discussion: Attention, Engagement, Authority, Influence that the model I came up with is too complex to be used by practitioners, consultants, etc.

Yes, well…back to the drawing board, I guess…

I do think the framework I supplied is as universal as it can be and I’ve shared some of the resulting formulations with Eric (I believe we’re going to be publishing them at the SF Emetrics. Is that accurate, Eric?). The big gain from the alternative formulations is that they allow for new and alternative metrics to be used when and as necessary.

Both Eric and I are busily throwing various data sets at the reformulation. Our definitions aren’t the same and they are complimentary. There are times it makes sense to use both as part of the same uber-metric, times it makes sense to use only one or the other, times it makes sense to mix and match elements of each with the other.

That was one of the points of the exercise (for me). Then again, one of my philosophies is that we are more together than any one of us can be apart. Too much social anthropology, probably. Sigh.

To Jeff Chasin: I guess I’d wonder if anyone is still paying attention. I could respond sooner and then I’d be sacrificing my desire for accuracy.

To Steve Jackson: The reformulation I gave Eric doesn’t handled “disengagement” so much as it recognizes an “engagement” scale (0-10, for example) that can be applied to a sales funnel (or X funnel taking into my account to create a unified metrics theory). This allows you to recognize that individual, some or all of your visitors are moving forward, static, backward, and can be tied to a variety of site and psychodynamic factors (de