Continuing the discussion: Attention, Engagement, Authority, Influence
This post picks up where I left off on Starting the discussion: Attention, Engagement, Authority, Influence, … and is a follow on to Eric’s two parter (Measuring Engagement Online: The Next Stage and Measuring Online Engagement: Step One). Eric references me as “Mr. Carrabis” somewhere in there. Oh, he must think I’m ancient to call me “Mr.” I will be going through the rest of the comments people made to my original post and commenting on them there. It just takes me time (nobody noticed, I’m sure) because I tend to think things over a bit before responding to them.
And FYI, I’ll be posting elements of the equation’s derivation on BizMediaScience. I doubt anybody is sitting on the edge of their seat waiting to see what I integrate and if I use Tau or Gamma functions, so me thinks that’s best. I will be posting my thought process here, though, and welcome comment, suggestions and so on.
Form Follows Function
Right now the definitions of Engagement that I’m working with are Eric’s (Engagement is an estimate of the degree and depth of visitor interaction on the site against a clearly defined set of goals.), the one we use at NextStage (Engagement is the demonstration of Attention via psychomotor activity that serves to focus an individual’s Attention.) and the dictionary’s (participation, involvement, involution).
I’d love to hear anybody else’s definitions as I’ll work to incorporate them provided they adhere to the philosophy proposed in Starting the discussion: Attention, Engagement, Authority, Influence, …:
1. What do you mean when you use the words “engagement”, “attention”, and “trust” online?
2. Can you repeatedly measure what you mean by them so that there’s a reasonable surety that what you’re measuring is what you mean by the terms you’ve used?
3. Can you make these measurements through a commonly used web-enabled device?
Both Eric’s and NextStage’s definitions pass muster on item 1 above as both are declaratory statements. It’s also worth noting that both definitions contain the meanings of participation, involvement and involution in them as conceptualized in Eric Peterson’s Engagement Project and the Engagement Equation, Part 1. So far, we’re doing real good.
Item 3 above is a definer to our variable set — we are only allowed to use measurements that can be made through a commonly used web enabled device. Note that a “definer” isn’t something that sets limits. A limit would be something like “You can only use screen size and color depth”. The phrasing of item 3 is intentional. As technologies change so will the measurements that can be made through a “commonly used web enabled device”.
Item 3 is really our “looking forward” element. When web analytics started (and when NextStage started, for that matter) web enabled phones, pdas, iPhones, Smartphones, …, were unknown. Fortunately I’ve never let something’s lack of current existence deter me from anticipating it (if you’ve read our patent, you have an idea).
Item 3 covers our “extendable” and “extensable” concerns. It allows us to create a mathematical form that can alter its requirements as new capabilities and technologies come on line without sacrificing its ability to calculate and return real (business) value.
Likewise, with Items 1 and 3 covered, all that remains is to isolate a set of easily captured variables that satisfy the definitions in item 1. The equation — the mathematical form I’m using — allows for new variables and interfaces as they arrive. In other words, the final equation will be good now and way, way down the road. Again, this is something I learned from writing up NextStage’s patents; be as far thinking as possible.
There are other concerns and considerations that (by my nature) I’m putting into the mathematical form. For example, accuracy. Accuracy is a function of target size, not mathematical rigor. Accuracy of 10% with three variables active can quickly rise to 90% accuracy with as few as four or five variables active. Let me give you a “marketing” example. You’re selling to a) 53yo b) white c) males and you’re capturing 10% of that market. But what if you’re selling to a) 53 yo b) white c) males in d) NH with who e) are business travelers? Ah, well, now perhaps you’re capturing 90% of the market.
Some people aren’t aware that the opposite can also be true; it’s possible to achieve (for example) 90% accuracy with three variables and dwindle it to 10% when more variables are present. Imagine a bullseye style dartboard. You can get lots of darts in the yellow and good for you; that’s high accuracy. Then again, there are only five colors you can hit (five variables in the equation).
Now imagine a more traditional dartboard with a very small center area and lots of other areas indicating different values and multipliers. Both types of dartboards are circles, yet add or change a few variables and accuracy as a percentage of dead-centers is shot to heck.
Thus the mathematical form should allow for various degrees of accuracy based on what variables one is able to measure. This means that careful definition of the initial and possible variable sets is critical to the success of the metric as well as its accuracy.
Am I boring you yet?
Without driving you to drink, I chose to make the equation solvable by using three spaces, A, I and V. A is the set of all possible solutions of “engagement” (note that I’m not specifying “online”). I is the set of all possible interfaces that will ever exist (note that I’m not limiting this to “online” or even “machine”) and V is the set of all possible measurements that can be collected for a given interface I (note that I’m not limiting this to real-valued measurements). From here we can go through about two pages of finalized equations and whole lots more to get there.
Or I can show you the next to the last form in the derivation. The image above is important for lots of reasons. First, it’s really nothing more than a mathematical shorthand for Eric’s definition. But wait, there’s more!
It’s also a mathematical shorthand for NextStage’s definition. Also for the dictionary’s definition.
Are you excited yet? I love this stuff! And it gets better!
It’s also a mathematical shorthand for Eric’s definition blended with NextStage’s. Or for any group of definitions that follow Items 1, 2 and 3 above. If you can define the term (1) and can measure it repeatedly through a standard, commonly used interface whatever that may be now or in the future (3) then you get the formula above and it satisfies (2).
Want to know the best part? The formula above is also a mathematical form of SQL statements. You have some data hanging around somewhere and you want to create something you call “engagement” and you want to interrogate the data to create it? No probs. The formula above does the job. It doesn’t matter the size of the query result set nor the initial parameters.
Let me give you an example
Say you have some huge data set of N records and you’re really only interested in some portion of those records. That’s “N - n”. And you want to know how many visitors in this “N-n” group were “engaged” as you define it. That’s “hi(vi)” and is also where Eric’s definition, NextStage’s, Eric’s and NextStage’s, Eric’s with five parts of NextStage’s, two parts of somebody else’s, three parts of another person’s, …, goes. Now say of those “N - n” records you want some sequencing (maybe most “engaged” to least “engaged”, maybe something else). That’s “xi - ti“. And you want everything else constant, like geographic location and income level and who knows what else. That’s “D[hk(vk)]”. Not only can you hold things like geography, income, etc., constant, you can also hold other definitions of “engagement” constant while plucking out the one form you want for a given application or deliverable. And you only want “engagement” for a given interface (web as opposed to mall kiosk, etc). That’s “V” or “I” or “A”. The above basically reads as
SELECT “hi(vi)” AS “ENGAGEMENT”
FROM “N - n”
WHERE “D[hk(vk)]” = somevalue
AND “D[hj(vj)]” = someothervalue
AND …
GROUP BY “xi - ti”
ORDER BY “V” (or “I” or even “A”)
The “But”
The only caveat to the above is that each solution to A and I and V requires the interface to be monitoring a thinking being. I won’t even suggest “person”, but (there’s the “but”) you have to be evaluating “engagement” with something we can most easily define (but not limit ourselves to) as “a visitor”. This “but” has some teeth (and a good thing, too!). This means the only vi available — the measurements and metrics you use — have to be something “a visitor” would do, which means they have to be recordable by a standard interface (item 3 above).
Okay. I’m done for today. It’s taken me about two weeks to come up with a derivation (that’s all the pages I’m not showing you) that works for any definition of engagement on any interface that has any metrics that fits items 1-3 above. Soon I’ll start showing how Eric’s solution fits and what inferences can be drawn from it.
Joseph Carrabis added the following ...
Howdy and thanks for your comments. I’ll do my best to respond to them.
Firstly. What are you trying to achieve? I’m working to create a framework that allows different disciplines to merge in ways that provide value. I’m pretty sure I’ve stated this several times and in several ways in several places.
what will it allow me to do as a consultant with my customers? or as a practioner? Before you answer that bear my second point in mind. I have no knowledge of you or your customers, hence am at a handicap and will do my best to explain what the formula allows me to do as a consultant with my customers and in my practice.
This reformulation allows me to let my customers know which definition of “engagement” or which mixture of definitions is best going to suit their business objectives.
I can determine which of all metrics is most directly affecting visitor “engagement”.
I can determine what — out of all possible modifications — needs to be modified and how to modify it in order to achieve business goals.
It shows me very quickly how different “engagement” paradigms come together and allows me to begin studying their overlap.
From here I can begin predicting which visitors will be engaged and at what levels, possibly providing a new segmentation for the sales funnel. (might have commented on this in more detail at Comment 199)
I find the above useful in my practice and none of them is as important (to me) as recognizing that the form shown above is the theory. Now that I understand the theory I can very easily determine which applications of the theory will work with browsers, with smartphones, with mall kiosks, … However, without first knowing the theory I could never fathom that a chosen method for determining “engagement” on a website would fail when attempting to measure the “engagement” of someone walking down the street looking for the nearest Starbuck’s on their smartphone.
Eric wrote at one point “The purported complexity of my calculation…” and went on to mention that some folks were having various challenges with it. One of the things that falls out of the above is three or four simplified versions of Eric’s calculation, each version adding a level of accuracy based on a) his definition of “engagement” and b) how much effort someone is willing to put in to determining how “engaged” visitors are.
This is quite valuable to me.
Secondly, showing the current example to my customers would result in me being showed the door and never being invited back. The average practitioner might be put off by it’s complexity. I would suggest this would be something that both NextStage and Eric would never try to explain to a client. At least when I’ve worked with Eric his thoughts are easily translated into non mathematical terms. I’m guessing that your clients, Eric’s clients and NextStage’s clients hire us for the same reason; to make the intractable tractable. Lots of NextStage’s clients come to us basically saying that web analytics doesn’t tell them what to do, only what has been done. One of the things NextStage does is come up with mathematical models that then become simple tools that provide simple yet concise instructions on what to do. This isn’t my attempt to sell anything here, it’s my attempt to share that without a thorough theoretical understanding then applications become vertical boundaries. Theoretical frameworks provide horizontal frameworks from which a wide variety of vertical applications can grow. And better, by knowing what doesn’t work in application A I can quickly determine whether or not something similar might work or not in application B.
Example: A hammer is a hammer is a hammer, and until you understand the concepts of force over distance and basic angular momentum principles (you don’t need to know those terms, only the principles behind them) then you’ll never understand that knowing why you should drive a nail by holding the hammer at the base rather than the neck explains how to hit a ball out of the park every time you swing (and connect, of course).
(And it’s probably worth noting that I am what is considered a “second-order” tool maker. I create tools that create tools.)
I guess my question is how do we ever communicate this easily when this is the way forward?
SELECT “hi(vi)” AS “ENGAGEMENT”
FROM “N - n”
WHERE “D[hk(vk)]” = somevalue
AND “D[hj(vj)]” = someothervalue
AND …
GROUP BY “xi - ti”
ORDER BY “V” (or “I” or even “A”)
The error is mine as I thought framing the formula as a SQL statement would have explained it a bit.
If this is an academic debate … I disagree that it’s an academic debate. That’s just me.
…that will explain to the minority who read this blog and understand your lines of thought (me included by the way) that there is a standard way to describe and understand engagement…yes. That’s the key. There is a standard way to describe “engagement” based on how the term is defined. It’s not a matter of correct or incorrect descriptions because (like relativity) the best definition depends on a) what you want to measure and b) how you want to measure it.
…then I’m for it because it allows me to see how the great minds in this field are thinking.
However I would never use it unless I could explain it to a 55 year old executive that has an extremely busy schedule, has never done web/customer analytics and doesn’t particularly care about the math, but does care about how to spend the budget she has for this quarter. As you wish (and using some minor modifications to Eric’s definitions to simplify the calculation); “If you, Ms. Executive, can provide me with click depth, loyalty, feedback and interaction values then I can provide you with a reasonably good idea of how ‘engaged’ the majority of your visitors are. If you can add in duration and recency then I can both blow your socks off and tell you what to do to get more of them engaged. If you can segment your audience into branded and non-branded events then I can tell you how to get more conversions. If you can provide the average number of visits it takes for a visitor to convert, we can fine tune further, possibly shortening the interval.”
However, without going through the math I did I would never be able to suggest a statement such as the one above because I wouldn’t have noticed which metrics (click depth, loyalty, feedback and interaction) have common factors, how to redefine those metrics so that they generate (depending on your requirements) either smooth curves or step-wise functions, that duration and recency require different scaling functions to behave correctly in the calculation or that branded and non-branded events are providing completely different information sets about site visitors.
I don’t see the point of creating anything which can’t be easily described and understood by everyone who sees it. That’s why I find this a tad irritating. If you can boil it down for my customers I could get really enthusiastic about it. Fair enough. My role in most of my jobs has always been to provide technical or research foundations for others to use in their practice so what I’m offering here might not be of immediate value to you.
Yes we may have some long and convulted equation that works for describing engagement but unless you can boil it down to something really simple like e = MC squared or better yet engagement equals click depth and duration then the people that matter (the clients paying our salaries) aren’t going to be impressed by any of this work. I’ve been likening the above to general relativity and Eric’s calculation (corrected) to special relativity in discussions for a while now.
We all have different definitions of engagement now and as I understand it translate that to the customer quite well.
Apart from attempting to define something not currently defined in mathematical terms what does this discussion achieve? I think I explain this above. Let me know if I don’t.
I mentioned before that Eric had published what I thought was a very comprehensive formula to describe engagement. I said that because his formula touched on every aspect of the customer lifecycle, it measured an aspect of the whole thing. But Eric freely admits that it wasn’t the answer to everyones’ engagement problem, merely an iteration of his own thoughts on the matter which span back years and allow him to determine his best source of leads from his own marketing initiatives. And with all due respect to Eric and you, the caveating that occurs in this paragraph is exactly why I went through the exercise. A definition of anything that only works for certain things at certain times under certain conditions is not a useful definition to me as I can’t guarantee that things, times and conditions will always apply. However, understanding why certain things, times and conditions make things work in certain ways I can translate the things, times and conditions so they’ll work in other ways as needed.
I’ve written before about the anthropologist and the microbiologist having lunch, the microbiologist looks at her watch and exclaims, “My goodness! I have to go destroy a culture!” and the anthropologist has a heart attack. Unless you recognize at what “culture” means then it’s hard to appreciate that two disparate fields could use the term with equal eloquency to describe different things.
Your version seems to be an even more in depth one than Erics. However it’s not in my opinion going to be the answer because it’s not going to be used easily.
I don’t have a definitive way to describe engagement, nor do I have a one rules all formula but I do have a way to describe the lifecycle and measure it in a client friendly understandable way. We call it REAN. Reach, Engage, Activate and Nurture where Engage is simply click depth and duration - in other words interaction metrics. For what it’s worth, the above formulation accomodates your definition of “engagement”. As I wrote in Comment 199, “So, are simple metrics enough to define engagement? Depends how accurately you want to define it. If you want to use “session duration” to measure engagement then just call it “session duration” and keep things in their simplest form (the KISS philosophy). You have high session duration and want to call it engagement? Then great! The majority of your visitors are engaged. Are the majority of your visitors doing something useful? To themselves or to you? No? Then they are not engaged in an economically useful way (and I’m using “economics” in the NextStage sense of exchange, not a simple money concept). Eric’s definition concludes with the concept of business goals. If simple metrics fulfill your business goals then you’re good. If they don’t, join the discussion.
Steve Jackson added the following ...
The hammer analogy is the best one I can think of to hit the point home (pardon the pun).
A hammer is easily understood. A hammer can be described in terms like “well you just hit the nail with this end and it sticks in the wall”.
That’s what is needed, no complex stuff is required.
Why do we need to go there when we all know that it works?
I took two looks at Erics formula originally. The first time I saw it worked. The second time I saw it worked in the context of the REAN model. I thought yep, cool REAN works as well - though I already knew this, I remember getting all excited about it and showing Eric at the time.
A 55 yr old exec (an average one anyway) wouldn’t get Eric’s formula but would get something like “Try this and you might generate more leads”.
I truly believe that you will crack it and you will be able to define Engagement to all.
But isn’t it a bit like trying to define the concepts of force over distance and basic angular momentum principles when all you’re really talking about is a hammer?
The moon was something no-one had conquered, so was Everest. Have we really not conquered engagement already?
You said;
A definition of anything that only works for certain things at certain times under certain conditions is not a useful definition to me as I can’t guarantee that things, times and conditions will always apply.
I agree.
That’s why we use dimensions (as defined by the WAA also) to group our metrics into a standard lifecycle model. The hammer without the theory? I dunno, I’m just kind of wondering out loud.
Daniel Shields added the following ...
Hey Joe/Eric…Collective:
For retail sites we’ve worked to try to define three subsets of engagement which helps us understand points at which behavior resonates with the action performed on the site. As a result, our engagement takes into account the impulse; which is the act of bouncing or not bouncing at the processing of a page (typically this is less than 6 seconds). We usually refer to this as simple engagement. Other folks probably have their own names for this. After this, there is an assesment of the mindset based on dispositions and understanding how people are using the information which is presented to them. Based on the values which can be segmented out from each data point and our naming conventions, we determine actionability. This is where a user has collected enough information to act on the secondary, post-impulse, idea of the page and the options which are presented. The last, is the measure of conviction. This is follow through. To what extent or in what level of frequency does this ‘type’ of user go through, or complete the action which they have committed to. Subsequent and subordinate measures help us understand where human decision processes are involved, and, simultaneously provide a steady measure of how our technical performance is being executed.
I know this is in some ways mathematically complicated to express. It is a work in progress and we’re trying all the time to elevate this further, but we’ve found that uncovering users intent, and plotting areas where they accelerate or hyper-accelerate their process gives us an advantage to provide similar information quality in areas where potential for these exhibited traits exist. I would love to work with anyone on developing this further and pushing wider understanding of these types of things.
Sincerely,
Daniel
Persuasive e-Marketing» Blog Archive » Should Dayparting be a part of your day? added the following ...
[…] route: Understanding how and why a visitor moves through your site in a given session towards […]
Daniel Shields at Web Analytics Demystified » Blog Archive » Thinking Outside of Out of the Box in Commercial Web Analytics added the following ...
[…] around for people to discuss many of their calculated metrics can be difficult. Joseph and Eric and their ongoing banter on “Engagement” certainly merits its share of attention. This was the primary topic recently at the eMetrics […]
Steve Jackson added the following ...
You asked at one point “Am I boring you yet?” I’m not bored I’m irritated.
I mean that in the nicest possible way and respect the work you’ve done so allow me to explain.
Firstly. What are you trying to achieve? what will it allow me to do as a consultant with my customers? or as a practioner? Before you answer that bear my second point in mind.
Secondly, showing the current example to my customers would result in me being showed the door and never being invited back. The average practitioner might be put off by it’s complexity. I would suggest this would be something that both NextStage and Eric would never try to explain to a client. At least when I’ve worked with Eric his thoughts are easily translated into non mathmatical terms.
I guess my question is how do we ever communicate this easily when this is the way forward?
SELECT “hi(vi)” AS “ENGAGEMENT”
FROM “N - n”
WHERE “D[hk(vk)]” = somevalue
AND “D[hj(vj)]” = someothervalue
AND …
GROUP BY “xi - ti”
ORDER BY “V” (or “I” or even “A”)
If this is an academic debate that will explain to the minority who read this blog and understand your lines of thought (me included by the way) that there is a standard way to describe and understand engagement then I’m for it because it allows me to see how the great minds in this field are thinking.
However I would never use it unless I could explain it to a 55 year old executive that has an extremely busy schedule, has never done web/customer analytics and doesn’t particularly care about the math, but does care about how to spend the budget she has for this quarter.
I don’t see the point of creating anything which can’t be easily described and understood by everyone who sees it. That’s why I find this a tad irritating. If you can boil it down for my customers I could get really enthusiastic about it.
Yes we may have some long and convulted equation that works for describing engagement but unless you can boil it down to something really simple like e = MC squared or better yet engagement equals click depth and duration then the people that matter (the clients paying our salaries) aren’t going to be impressed by any of this work.
We all have different definitions of engagement now and as I understand it translate that to the customer quite well.
Apart from attempting to define something not currently defined in mathmatical terms what does this discussion achieve?
I mentioned before that Eric had published what I thought was a very comprehensive formula to describe engagement. I said that because his formula touched on every aspect of the customer lifecycle, it measured an aspect of the whole thing. But Eric freely admits that it wasn’t the answer to everyones’ engagement problem, merely an iteration of his own thoughts on the matter which span back years and allow him to determine his best source of leads from his own marketing initiatives.
Your version seems to be an even more in depth one than Erics. However it’s not in my opinion going to be the answer because it’s not going to be used easily.
I don’t have a definitive way to describe engagement, nor do I have a one rules all formula but I do have a way to describe the lifecycle and measure it in a client friendly understandable way. We call it REAN. Reach, Engage, Activate and Nurture where Engage is simply click depth and duration - in other words interaction metrics.