This post picks up where I left off on Starting the discussion: Attention, Engagement, Authority, Influence, … and is a follow on to Eric’s two parter (Measuring Engagement Online: The Next Stage and Measuring Online Engagement: Step One). Eric references me as “Mr. Carrabis” somewhere in there. Oh, he must think I’m ancient to call me “Mr.” I will be going through the rest of the comments people made to my original post and commenting on them there. It just takes me time (nobody noticed, I’m sure) because I tend to think things over a bit before responding to them.

And FYI, I’ll be posting elements of the equation’s derivation on BizMediaScience. I doubt anybody is sitting on the edge of their seat waiting to see what I integrate and if I use Tau or Gamma functions, so me thinks that’s best. I will be posting my thought process here, though, and welcome comment, suggestions and so on.

Form Follows Function

Right now the definitions of Engagement that I’m working with are Eric’s (Engagement is an estimate of the degree and depth of visitor interaction on the site against a clearly defined set of goals.), the one we use at NextStage (Engagement is the demonstration of Attention via psychomotor activity that serves to focus an individual’s Attention.) and the dictionary’s (participation, involvement, involution).

I’d love to hear anybody else’s definitions as I’ll work to incorporate them provided they adhere to the philosophy proposed in Starting the discussion: Attention, Engagement, Authority, Influence, …:

1. What do you mean when you use the words “engagement”, “attention”, and “trust” online?
2. Can you repeatedly measure what you mean by them so that there’s a reasonable surety that what you’re measuring is what you mean by the terms you’ve used?
3. Can you make these measurements through a commonly used web-enabled device?

Both Eric’s and NextStage’s definitions pass muster on item 1 above as both are declaratory statements. It’s also worth noting that both definitions contain the meanings of participation, involvement and involution in them as conceptualized in Eric Peterson’s Engagement Project and the Engagement Equation, Part 1. So far, we’re doing real good.

Item 3 above is a definer to our variable set — we are only allowed to use measurements that can be made through a commonly used web enabled device. Note that a “definer” isn’t something that sets limits. A limit would be something like “You can only use screen size and color depth”. The phrasing of item 3 is intentional. As technologies change so will the measurements that can be made through a “commonly used web enabled device”.

Item 3 is really our “looking forward” element. When web analytics started (and when NextStage started, for that matter) web enabled phones, pdas, iPhones, Smartphones, …, were unknown. Fortunately I’ve never let something’s lack of current existence deter me from anticipating it (if you’ve read our patent, you have an idea).

Item 3 covers our “extendable” and “extensable” concerns. It allows us to create a mathematical form that can alter its requirements as new capabilities and technologies come on line without sacrificing its ability to calculate and return real (business) value.

Likewise, with Items 1 and 3 covered, all that remains is to isolate a set of easily captured variables that satisfy the definitions in item 1. The equation — the mathematical form I’m using — allows for new variables and interfaces as they arrive. In other words, the final equation will be good now and way, way down the road. Again, this is something I learned from writing up NextStage’s patents; be as far thinking as possible.

There are other concerns and considerations that (by my nature) I’m putting into the mathematical form. For example, accuracy. Accuracy is a function of target size, not mathematical rigor. Accuracy of 10% with three variables active can quickly rise to 90% accuracy with as few as four or five variables active. Let me give you a “marketing” example. You’re selling to a) 53yo b) white c) males and you’re capturing 10% of that market. But what if you’re selling to a) 53 yo b) white c) males in d) NH with who e) are business travelers? Ah, well, now perhaps you’re capturing 90% of the market.

Some people aren’t aware that the opposite can also be true; it’s possible to achieve (for example) 90% accuracy with three variables and dwindle it to 10% when more variables are present. Imagine a bullseye style dartboard. You can get lots of darts in the yellow and good for you; that’s high accuracy. Then again, there are only five colors you can hit (five variables in the equation).

Now imagine a more traditional dartboard with a very small center area and lots of other areas indicating different values and multipliers. Both types of dartboards are circles, yet add or change a few variables and accuracy as a percentage of dead-centers is shot to heck.

Thus the mathematical form should allow for various degrees of accuracy based on what variables one is able to measure. This means that careful definition of the initial and possible variable sets is critical to the success of the metric as well as its accuracy.

Am I boring you yet?

Without driving you to drink, I chose to make the equation solvable by using three spaces, A, I and V. A is the set of all possible solutions of “engagement” (note that I’m not specifying “online”). I is the set of all possible interfaces that will ever exist (note that I’m not limiting this to “online” or even “machine”) and V is the set of all possible measurements that can be collected for a given interface I (note that I’m not limiting this to real-valued measurements). From here we can go through about two pages of finalized equations and whole lots more to get there.

Or I can show you the next to the last form in the derivation. The image above is important for lots of reasons. First, it’s really nothing more than a mathematical shorthand for Eric’s definition. But wait, there’s more!

It’s also a mathematical shorthand for NextStage’s definition. Also for the dictionary’s definition.

Are you excited yet? I love this stuff! And it gets better!

It’s also a mathematical shorthand for Eric’s definition blended with NextStage’s. Or for any group of definitions that follow Items 1, 2 and 3 above. If you can define the term (1) and can measure it repeatedly through a standard, commonly used interface whatever that may be now or in the future (3) then you get the formula above and it satisfies (2).

Want to know the best part? The formula above is also a mathematical form of SQL statements. You have some data hanging around somewhere and you want to create something you call “engagement” and you want to interrogate the data to create it? No probs. The formula above does the job. It doesn’t matter the size of the query result set nor the initial parameters.

Let me give you an example

Say you have some huge data set of N records and you’re really only interested in some portion of those records. That’s “N - n”. And you want to know how many visitors in this “N-n” group were “engaged” as you define it. That’s “h_i(v_i)” and is also where Eric’s definition, NextStage’s, Eric’s and NextStage’s, Eric’s with five parts of NextStage’s, two parts of somebody else’s, three parts of another person’s, …, goes. Now say of those “N - n” records you want some sequencing (maybe most “engaged” to least “engaged”, maybe something else). That’s “x_i - t_i“. And you want everything else constant, like geographic location and income level and who knows what else. That’s “D[h_k(v_k)]”. Not only can you hold things like geography, income, etc., constant, you can also hold other definitions of “engagement” constant while plucking out the one form you want for a given application or deliverable. And you only want “engagement” for a given interface (web as opposed to mall kiosk, etc). That’s “V” or “I” or “A”. The above basically reads as

SELECT “h_i(v_i)” AS “ENGAGEMENT”
FROM “N - n”
WHERE “D[h_k(v_k)]” = somevalue
AND “D[h_j(v_j)]” = someothervalue
AND …
GROUP BY “x_i - t_i”
ORDER BY “V” (or “I” or even “A”)

The “But”

The only caveat to the above is that each solution to A and I and V requires the interface to be monitoring a thinking being. I won’t even suggest “person”, but (there’s the “but”) you have to be evaluating “engagement” with something we can most easily define (but not limit ourselves to) as “a visitor”. This “but” has some teeth (and a good thing, too!). This means the only v_i available — the measurements and metrics you use — have to be something “a visitor” would do, which means they have to be recordable by a standard interface (item 3 above).

Okay. I’m done for today. It’s taken me about two weeks to come up with a derivation (that’s all the pages I’m not showing you) that works for any definition of engagement on any interface that has any metrics that fits items 1-3 above. Soon I’ll start showing how Eric’s solution fits and what inferences can be drawn from it.

Based on my experience, the majority of web analytics (WA) tools are currently managed by a single, in-house person.  More than likely, the WA tool is underutilized and the WA team is too small and undertrained.  Heck, web analytics is hard. ;)  I predict that within three years, media agencies will build out ‘web insights’ specialties and capitalize on the web analytics opportunity.  Why will they be successful?  (Disclosure: I work for an Omnicom Agency)

Current Lack of In-House Ownership

Where does WA expertise & ownership reside in your organization?  Marketing?  IT?  Somewhere in the middle?  Both?  There is no consistent answer.  And I doubt everyone in your organization that could find value in WA data uses it (or even knows it exists).  Rarely is WA tied into major company marketing priorities, but most analysts agree it should be.  The proverbial WA ball has fallen (or will fall) through the cracks somewhere between Marketing, IT, Web Dev firms, and interactive agencies.  My argument: the group spending the most money to drive web traffic should “own” the full user experience and thus, web analytics.  Typically, this group is an outside agency focused on display, email, paid search & SEO.  Website performance, usability, and page optimization has a large impact on these online investments and are factors directly tied to performance/ROI and overall marketing strategy.

Current Lack of Talent & Economies of Scale

There are not enough talented analysts to meet the current demand, much less the future needs of our data-driven, B.I.-focused industry.  “Experience”, although important, is not the same as talent.  Similar to SEO, scarcity of talent leaves a large gap between those who dabble and those who specialize/excel.  Independent specialists/consultants will realize that nearly the same WA tactics (data insights and actions) have major gains for clients and economies of scale are found.  To get access to the top tier of clients (and dollars), these specialists will either join or become acquired by large agencies to add additional value to the current agency optimization techniques.

WA Insights Improve Agency Work

Tastes Great or Less Filling?  …Put it on the website and see what visitors respond to more.  If a company invests in a regional TV/Radio media buy, what results were seen online from the area – and was it ROI positive to justify expansion into new markets?  For the Fortune 500, agencies are firmly entrenched in building the websites and driving the traffic to increase the company’s revenue.  With WA being the ultimate tool of understanding the online experience and improving it, shouldn’t agencies put it to work to support their vision?  I can’t think of a more appropriate group to gain insights and take action from the data that results in a major impact to the entire business.  When you can make millions of dollars in online spend convert 1% better by improving the cart process, you just made the client a heck of a lot of money and the agency more valuable to the client by increasing the media ROI.  Further support of the growing importance of the media agency was seen in AdAge’s “Why You Should Be in the Media-Agency Business.”  The article cited a recent Booz Allen Hamilton study that asked marketers which organizations would become more important to them by 2010.  Media companies, media planners and communications planners topped the list, with 52% of respondents believing they would be more integral.

Media strategists will seize the WA opportunity.  Clients are demanding agencies to be more accountable for online performance as well as more data-backed “proof” for business strategies and tactics.  The recent trend of the digitalization of offline media will only increase the need for talented analysts to interpret the data for the media agencies.  Only time will tell.  In the meantime, please share your thoughts.

Jeff Campbell
VP, Product Development
Resolution Media

Despite being a newly minted Omniture customer I constantly find myself somewhat out of sync with the announcements the company makes about being the “first ever” and “industry’s only”. Usually I chalk this confusion up to their being excellent marketers and sales people and my being, well, not an Omniture employee. But last week I read a press release from the company that had me wondering if I’ve been working in the same industry as the nice folk from Orme.

According to this very well written press release, “Omniture Fusion(TM) accelerates time to action for their customers” and is “a new industry implementation methodology that provides customers with an explicit roadmap for taking action on their analytics data, allowing them to improve their business in weeks if not days.” The press release goes on to say that “Omniture Consulting’s Fusion methodology takes a proactive, customer-centric approach by conducting a thorough business assessment that helps to determine vertical key performance indicators and business objectives” and that “Omniture Consulting formed the Fusion methodology in 2007.”

Now, I’m not in Utah this week and so I can’t just find Matt Belkin or one of his guys and ask, but there absolutely has to be something I’m missing about Omniture Fusion(TM). I mean, doesn’t every vendor use this exact same methodology when they implement a for-fee analytics solution? The exact same implementation strategy I wrote about in my now classic book Web Analytics Demystified back in 2004?

I have to be missing something … so the conversation I’d like to have here in the Future of blog is about the future of how the technology is implemented into the business, not just he nuts and bolts of the web site.

Setting aside for a moment Ian’s bold prediction, let’s assume that vendors like Omniture, WebTrends, Coremetrics, etc. continue to provide for-fee analytics solutions that need to be integrated into the existing business infrastructure. And let’s also assume, despite Ian’s recent announcement that Gatineau has an automagic integration solution, that Omniture Fusion(TM) Certified Consultants, Omniture’s competitors, and their ilk in the private sector are still required to manage the integration process …

If this is true, what does that process look like in the Future?

Rene, in his typically prescient fashion, has already outlined OX2/LBi’s vision for how integration consulting will happen as our industry matures, citing various specializations. But what if Ian is more right than wrong, and what if the stuff we’re doing today with so-called “Enterprise” software evolves requiring integrations to be held to a much higher standard? What if instead of proprietary data-warehouses and unfortunately thin (or non-existent) APIs we used SAS, Terradata, Oracle and SPSS to understand visitor behavior?  What if the finance organization starts looking for real forecasting information from web analytics, not just crappy estimates based on recently collected data?

I’ll put a stake in the ground and say that nothing changes about the process (either the one Omniture formed in 2007 or the one I first described in 2004 after having implemented for WebSideStory since 2002, whichever you want) but that the work we do becomes increasingly and perhaps insanely complex. Think about the work you do today — whether you’re a vendor like Ian, a consultant like Rene, an industry analyst like John, or those of you who are practitioners — and how it will change if web analytics is not so much integrated into your CMS, but part of the true Enterprise infrastructure?

If this vision plays out, the business objectives become bigger, the KPIs cross channels, and the business assessment needs to be conducted by an entirely different class (or a differently trained class) of individuals.  Instead of well-meaning folks just out of college, the true Enterprise might start looking to global business consultants like Accenture, EDS, and IBM for their web analytics implementations.  Perhaps this is what Accenture was thinking when they bought Maxamine and Memetrics recently, who knows?

Anyway, I suspect Kristi Knight will call me tomorrow and explain what Omniture Fusion(TM) is really all about which will be nice.  But I will still be very interested in what you, the Future Collective, think about where integrations are heading as web analytics continues to grow up.

Eric T. Peterson
CEO, Web Analytics Demystified, Inc.
http://www.webanalyticsdemystified.com