Basic premise is that when you teach someone a new idea, there are four levels of learning.
1. Reactions (they heard you) 2. Learning (they understood and retained) 3. Transfer (they took in the information and can apply) 4. Results (they use the learning to achieve a goal)
An example would be telling your child about tieing his shoes, him understanding the principle of making the bunny ears, doing the task with your coaching, and finally tieing his shoes alone.
I propose a four-level model for measuring engagement:
1. Click - A reader arrived (current metric) 2. Consume - A reader read the content 3. Understoon - A reader understood the content and remembers 4. Applied - A reader applies the content in another venue
Each of these could be measured (with accuracy pretty closely connected to your budget) and compared with other ways of getting the information out.
What do you think?
Jevon
· 3 years ago
This is an OLD topic of conversation. I can remember diving into it in 2001 pretty seriously, the writing was on the wall back then.
It would be great to dig up some of that old material and find out what others have been doing with this as well.
Drew
· 3 years ago
Good point on engagement. This is the part of the conversation that breaks out of the traditional CPM model (or ups the cpm cost depending on how you break it down); each eyeball has half a mouth. :)
Michael Cage
· 3 years ago
Direct marketers have been using engagement measures on the web for a while, it's easy with the logging available on a web site.
"The more people interact with you the more they will buy from you."
brem
· 3 years ago
whoa... one of your best posts in a long time. indeed, it's hard to parametrize an audience, but I think engagement is definitly something to be measure more accurately for marketers.
engtech
· 3 years ago
I was thinking about this the other day from the point of view of blog promotion.
Social networks are fine and dandy, but the ideal audience for a blogger is other bloggers because they can link, they are likely to get involved, and they have their own audience.
murph
· 3 years ago
One of the registers failings is that it very seldom links to anything (this is a generic problem with "news" type sites - article about something or someone and no link to the product page or their bio, it can be done - the BBC does it very well).
colin
· 3 years ago
"...no one is willing to click a link, that means their audience has low engagement..."
Or they are claiming that their audience has high engagement with THEIR site and its articles and so doesn't want to leave to go to another site. If content is too compelling (esp. video) surely there's a tendancy for one to not click on a link, which is why video content needs video advertising and we're back to CPM deals again instead of CPC.
kbeegle
· 3 years ago
I couldn't agree more with the idea of measuring engagement which I also refer to as the energy an individual or group will commit to a product/idea. So, to tackle the idea of measuring it.
1. While I recognize it's obvious, track which sites are driving the most viewers to the product/idea specific page. 2. Measure how much discussion is taking place about the product/idea by creating a metrics tool that uses Technorati, Google Groups, and so forth. The software will then pull a random sample of comments so that somebody can put a qualitative score on whether the discussion is favorable or not to and how relevant is the discussion.
For videos, a question I have regarding Quicktime development is whether it is possible to (1) layer graphics over the quicktime movie which pull from the internet and (2)can some rudimentary logic (through Qscript) be built into to Quicktime to test for an internet connection and if so, display the track that uses graphics from a web site otherwise use the track that has graphics imbedded into the movie? If that is possible, it seems this is the answer to how much of a video someone has watched and then the ability to track whether they took action on a link in the video.
My final idea related to this is a quicktime type server that wraps the video track with a track containing links back to the server so that each video downloaded has a unique identifier built into the link. It's then possible to get an idea of how often people are sharing the video by checking which IP addresses between initial downloads of the video and links via the video.
Christopher Coulter
· 3 years ago
While I see the value for better overall definition of an audience, "engagement" falls into that vague "eyeballish" zone. It doesn't really mean anything. And "engaged" exactly how, as 'engaged' audiences might be twice the effort and cost, for the same result. Maybe for some advertisers better to narrow on the extreme targeted or busy types who want raw facts fast. You can't expect the CFO's to play engagement games.
Some people are busy running companies and don't want to cuddle up with vendors all the live day long. If said product doesn't work or doesn't meet needs or isn't at a good pricepoint, they will just get a new vendor. And sometimes (in fact, imho, most often) the most engaged audiences are those with the least amount of power (as they have the raw time to engage). Don't discount the busy and aloof, that's sometimes the real gold.
And you can't really compare USA Today audiences with bloggers, maybe only a few clicked thru, but maybe one was a busy CIO (who has no time for blog chatter) but saw it in USA Today when they shoved it under his hotel door, a CIO who decides to order 100 copies. Clicks aren't the end-all, that's too blogger nose-leveled view.
The only metric (to the advertiser that counts) is increased marketshare, engaged or not. But don't get me wrong, "engagement" or happy customers is vital too, just not the end all. Engaged customers do spread that 'word' around. And CIO types ask other CIOs and pick up the general vibe. And "engagement" gives a product a sense of being alive and provides a support framework.
Vic Berggren
· 3 years ago
In the ecommerce world we call this 'click to sale conversion rate'
Christopher Coulter
· 3 years ago
PS - Getting Steve Gillmor on anything would just buzzword it all up, angel-dust freebasing pure unintelligible nonsense, and scare off 90% of advertisers. You really think anyone from Peoria would understand a word Steve says?
And Steve is well just loony and agenda-driven, witness...
TV is dead because of the Internet. TV is dead because we don't have time for it.
Geee really? With Full Season DVDs, with Tivo/Replay, Media Center's and PVR's de jour, with Portable Video Players and iTunes and ilk, and even gray-area MPEG4s, (XVID, DIVX) formats on MASSIVE Torrent swarms, we have MUCH MUCH more time than ever before. The Internet ENHANCES TV, makes it MORE valuable, it provides that "engagement" level. The long-story arc becomes more viable, giving us better TV; death to the sitcomy episodic. Heck, has he never met the Battlestar Galactica, Veronica Mars, Lost or 4400 fan? People spend hours on forums talking, debating about TV. And what does he think makes up 50% of YouTube....TV clips. Oh we are watching, much much more than before, the big question is how to fund it all, those models are changing.
joeduck
· 3 years ago
Very good points. Comscore vs Danah on Myspace stats recently got into the challenges facing the difference between a user who simply loads a page and leaves vs a user who engages with the site.
Experiments are needed, since it may be as simple as taking a ratio of total unique visitors to total time online to get a sense of how engaged the visitors are.
Of course that does NOT necessarily translate into a buyer. I'm engaged here at Scobleizer, but not looking at your ads.
joeduck
· 3 years ago
Hey, I was wrong - I did know you advertised Naked Conversations (which I have and enjoyed reading). You have no other ads up so I can't say I didn't see them!
Robert Scoble
· 3 years ago
joeduck: but increasingly the advertising will be inside the content. See PayPerPost.
Russ Henry
· 3 years ago
I support the local climbing shop for a reason. Even though the local price is higher than online. Sorry it has taken me so long to purchase your book Robert.
It is my humble way of saying I support your efforts to enlighten the messes ; ).
Greg
· 3 years ago
Brilliant post. Keep up the good work!
Jitendra
· 3 years ago
Great post...Even the pages views are no good these days because of all the AJAX that is used. Indeed, what we need is something to capture engagement...
AttentionTrust approach could be used to capture the engagement metric. See my take on them at
ACTIONABILITY go beyond GestureLab, comSCORE seems overly qualified
INFLUENCE POWER wait for the algo look toward political marketing agencies bet on real social research that seeks to solve real problems beyond meme optimization
chinonofuji
· 3 years ago
hot potato! it is an area oft tread but not thoroughly reviewed nor defined. investigations have been cursory for predefined needs. will look forward to seeing all that this elicits.
John Federico
· 3 years ago
Robert,
Check out my original proposal for AxL: The Attention Exchange Language:
Compete (www.compete.com) just released a product that features 13 month traffic trends on any and every site our member community has ever visited… Including two very important components of engagement - Average Stay in minutes and Pages viewed per Visit.
*Think Alexa on steroids*.
Disclosure: Compete is my Employer
Podesta
· 3 years ago
Again, I am reminded why I like Chris Coulter. It is because he is smart.
'Engagement' is difficult to define, not to mention measure. And, truth be told, just going where the herd is directed is very common in the blogoshpere. I remember being pleased to be a popular blogger back in the day, only to be surprised by how little time most readers spent at my blog when they visited. Often, it was not long enough for them to have actually read a single entry in full. On the other hand, merely leaving a page open for a longer time can be confused with being engaged. Often, the person just forgot to close that window in his browser.
As for likeability, that is an invitation for superficiality. Often people who are intelligent and know their craft well are not particularly likeable. I would rather read or watch that kind of blogger than someone considered 'nice.'
Chris Heuer
· 3 years ago
One of the important measures of engagement is the ratio between posts and comments as well as the links to post ratio. This can be further determined by some of the attention issues around number of posts bookmarked and number of bookmarkers.
But I dont honestly know if engagement is the thing - it is a better thing to strive for then reach and popularity, but I tihnk we can do better even.
Supposedly the big advertising associations are thinking about this same metric (engagement via social media) but I don't trust them with it. It is kind of like asking the car salesman which bicycle to buy...
Dave
· 3 years ago
Engagement is a difficult thing to measure as it is so dependent on time. For me the best way to tell if I have been 'engaged' by a site (or anything else for that matter) is to see if I am still using it after 2 or 3 months. Humans love following fads (hence web 2.0!) and I find myself using new sites/products intensively for a matter of days or weeks before moving on to something else. Only a very small minority of these will see me returning consistently over a long period of time. I've no idea how you are meant to measure that!
Alain
· 3 years ago
I'm amazed at all of the web navel gazing here and that no one has stepped back and asked the obvious question. What is the difference between USA Today readers and Robert's readers? That is the simple Marketing 101 question. Robert's readers are by definition more technically minded and much more likely to be attracted to ActiveWords as a product. It's not a question of engagement, it's a question of segmentation and matching the product's attraction to the most profitable market subsegments. What Buzz is actually saying is that USA Today readership is a broad market with a low Heavy Usage Index (a measure of the relative intensity of consumption) for the software product category under which Active Words falls. Clearly Scoble's readers who are largely techs and business people who work for tech companies are focused on the cutting edge and are much more likely to purchase Active Words because they are willing to implement a productivity solution on their computer and futz around with Macros. Engagement? That is a contrived metric because it pretends to say that it measures how willing people are to "engage" with a website, when the reality changes depending on the consumer segment who is attracted to your particular market. The true measurements of interest should be metrics like purchase habits, loyalty, heavy usage index, and segment utilities. When you are able to properly target your products, then engagement comes naturally. So the real question is, how tightly does a website align with the key interests that define your highest potential consumers and how frequently do they come back to it.
Russ Henry
· 3 years ago
Most metrics from enterprise system administrators gage logon’s and time in the system as key indicators of favorable use of the system. Which is bunk, mainly due to it not indicating problems with crashes, analysis stalling, etc etc.
I agree with the “likeability metric” per user. The system “sucks or not” and reasons why would give better feedback to key developers and CIO’s without the need for costly and ineffective surveys which can be fudged or skewed in favor of the old “what do you want it to be?” guys which may be linked to profit driven support.
RSS KISS SOS. INNOVATE don't stagnate. This new conversation media is a step in the right direction to both consumer satisfaction and bottom-line profit if used correctly. IMHO and probably wrong O. No flaming required. ; )
Enrique Gonzales
· 3 years ago
As someone who has worked in web metrics for the past four years, I can tell you all web metrics are flawed. I can go one further and say all off-line media metrics are flawed as well.
Metrics are a currency for valuing advertising and media properties. The only thing a metric needs to be is consistent and accepted as a unit of measure between the those selling and buying media properties and advertising.
That's way easier said than done. Setting a metrics standard can put the thumb on the scale in favor of one player or another - Rocketboom or Zefrank.
A metric's only good if it can be accepted as a medium of exchange.
This issue is something the web analytics association has been grappling with for a while.
Robert - add to your equation "relevancy" which people seem to be forgetting. Even if your readers click on my link to Diva Marketing (engagement) they will not stay long, go deep or come back unless the content is relevant to them.
So while engagement is the first step relevancy is critical and should be integrated into any measurement model. Without it we are counting old fashioned 'hits' which may be great for the ego but from a business stance is a big So What.
raincoaster
· 3 years ago
Why has nobody mentioned YouTube in this context yet? I know when I look at a video (before I hit PLAY) I see how many video responses it's gotten, how many times favorited, how many comments, the star rating, etc, rather than just number of viewings.
They do an excellent job of articulating the degree of audience engagement for each and every video, I just wish there was some way to search for "most video responses" videos, etc.
Shahar Nechmad
· 3 years ago
Hi Robert (and all other readers). I'm the CEO of a NuConomy. You probably didn't hear about us until now as we were in stealth mode until a few days ago. Our offering is a platform for Internet sites that does just that - Gives you a sophisticated generic way to develop your own engagement formulas (we call it ranking for now. By using our platform you can rank your users and learn more about your business, and then also run financial schemes upon the results (rewards programs, revenue sharing, etc...). I would love to hear what you think about our offering (the good and the bad). Feel free to check out our web site or contact me directly in shahar at nuconomy.com
Alchemist
· 3 years ago
It goes back to the whole idea of mind share that was so hot back in the dot-com era. A site can get a lot of hits, and you can interpret it as a measure of popularity and awareness of the brand. However, does it actually mean anything? Are users actually going to come back or even remember your site? The current system of measuring audience engagement is a throwback to older methods of measuring ratings, but better tools can now be deployed to more fully gauge what something means to an audience.
I am happy this discussion is unfolding on this side of the World, rather than in the Marketing / Advertising side. Generally, it's the reverse.
free RAZR cell phone
· 3 years ago
i got a free razr from this website and thought i would share :-)
briansolis
· 2 years ago
I wrote about this recently here. BuzzLogic sent a note after reading it, that they have built a complete suite that measures engagement and conversations.
kjIJ15cpMF
· 2 years ago
Hi! Very nice site! Thanks you very much! d0xsWxt2KW
Chas Warner
· 2 years ago
Is the difficulty of measuring engagement inherent to the browser or the browser-based way we engage on the internet? It seems users need to be empowered in some way to measure their own level of engagement - but it needs to happen on the client side because so much of it is opaque to the server.
All Comments
Link rank can and is gamed, clickbots are out there, but how does Nelson rate TV shows now that we all have remotes in our hands surfing the dial?
http://coe.sdsu.edu/eet/articles/k4levels/index...
Basic premise is that when you teach someone a new idea, there are four levels of learning.
1. Reactions (they heard you)
2. Learning (they understood and retained)
3. Transfer (they took in the information and can apply)
4. Results (they use the learning to achieve a goal)
An example would be telling your child about tieing his shoes, him understanding the principle of making the bunny ears, doing the task with your coaching, and finally tieing his shoes alone.
I propose a four-level model for measuring engagement:
1. Click - A reader arrived (current metric)
2. Consume - A reader read the content
3. Understoon - A reader understood the content and remembers
4. Applied - A reader applies the content in another venue
Each of these could be measured (with accuracy pretty closely connected to your budget) and compared with other ways of getting the information out.
What do you think?
It would be great to dig up some of that old material and find out what others have been doing with this as well.
"The more people interact with you the more they will buy from you."
indeed, it's hard to parametrize an audience, but I think engagement is definitly something to be measure more accurately for marketers.
Social networks are fine and dandy, but the ideal audience for a blogger is other bloggers because they can link, they are likely to get involved, and they have their own audience.
Or they are claiming that their audience has high engagement with THEIR site and its articles and so doesn't want to leave to go to another site. If content is too compelling (esp. video) surely there's a tendancy for one to not click on a link, which is why video content needs video advertising and we're back to CPM deals again instead of CPC.
1. While I recognize it's obvious, track which sites are driving the most viewers to the product/idea specific page.
2. Measure how much discussion is taking place about the product/idea by creating a metrics tool that uses Technorati, Google Groups, and so forth. The software will then pull a random sample of comments so that somebody can put a qualitative score on whether the discussion is favorable or not to and how relevant is the discussion.
For videos, a question I have regarding Quicktime development is whether it is possible to (1) layer graphics over the quicktime movie which pull from the internet and (2)can some rudimentary logic (through Qscript) be built into to Quicktime to test for an internet connection and if so, display the track that uses graphics from a web site otherwise use the track that has graphics imbedded into the movie? If that is possible, it seems this is the answer to how much of a video someone has watched and then the ability to track whether they took action on a link in the video.
My final idea related to this is a quicktime type server that wraps the video track with a track containing links back to the server so that each video downloaded has a unique identifier built into the link. It's then possible to get an idea of how often people are sharing the video by checking which IP addresses between initial downloads of the video and links via the video.
Some people are busy running companies and don't want to cuddle up with vendors all the live day long. If said product doesn't work or doesn't meet needs or isn't at a good pricepoint, they will just get a new vendor. And sometimes (in fact, imho, most often) the most engaged audiences are those with the least amount of power (as they have the raw time to engage). Don't discount the busy and aloof, that's sometimes the real gold.
And you can't really compare USA Today audiences with bloggers, maybe only a few clicked thru, but maybe one was a busy CIO (who has no time for blog chatter) but saw it in USA Today when they shoved it under his hotel door, a CIO who decides to order 100 copies. Clicks aren't the end-all, that's too blogger nose-leveled view.
The only metric (to the advertiser that counts) is increased marketshare, engaged or not. But don't get me wrong, "engagement" or happy customers is vital too, just not the end all. Engaged customers do spread that 'word' around. And CIO types ask other CIOs and pick up the general vibe. And "engagement" gives a product a sense of being alive and provides a support framework.
And Steve is well just loony and agenda-driven, witness...
TV is dead because of the Internet. TV is dead because we don't have time for it.
Geee really? With Full Season DVDs, with Tivo/Replay, Media Center's and PVR's de jour, with Portable Video Players and iTunes and ilk, and even gray-area MPEG4s, (XVID, DIVX) formats on MASSIVE Torrent swarms, we have MUCH MUCH more time than ever before. The Internet ENHANCES TV, makes it MORE valuable, it provides that "engagement" level. The long-story arc becomes more viable, giving us better TV; death to the sitcomy episodic. Heck, has he never met the Battlestar Galactica, Veronica Mars, Lost or 4400 fan? People spend hours on forums talking, debating about TV. And what does he think makes up 50% of YouTube....TV clips. Oh we are watching, much much more than before, the big question is how to fund it all, those models are changing.
Experiments are needed, since it may be as simple as taking a ratio of total unique visitors to total time online to get a sense of how engaged the visitors are.
Of course that does NOT necessarily translate into a buyer. I'm engaged here at Scobleizer, but not looking at your ads.
It is my humble way of saying I support your efforts to enlighten the messes ; ).
AttentionTrust approach could be used to capture the engagement metric. See my take on them at
http://karmaweb.wordpress.com/2006/10/19/attent...
Thanks, Jitendra
Perhaps you just got a link from one of Orlowski's pages. Nobody reads those. :)
LIKEABILITY see http://en.wikipedia.org/wiki/Q_Score, although I can't say much for Nielsen Ratings these days, maybe next year
ACTIONABILITY go beyond GestureLab, comSCORE seems overly qualified
INFLUENCE POWER wait for the algo look toward political marketing agencies bet on real social research that seeks to solve real problems beyond meme optimization
Check out my original proposal for AxL: The Attention Exchange Language:
http://johnfederico.brandbrains.net/archives/20...
http://johnfederico.brandbrains.net/archives/20...
Comments are welcome, of course.
Regards,
-jf.
--
John Federico
http://www.newrules.com
http://johnfederico.brandbrains.net
http://www.odmcast.com
We could not agree with you more.
Compete (www.compete.com) just released a product that features 13 month traffic trends on any and every site our member community has ever visited… Including two very important components of engagement - Average Stay in minutes and Pages viewed per Visit.
*Think Alexa on steroids*.
Disclosure: Compete is my Employer
'Engagement' is difficult to define, not to mention measure. And, truth be told, just going where the herd is directed is very common in the blogoshpere. I remember being pleased to be a popular blogger back in the day, only to be surprised by how little time most readers spent at my blog when they visited. Often, it was not long enough for them to have actually read a single entry in full. On the other hand, merely leaving a page open for a longer time can be confused with being engaged. Often, the person just forgot to close that window in his browser.
As for likeability, that is an invitation for superficiality. Often people who are intelligent and know their craft well are not particularly likeable. I would rather read or watch that kind of blogger than someone considered 'nice.'
But I dont honestly know if engagement is the thing - it is a better thing to strive for then reach and popularity, but I tihnk we can do better even.
Supposedly the big advertising associations are thinking about this same metric (engagement via social media) but I don't trust them with it. It is kind of like asking the car salesman which bicycle to buy...
I agree with the “likeability metric” per user. The system “sucks or not” and reasons why would give better feedback to key developers and CIO’s without the need for costly and ineffective surveys which can be fudged or skewed in favor of the old “what do you want it to be?” guys which may be linked to profit driven support.
RSS KISS SOS. INNOVATE don't stagnate. This new conversation media is a step in the right direction to both consumer satisfaction and bottom-line profit if used correctly. IMHO and probably wrong O. No flaming required. ; )
Metrics are a currency for valuing advertising and media properties. The only thing a metric needs to be is consistent and accepted as a unit of measure between the those selling and buying media properties and advertising.
That's way easier said than done. Setting a metrics standard can put the thumb on the scale in favor of one player or another - Rocketboom or Zefrank.
A metric's only good if it can be accepted as a medium of exchange.
This issue is something the web analytics association has been grappling with for a while.
http://www.webanalyticsassociation.org/
Enrique
So while engagement is the first step relevancy is critical and should be integrated into any measurement model. Without it we are counting old fashioned 'hits' which may be great for the ego but from a business stance is a big So What.
They do an excellent job of articulating the degree of audience engagement for each and every video, I just wish there was some way to search for "most video responses" videos, etc.
I'm the CEO of a NuConomy. You probably didn't hear about us until now as we were in stealth mode until a few days ago.
Our offering is a platform for Internet sites that does just that - Gives you a sophisticated generic way to develop your own engagement formulas (we call it ranking for now.
By using our platform you can rank your users and learn more about your business, and then also run financial schemes upon the results (rewards programs, revenue sharing, etc...).
I would love to hear what you think about our offering (the good and the bad). Feel free to check out our web site or contact me directly in shahar at nuconomy.com
Actually, I have found Hitslinkt to be a useful tool. Your post in part inspired me to write this one for my blog.
I am happy this discussion is unfolding on this side of the World, rather than in the Marketing / Advertising side. Generally, it's the reverse.