Scobleizer - Latest Comments in Real-time systems hurting long-term knowledge?

Re: Real-time systems hurting long-term knowledge?

technogran — Sat, 22 Aug 2009 11:16:22 -0000

You know Robert, you are now voicing what I thought when I began to realise that by spending loads of time 'twittering' I have been neglecting my blogs terribly.
Of course it all depends on your audience and who you want to reach, but I think this is definately food for thought.
Good post, and as always an interesting observation....

Re: Real-time systems hurting long-term knowledge?

cgyw — Tue, 04 Aug 2009 07:10:03 -0000

jimmy choo shoes

Re: Real-time systems hurting long-term knowledge?

Tibi Puiu — Thu, 02 Jul 2009 10:19:50 -0000

Can definitely relate to what you said, Robert. There's simply no chance blogs can be dethroned from the social media pedestal at the moment - lifestreams are great, but very limited, still very, very limited. Hopefully we'll some improvement in both search functionality and posterity storage.

Re: Real-time systems hurting long-term knowledge?

Steve Hurben — Wed, 01 Jul 2009 04:59:31 -0000

The Google Wave is coming..

Re: Real-time systems hurting long-term knowledge?

Michiel Sikkes — Tue, 30 Jun 2009 13:22:31 -0000

I never thought of Twitter and FriendFeed as knowledge bases. More static information like webpages, blogs, wiki's etc. are. Twitter and FriendFeed are a great portal and source to find out opinions and what people are talking about, but in an ideal situation after having talked about a subject for a bit, someone should summarize the conversation or research the matter and turn it into actual archivable re-findable knowledge.

That's why we still have school books, right? Twitter and FriendFeed are the teachers during class.

Which is to say: Great article :)

Re: Real-time systems hurting long-term knowledge?

Ron — Mon, 29 Jun 2009 23:17:05 -0000

Books and newspapers still work for me. I'd rather spend time with a good article than read the excerpt on Twitter or FF.

Re: Real-time systems hurting long-term knowledge?

Borislav Agapiev — Mon, 29 Jun 2009 22:08:04 -0000

Excellent points, however there is more to this story :)

The problems you mention are very true and due IMO to the "sliding window" nature of current real-time search. By that I mean the current search window is some fixed interval that keeps sliding forward and dropping off old entries. The end result is that real-time search is at present good only for things happening RIGHT NOW and even relatively recent stuff is quickly forgotten.

For mathematically inclined, one can say

Real_Time_Search(t) = dSearch(t)/dt

where dt is effectively the sliding window over which we are searching.

There is no real reason to do it this way, apart from expediency of initial implementations, perhaps. Real-time search should NOT be forgetting older entries. It should be definitely taking freshness as a MAJOR ranking signal but definitely keeping old stuff.

This way, assuming it is implemented well, real-time search should, with time, start converging to general search ala Google. That is actually the flip side of the above equation.

Re: Real-time systems hurting long-term knowledge?

David Ulevitch — Mon, 29 Jun 2009 17:55:34 -0000

I've thought this real-time nonsense is a waste. I prefer to be a producer rather than a consumer.

Re: Real-time systems hurting long-term knowledge?

etorsten — Mon, 29 Jun 2009 15:29:04 -0000

Remember the meaning of the word "twitter". Human communication is not meant for eternity. So you can´t find it later via search. It´s a bit like in the past. We lived even without search...

Re: Real-time systems hurting long-term knowledge?

etorsten — Mon, 29 Jun 2009 15:27:52 -0000

Remember the meaning of the word "twitter". Human communication is not meant for eternity. So you can´t find it later via search. It´s a bit like in the past. We lived even without search...

Re: Real-time systems hurting long-term knowledge?

disqus_n5j7jNaPkL — Mon, 29 Jun 2009 14:48:25 -0000

Another method to be found by Google is to post well named Flickr and Youtube submissions. Check out Google search results for "safari firebug", "reuters app", "steampunk maker faire", "angora ridge fire", ... In each case a Flickr photo or YouTube video is in the top five results.

Re: Real-time systems hurting long-term knowledge?

garrick Van Buren — Mon, 29 Jun 2009 13:30:27 -0000

Scoble - I agree. I originally built Cullect.com to parse Twitter feeds, but after a year, I've found those messages so ephemeral as to have no long term value. Since Cullect was designed to find 'importance' in feeds over the long term it quickly becomes clear how much noise these services generate.

Re: Real-time systems hurting long-term knowledge?

Shem Quo — Mon, 29 Jun 2009 13:21:04 -0000

Why not just combine authoritative sources like Google with Twitter (using Twitter as a way to attach optional commentary or for ranking content better) like was demonstrated with tweetnews which uses Yahoo! BOSS and Twitter Search

http://tweetnews.me

I really like this model and am really surprised this hasn't become the norm or taken over search.twitter.com
I just don't see my mom or dad searching Twitter directly as the results are noisy and unreliable
But with this model it adjusts to their normal setting.

Re: Real-time systems hurting long-term knowledge?

therealmccrea — Mon, 29 Jun 2009 13:08:28 -0000

Great post, Robert! I've been meaning to write one related to this topic. Let's see if I can summon the energy and clarity this week. :)

Re: Real-time systems hurting long-term knowledge?

Anthony Farrior — Mon, 29 Jun 2009 12:58:35 -0000

So confused right now. You just crossed the yellow light and made a left into Google search Town... Please explain further...

Re: Real-time systems hurting long-term knowledge?

Erick Schonfeld — Mon, 29 Jun 2009 12:50:00 -0000

This is why we need better real time search: http://tcrn.ch/4jq

Re: Real-time systems hurting long-term knowledge?

AlexSchleber — Mon, 29 Jun 2009 12:42:55 -0000

Jesse, the Google search on your Tweet Stream is a very poor substitute, as it still has more holes than a Swiss cheese. So far I've found only piping of (single user) Twitter RSS feeds into Tumblr to be a reasonably simple/sound solution as far as archiving is concerned (tweet frequency can't be too high, else Tumblr will shut it down).

Of course that won't solve Robert's wish for Time-bound snap-shots that can be easily resurfaced: Google itself, even with the recent "Recency Operator" improvements (Last 24hrs/week/month/year), still has a way to go before you can say "give me date range from: ... until: ..."

Sound to me like there are plenty of business opportunities around providing archiving abilities around given real-time queries. I.e. Robert can find what he wants on a given day or recent sequence of days SHORTLY AFTER the event, and then says: "Hoover up" all of this info and archive it for later... e.g. for the recent "140conf" or "140tc" etc.

That implies taking timely action because Twitter seems in no mood to let you back-search past about 30 days (at best, often it's only about 7 days anymore during daytime loads). They may of course already be selling access to the full range to corp. researchers for a lot of money.

Re: Real-time systems hurting long-term knowledge?

dave — Mon, 29 Jun 2009 12:14:03 -0000

I agree. And I have an idea -- delete both your FF and Twitter accounts.

Re: Real-time systems hurting long-term knowledge?

StephenPickering — Mon, 29 Jun 2009 10:55:15 -0000

OK, cool. I wasn't singling out Ourdoings, for instance. It looks great. I was just trying to make a general point to Robert that blogs maybe have changed and morphed, but remain a centralized hub of one's net identity. Actually, services like yours only help blogs by giving them more reach and connectivity with interesting products like yours. :-) I just thought a couple days ago he felt that blogs didn't matter anymore and to hinge everything one is doing kind of out there on the branches, when I felt the main trunk was still key. Just a philosophical thought. Thanks for your info though!

Re: Real-time systems hurting long-term knowledge?

Herschel — Mon, 29 Jun 2009 09:50:32 -0000

Robert,
There is an old saying "Don't put all your eggs in one basket."

With all things human, these pearls of wisdom can be broadly applied. As with our virtual interactions with each other we must balance the use of all technologies.

As you have documented so many times, instant communications such as Twitter and Friendfeed have great advantages. But as you bring up there are restrictions or draw-backs of these technologies.

Could you imagine trying to catalog, quantify and qualify just your own Tweet/Friendfeed "though channel" over a period of 10 years?

Just the massive amount of data would be overwhelming.

So, I guess if there are thoughts you want to preserve for historic reference you might want to sit down and give those thoughts the time and effort they deserve to be preserved.

Although chatter may lead to knowledge and action as demonstrated with the recent Irainian elections, when you have a lot of chatter you have a lot of noise as well and the good thoughts most likely will get drowned out and not archived very well.

One problem with our "gotta get it now" and "24 hour news cycle" is that people really don't take the long view of humanity and knowledge. Ask yourself this... How will you ensure that your best thoughts and actions will be remembered in 100 years?

Re: Real-time systems hurting long-term knowledge?

RobertHiggins — Mon, 29 Jun 2009 09:43:37 -0000

This is something that I have been thinking about recently, so the coincidence control center is working over time.

The twitter knowledge, the blog knowledge it is all very cumbersome. Blogs in particular haven't really changes since um early 80's. Its the same thing, dial in on my dads 9600 baud modem to the local bulletin board, yeah maybe you had to wait 30 minutes. Read the message type a message.

How far back can we retrieve those conversations? Now I would bet, that everything that has ever been twitted is stored some where. Could be wrong, but it would not make sense for twitter to destroy their content. It would kind of being like burning money.

Back to basic blogging and such, the problem is that the original poster says something. People comment and yes if you scroll through and read them you can parse some Knowledge from them. But why doesnt someone have a closing space on a blog. The keypoint is this the thought leaders post and let loose and move on to the next, it is so fleeting. If they were truly interested they would close the gate once they opened it. They would analyze the comment contents and they would deliver the meaning of what they learned. How they changed from the shared distributed knowledge.

That is the creative process, but blogging is mostly about posting something, letting the trolls take it and moving on to the next. yeah it is retrievable but is it usable?

Re: Real-time systems hurting long-term knowledge?

Marko Saric — Mon, 29 Jun 2009 08:29:18 -0000

You are spot on. By publishing all this content of yours on a blog it will stay there forever and people will easily be able to find it in your archives via Google... you can still have your discussions / conversations on FriendFeed / Twitter but always make sure that it all starts with something you post on your blog.

Re: Real-time systems hurting long-term knowledge?

DGentry — Mon, 29 Jun 2009 08:19:09 -0000

Lets take your mountain bike example: assume that the twitter search problem is solved, so you can retrieve every relevant tweet in addition to every friendfeed response. Now what do you have? Hundreds of suggestions, probably clumping into a small number of popular choices and a large number of outliers. So now what?

Go buy the bike with the most suggestions? That will get you the most popular bike, but not necessarily the best one for you. This happens with camera recommendations all the time: a prosumer DSLR is not the best choice for many questioners, but its what the camera enthusiasts use.

Investigate every suggested model? Possible I suppose, but not very practical.

Facebook can improve this situation somewhat, in that your social graph should eventually allow you to express whose opinion you trust more than others. So you'd have a weighting based on who made the suggestion. Nonetheless this is still imperfect, it isn't realistic to have a single trust metric for every possible subject.

What we lack is a measure of authoritativeness of the _answer_: both an inherent quality of the person plus an indication of how sure they are of the answer they are giving. Suggesting a bike based on their own experience of test-riding a dozen models six months ago is far more authoritative than suggesting the cool bike they saw on University Avenue yesterday. There is also a timeliness function: test-riding bikes 6 years ago is less useful than 6 months.

How to generate this trust metric? Requiring people to do anything manually, to rate the strength of their answer, is doomed to fail. It is both annoying and easy to game. Measuring how many recent events in their lifestream are relevant to the question might be one way. If they mention their timing to bike to work each morning, they probably have much more interest in and knowledge of bicycles than most.

Then again, the person who bikes every day isn't going to be the best source if you're only planning to use it twice a month. They're likely to suggest "too much bike". You can still see how access to the lifestream would help though, by gauging how closely their activity in this area matches yours.

Re: Real-time systems hurting long-term knowledge?

brlewis — Mon, 29 Jun 2009 07:52:41 -0000

OurDoings won't go out of business because it hasn't gone into business yet. I developed it and continue to build it as a side project. Also, the comments live on FriendFeed and Disqus, just like on this blog.

Re: Real-time systems hurting long-term knowledge?

Atle Iversen — Mon, 29 Jun 2009 07:26:58 -0000

Twitter is for short-term chatting, blogs are for long-term knowledge.

If you are "part of the conversation" in Twitter, you MAY learn something, if not it is more or less gone, like any other conversation anywhere.

If you want to share knowledge with people, Twitter is probably the worst tool. If you want to inform people about the latest happenings in your life, or ask a "random" bunch of people a question, Twitter MAY be useful.