-
Website
http://www.scobleizer.com/ -
Original page
http://scobleizer.com/2008/01/03/what-i-was-using-to-hit-facebook/ -
Subscribe
All Comments -
Community
-
Top Commenters
-
danja
44 comments · 4 points
-
polizeros
52 comments · 1 points
-
AndyBeard
69 comments · 4 points
-
Zachary Adam Cohen
35 comments · 8 points
-
dbarefoot
40 comments · 3 points
-
-
Popular Threads
-
The best and worst thing Twitter did in 2009: RT
13 hours ago · 19 comments
-
World-brand-building mistakes France’s entrepreneurs make
1 week ago · 181 comments
-
2010: the year SEO isn’t important anymore
6 days ago · 66 comments
-
iPhone developers abandoning app model for HTML5?
6 days ago · 51 comments
-
A 2010 real-time app development platform from Kynetx
10 hours ago · 2 comments
-
The best and worst thing Twitter did in 2009: RT
BTW, was able to download (not synch) with the Fb app FriendsCSV THEN will synch with Plaxo (after the import into Outlook). Ya, a lot of steps, but has worked.
One piece MISSING from the app though.. no e-mail address.
The disruption continues... ;-)
Names and email address and birthday?
Is that NOT personal?
So, that's going to be the demise of Open Web
Eventually, all the kids on the playground get into a fight -
Actually we don't think we were caught by the script itself. But we think another blogger sold me out. :-)
Not sure if I'm going to run it or not....
I understand Facebook's position, but I sure would like all my friends' contact info in one place.
And if everything fails, you just pay somebody oversees to type it in manually, you will get your data.
Because let's face it: I always will have data which is not in the systems by personal effort plus not all data will be in those systems.
Plaxo is the least of all evils at the moment for syncing (thought they did screw up most of my contacts and got rid of the mail address - hu?), and it allows me to take the data with me on my mobile phone.
As long as facebook is not giving the data, we will just get it somewhere else, basta. Deal with it, or just go back to being insignificant for that usage.
Will have a look at FriendsCSV now. :)
2. That's FB's get out of jail and get Scoble back in card. They can always say something like: "Oh well if that's all you're trying to do then that's OK."
3. having nibbled at that one then it becomes a full assault on all your/my social graph data?
Also, why can Facebook import from GMail? Those users didn't give permission for THEIR info to be taken off of Gmail.
http://apps.facebook.com/friendcsv/
And no where in the user agreement for Facebook does it say you're consenting to allowing you data to be automatically scraped and put into another service. With services which have long had APIs to let you access their data - Google and the like - this isn't an issue.
If Facebook turned around and said it was allowing third-party services to access my data on there, it would be extending the terms of service in exactly the same way that Google was criticised for doing with Reader. Personally, I wouldn't mind - but other people might.
So the thing is that whatever Plaxo Pulse was running (as a script) was actually unnecessary. There's absolutely no need to run a script when you can just have a separate application that makes a call for all of your friends' data. The "matching" with Plaxo contacts can then happen on Plaxo's server, and sent to the individual. So in short it looks like Plaxo just didn't realize that they could get these data w/o violating TOS.
If they were actually going a step further and scraping pages and trying to do some OCR on the e-mail addresses, then it's more of an issue.
But right now, with FriendCSV, if you have a gmail account, we facilitate the exact same matching that Scoble was talking about, and thousands of people have used this app without problem.
Except that when you sign up to Facebook, you know that no one in the real world is actually going to do that for 4999 friends - so there's a clear expectation that data isn't going to scraped en masse.
People used to bitch because they had to enter each contact record one at a time - then they wanted automated tools so that they could point to their address books and save them the hassle of entering manually.
Now, it's come full circle, and because people decided that they now want to take their data someplace else, they are starting to complain again.
face it - people are never going to be happy with anything that is available
Well, actually you CAN, in fact you were doing it.
Only, it is not ALLOWED by their TOS...
as Nicole already commented: "[...] And if everything fails, you just pay somebody oversees to type it in manually, you will get your data [...]"
In fact, I wonder how they are going to detect THAT.
But it is always that way: when technology can't really stop it, then TOS and policies and ultimately laywers get involved....
maybe facebook will wake up and smell the fresh air of open-ness someday
It's the Users that Create the Value to these Social Networking Sites! Facebook isn't Valued @ $15 Billion for the Crap Site it is thass fer darn sure*
It's the Eyeballs Stoopid!
;))
I wouldn't mind Running that Script on Flickr to not only get my 7,000 Contacts outta there but my 35,000 Photos + 40,000 Faves + 100,000 Comments + over 5 Million Views!!
Cheers!! ;))
Personally I wouldnt want "friends" on facebook to easily save my data such as post/wall/address/phone number etc onto their hard disk...it just make things easier for stalkers..god know how many stalkers there are on campus already.
I'm all for there being a way for people to reuse their information with other sites but this seems to be driving that reuse from the wrong angle. Ideally Facebook would provide an export interface and only those who had given permission to export their data would appear in the output. In the absence of this though I'm not sure you're right to scrape their pages anyway.
As mentioned by others the data your harvesting is actually pretty sensitive. One could argue that people shouldn't make this information public within their profile but people don't always realise they've done this or realise the implications of doing so - particularly when they're new to such a site.
"Same with Facebook. I’m not moving away from it. Why? I have 5,000 reasons why (and another 500 already who want to be included in my Facebook network)."
http://scobleizer.com/2007/10/02/steve-ballmer-...
Plaxo WAS using OCR to scrape these email addresses. I just wrote about that here (sorry to link-spam in the comment section, Scoble):
http://venturebeat.com/2008/01/03/i-was-about-t...
And thereby hangs a tale. Shall I make popcorn?
This has an interesting implication. It suggests that your Amish shunning at Facebook resulted not from hitting an automated tripwire -- which seems more reasonable; computers multiply inefficiency as well as efficiency -- but rather that your account was pulled by a HUMAN BEING.
Which seems all kinds of inexplicably stupid.
Hel-lo, he has maxed out the number of friends, lots of activity on his account, name "Scoble" seems vaguely familiar -- don't you think you'd maybe want to escalate that one up to Zuckerberg-level before pulling the plug?
No?
Someone at Facebook doesn't understand the way social networks work In Real Life.
http://www.dcs.shef.ac.uk/~mrowe/foafgenerator....
There is a lot of history behind Foaf as an open standard vocabulary for expressing metadata about people, and their interests, relationships and activities. http://xmlns.com/foaf/spec/
There are people on my Facebook friend's list I barely know who asked me to add them because they read my blog or are fans of software I've written. Should they now be able to extract my phone number, birthday, home address, email, education history, etc into "Random Web 2.0 Wannabe Social Network" because of this relationship? As a Facebook user, my answer is NO.
More thoughts at http://www.25hoursaday.com/weblog/2008/01/03/Fa...
Scoble, what's important, is that we will always be friends *in our hearts* whether you can't fit me on Facebook because there's a limit, or because Twitter is down, or because they booted you from Second Life, and *we'll always have Kyte.TV*.
But I'm glad someone tried it. FB has been allowing users to scrape App data for a LONG time, but someone definitely needed to make the OCR move to test the limits.
First off, this is like saying that once you've given your girlfriend your phone number, you have no way of taking it back once you break up. She still knows your number! You can't, like, force her to get amnesia or have a lobotomy! O the horror!
Presumably if you hand out information to people you trust, you trust them enough to not misuse the information later? Or should Facebook stop referring to your contacts as "friends," and instead call them "random strangers who might abuse the information I give them at some point in the future?"
Second, it's not like you can't accomplish the same thing BY HAND, reading all the information off the web page and typing it into Outlook. You are still able to "export information" out of Facebook; it's just not especially feasible to do it that way for 5,000 "friends." If you have 10 friends, there's nothing stopping you. In effect, Facebook is penalizing you for being a bigger user/having a larger network.
I think Dare's comment (and blog post) sums up my thoughts much more accurately.
Data can always be extracted by hand but it's obviously somewhat more time consuming and discourages mass abuse of data.
You'd think but isn't that exactly what Scoble is doing? I'm sure some of the people in his friends list didn't imagine he'd do this with their data.
Well, they can already do this NOW. They just have to do it "by hand" instead of using a script.
It isn't a question of whether it's possible to extract this information out of Facebook, so much as whether you can AUTOMATE the extraction.
Facebook has made extracting the information artificially inefficient. It's a kind of "regressive tax," if you will: the more friends you have, the more you use Facebook, the more onerous the task of extracting the information.
Namaste. Peace out. Free Scoble's Data. ;-)
I bet a huge portion of Facebook's members would take the 10 bucks, move to another network, buy some songs on iTunes with the money and never look back!
That's what venture capital will be used for.
$ 10 million gives you 1 million members in maybe as little as 30 days!
And let's say every member has 20 friends.
= 21 million members
Cost of one member: $ 0.48
Those members log in let's say 50 times a year and they look at 2 pages.
That's over 2 billion page views!
CPM : $ 15
That's 30 million a year!
And when you don't care about the profits in your first year on the market, well then, offer those 1 million members $ 30 !
Perhaps it's not that easy but I'm sure you get the picture of where this might end up going for Facebook.
** Scobleicious **
The press and attention Plaxo and Scoble are getting from his being Scobled (waxed, vaporized, disabled abruptly and without warning) is * scoble-icious!! *
:-)
He probably didn't mind because it could benefit him (and it did - lots of traffic to his site), but what if FB takes this to court?
Don't get me wrong, I applaud what he did, but at the very least Plaxo should take up the responsibility and pay Robert's bills in case it goes wrong.
I bet they didn't make him any promises whatsoever.
All this is bad publicity for Facebook, and it would get worse if they took it one step further, but the way I see it this ain't good news for Plaxo.
They support open ID which is good, but in a way they used Robert.
Now I know this ain't unusual in the corporate world, but if that's what they stand for then I guess it's time for some good alternatives!
How do you figure? When they (foolishly) give FB their Gmail password, they are giving FB permission to take their Gmail info.
It's bullshit to think that because I'm doing a thing one way instead of the other (oh, this HTTP GET is somehow different from the other HTTP GET) that I'm having some kind of adverse effect on the servers.
The fact that someone is my friend IS MY INFORMATION, just like the fact that I have someone in my gmail address book is a fact. I want to exploit these facts to my benefit in as open a way as possible. This means that i want to share this information with the other tools that I use.
Does anyone really care HOW I ACCOMPLISH THIS?
Facebook certainly was not able to detect Robert's use of this technology, unless his script presented itself with a new USER_AGENT, geekdave wonders if the USER_AGENT of the script was other than Mozilla (ie).
Even given that, I'd love to see the log analysis system that they have. I bet they wouldn't even detect the 1800 requests that were made from plaxo on behalf of Robert...
Scoble, I hope that FB bucks up, and gives you your contact information, and as much history as possible. I hope you then ditch facebook - I'll follow and go somewhere that's a bit more open and permissive.
Thanks, and best of luck.
@geekdave
perhaps a test account set up purely for that purpose or if they wanted to stress test it with larger numbers of members i'm certain that via his many friends etc he could have created a new profile just for that purpose ;-)
He also reviewed the T&C's before and noted that scripts weren't allowed yet chose to carry on. He was playing with fire to start with.
Get him his account back but be more careful next time ;-)
In instances like this a "sorry" probably wouldn't go amiss in appeasing the facebook owners. nothing wrong with being humble especially as he did run a script which is not allowed
testing on a live account and with live data.....
My Name and my DOB is registered with the local Birth, Deaths and Marriages Registrar and is available to anyone researching their genealogy.
My Email address is usually everywhere somehow and the ones that are really private have never been posted in public. ( its not that hard really )
Want to know where I have worked or where I might work? You can Google my name.
I got elected as a Parish Councillor. As part of that process I had to register my interests. In fact we expect public officials to make all their "interests" and financial dealings available to public scrutiny. It will be out there long after I leave my office.
But heres the real point. If this aggregation leads to a de - duplication of records in plaxo and facebook and a correction to current data then count me IN.
http://www.centernetworks.com/whose-data-is-it-...
I am (was?) one of your "Facebook Friends." (Though, if you have 5,000 of them, it's stretching the definition of "friendship" into "mild acquaintanceship.)
I am, therefore, given you permission to use MY data (not yours, MINE) on the Facebook site.
Personally, I'm not fussed whether you want to store it on the Facebook server, or on Plaxo Pulse, or for that matter chiseled into a brick on your house (along with 4,999 other bricks) as a piece of "installation art." [1]
That having been said, I can understand how some Facebook users might be happier in a walled garden, and therefore UNHAPPY that THEIR data got scraped out.
Mark in West Sussex, England [2]
[1] If you do this, and win a major art award, I want an invite to the prize ceremony :-)
[2] I think my location IS significant. We have a very different "social norm" of privacy information in Europe compared to the US. Many things that are common in the US would be criminal offences in the UK!
Had many girlfriends, Ed?
I kid, I kid! :-)
Also once that information is published elsewhere you no longer have the ability to restrict it’s access to only those friends that have seen it to date.
I guess I'm just puzzling over why Facebook should work any different than Real Life? You give your business card to someone, guess what, you "no longer have the ability to restrict" that information. It's gone. Free, as in buh-bye. You can't turn around later and ask for the business card back. You don't want someone to have the information, you don't trust them? Then don't give them your card, don't make them your friend.
You want to give your card to someone you don't trust? Then don't put any non-public information on your card. Leave off your birthday, SSN, Visa card number and underwear size.
People seem to be treating Facebook as some kind of system where 5,000 people can know your birthday one day, but if you change your mind later, you can "revoke" the information and make those 5,000 people forget when you were born. Alas, the toothpaste does NOT go back in the tube. Facebook is a bag what formerly held a cat.
You’d think but isn’t that exactly what Scoble is doing? I’m sure some of the people in his friends list didn’t imagine he’d do this with their data.
Do what, exactly? Make a copy of it?
"OMG I gave someone my phone number and they had the nerve to make a copy of it! Yeah! They actually wrote it down in TWO places! They typed it into Outlook AND wrote it down on a Post-It Note! Yeah! And the Post-It Note didn't even have a password!"
I don't know what Scoble is doing with his Facebook data, but presumably if I trusted him enough to give him my information, I trust him enough to not do something completely EVIL with it?
"Karim - No you can’t expect her to forget it but I think you can reasonably expect that she won’t publish it publically elsewhere for other people you don’t know to see it."
You obviously aren't familiar with the endless supply of "Pics of My Ex-Gf" porn sites around. :) Obviously, your "reasonable" expectation just doesn't hold water in reality (especially when talking about the demographic using FB and MySpace).
The short answer is this - users publishing sensitive information on social networks are still naive to think that their information is somehow "protected". The reality is that once it is published even in a limited public form one must assume that it can be leaked to those who you do not want to have this information. Users will greatly benefit from changing their thinking about what information they should readily make available to "friends".
Times 5000? I would have kicked your butt off, too.
Next time use one of the applications that seem to know how to do this right.
This isn't a facebook issue. This was about the application you used. But it sure got you in Techmeme, didn't it?
This got me to wondering what Plaxo would do with all of the information it harvests.
I haven't investigated Plaxo much, but does it let you get your information out of Plaxo? And what else does Plaxo do with the information?
These information ownership issues are getting VERY interesting.
No, but I've been happily married for a few years now. I'm hoping my wife won't plaster my email addresses and telephone numbers everywhere. ;-)
I see your points Karim but I think the issue isn't about whether or not your data will get distributed minimally (in manual form) by some users but that making it easy to bulk extract data makes it more likely it'll be abused.
I think in Scoble's case it's the fact that he bulk uploaded it to another service and I doubt that's something many would be happy with. They might be happy with Scoble's individual use of their data but can he ensure that Plaxo won't abuse that data.
There is a thingie you check off not to have Facebook Googleized though, and I wonder at what point that thing kicks in.
But all the other stuff of Facebook, the guts of Facebook, like the vampire bites, the videos, the graffiti on the walls, the books reviewed -- that shouldn't be able to be ported out because it's content that the author makes.
The copyright of a letter belongs to the author. Isn't it kind of like a letter?
So are we saying here that it's ok to individually upload and download stuff but not in batches of 5,000? Well, Scoble isn't going to spam people with Viagra ads in their various social media services but you can't expect everyone who tries to port that many names to be as scrupulous.
I still ignore Plaxo requests that come to me via email (I never got over their initial launch tactics), so having the information about myself that I entered into Facebook never making it's way into Plaxo is an important Facebook feature.
Sure, the faceborg might give you your account back but how is that fair to others who had accounts shut down for legitimate scraping?
I am not even talking about dapper.net (for the non programmer). All you have to do is watch the java and python mailing lists and irc channels to get a sense of the level of scraping going on. Its getting ridiculously easy to write complex scrap(p?)ers.
There is no such thing as private data on the internet...ask Bush if you don't believe me.
Um, 'cause 5,000 is a really big number to a computer? Facebook's TOS says users must not use the site in a way that could "damage, disable, overburden or impair the Site," but they don't tell you precisely what volume of HTTP GET requests over what period of time will cause the Turbo 386 under Zuckerberg's desk to burst into flames.
Instead of figuring out how to throttle excessive requests on their end, they'd rather tell their users they're not allowed to make "too many" requests -- whatever that is.
So are we saying here that it’s ok to individually upload and download stuff but not in batches of 5,000?
Apparently! There is a mysterious line, somewhere between viewing ONE page and viewing 5,000 pages that, once stepped over, results in your becoming a Facebook Unperson.
If I was a Marketer, based on the historical track record, once the belly-laugh Scoble comes at you with the shaky cams, offering free media coverage, don't bite, as it's not 'free', not in any sense of the word, it will cost you dearly in terms of the 'curse'.
A few points...
1. Your DATA? Who made it YOURS? If I become a "friend" that gives anyone (Plaxo/Scoble/Spammers) the right to cull my data? I think not. Facebook is right.
2. If you want a controllable PIM, use Outlook or ilk. Not some privacy-invading advertiser network, oh sorry "social software", that slaps a TOS on every move. Reap what sow.
3. Plaxo doesn't exactly have a good rep. here, having spammed me to death with 'update' requests in the past. Why should they need to match my Name, Email and Birthday with Facebook data anyways? Even if people have both accounts, maybe they don't want to be compatibility matched.
I hereby dub this Plaxo Scrapegate.
If users are that concerned about privacy, they should not join any social networks. Once your info is out there, it's like trying to get your pee out of a pool...
(hCard: http://microformats.org/wiki/hcard)
After all, if you can move your friends' addresses elsewhere, how are FB going to make money putting adverts in front of you?
[Aside: My Facebook profile has my DoB, mobile number and home address; I only add people as friends on FB if I'd be willing to give them that data in real life.]
Scoble scraping = good.
Beacon scraping = bad.
Why is it that the privacy people who crowed about Beacon aren't furious at Scoble?
If it wasn't that big of deal to take this, why not do an opt-in? How do I know Scoble won't lose my data? Sell it? lose it to someone who will sell it?
That's called "strategic" ... I call it "evil"...
Shelly from burningbird.net (commenter upthread), your website crashes safari, and firefox on a mac.
dont forget Scoble hiring you:
Coultergate.
http://cogdogblog.com/2007/11/18/stingybook/
Anyone can scrape away at my FB profile; they will find obvious lies on my birthdate (Apr 1), that I graduated in 1935 with a PhD from Harvard, and I was once a CEO of a large software company before that Bill guy came on board.
I don't think Irony has anything to do with it. What it is is unethical. Just about everything Microsoft vends works this way too, except in cases where extreme user pressure or legal pressure has been applied.
If users held the companies they used to any kind of ethical standard the world would be a better place, and companies such as Facebook would either change or go out of business. In Facebook;s case I hope it is the latter.
Can you email me more details? shelleyp@burningbird.ndet.
Oh, and Robert: don't enter that into Plaxo. Oh that's right: I'm not one of your friends. Not nice enough.
I was in the manual friending mode and got bumped . . . wasn't the point "making" friends?
http://carterfsmith.blogspot.com/2008/01/how-to...
Because there's a difference between Facebook, WITHOUT YOUR PERMISSION, informing all your friends that you just bought a crate of extra-strength Depends and rented a copy of "Brokeback Mountain" for the 12th time this year...
...and one of your friends making a copy of an email address YOU ALREADY ALLOWED HIM TO SEE.
I know, subtle difference, right?
If Google want to block Facebook scraping their site, let Google do so... and we can all make a decision whether to carry on using GMail.
If Facebook want to block others scraping their site, let Facebook do so... and we can all make a decision whether to carry on using Facebook.
You may not like Facebook's terms of service...
- You have a right not to like them.
- You have a right not to use Facebook.
- You have a right to wish that Facebook would go out of business if you want :-)
- What NO-ONE has a right to do is extract data out of it and give it out to a company I've chosen not to do business with.
I don't care whether the company is Amazon and the mechanism Beacon... or whether the company is Plaxo and the mechanism Facebook.
Also, why should i not be able to pull this information off, it is available to me anyway. if i had a million hours in a day I could just go through each friend and re-type all this information into my apple address book or wherever i want, why not just make it a little nicer and easier for me?
Good point, it was in the cards (I needed the $), but being Podtech, it went well beyond just the usual Scoble Curse. But water over the bridge now.
Debatable, but let's assume it is. Imagine how valuable that could be to some spammers, direct marketing firms, or any one else in the industry. Point is, it can be gathered very quickly, and then sold or passed on. Sure Scoble my have your email and birthday and could certainly manually move it to Outlook or Plaxo, but I rather doubt he would take the time to it with "5000 friends". The point here is, while I may have implicitly given Scoble this data, I didn't given him permission to do whatever he wants with it. I think there is an assumption, right or wrong, that this information will stay within the facebook garden and people won't be running tools to harvest it to be used or sold.
Shame on you
But if the application collected name, email address and birthday, is that really valuable personal information you don't want to give out? Then take it off your Facebook profile!
It would take most people about 30 seconds to find my name and email address on countless websites, including my business, and my blog. And despite the fact I tend to keep my birthday private, I'm fairly sure that it would be available on some sites I've forgotten.
If I've done that, then it's publicly available and out there, whether it's to five friends, or five thousand. It's down to you to decide what information your friends might scrape, or copy and paste into Outlook, and what you might wish to remain private.
After all, how do ya think Spam email works, for crying out loud?
If you think this is scary, I take it you haven't heard of Spokeo?
http://thewayoftheweb.blogspot.com/2007/12/spok...
Monitor all your friends, across lots of networks, without them knowing...
I don't understand if it is legal or not ? can we keep somewhere a list of profiles ?
With this http://developers.facebook.com/documentation.ph...
, the Plaxo process is not legal, no ? yes ?
Victor
The discussion here :
http://groups.google.fr/group/social-network-po...
I happen to use Plaxo,trust them etc, but imagine this pre-release software is compromised and another party accesses it?
I'm off to Vegas now baby!
I have now restricted access to my email address until the privacy filters down to the new friends lists functionality.
Keep up the good work but don't waste all your time on Facebook;)
jp
Scoble, I think this is fantastic. Not so much that the data were sent to such-and-such corp, but that (1) you were able to automate it and (2) that it woke people up to data security who are outside of IT by a long shot. It's precisely this kind of destruction of naiivete that is going to help push end users to learn what they're dealing with and how to protect themselves if they are uncomfortable with spreading their personal data all over.
Also, I don't think anyone really has 4,999 friends. I seriously doubt that anyone has more than 8 really close friends with whom they would feel totally comfortable sharing very personal feelings.
That would is both very derogatory and both very offensive to people who are mentally challenged.
I don't think that she knows what that word means or the full ramiforcations of it.
I don't think that Ms.Klein has been exposed to be who are mentally challenged.
If she was she wouldn't be making the disparaging remarks that she did.
Ms.Klein needs to show some empathy towards people who are relatively different than what she is.
please watch my site : http://www.brandrevitalize.com