Scobleizer - Latest Comments in Help, I’m clueless about Web Service scalability

Re: Help, I’m clueless about Web Service scalability

frederic sidler — Wed, 08 Oct 2008 17:35:46 -0000

Hosting
Which provider would they choose today if they would need to built a new service that could face a scalability problem.

Limitation/Marketing
Is an invitation procedure a good way to manage the number of users accessing the platform. Would it frustrate the people not authorized to access the system or would it force them to find alternative ways to beta test it ;-)

Re: Help, I’m clueless about Web Service scalability

robw — Tue, 07 Oct 2008 15:53:45 -0000

A good conversation starter would be to ask them about which dimensions they found to be the most challenging to scale along.

Software/system architecture is only one aspect of successfully building a scalable system. Operations, development/deployment processes, monitoring and so on all make a big different to the scalability of any system.

Re: Help, I’m clueless about Web Service scalability

Kent Langley — Tue, 07 Oct 2008 15:46:51 -0000

My Question:

How have you changed the way you develop software to improve scalability?

Re: Help, I’m clueless about Web Service scalability

matt m — Tue, 07 Oct 2008 15:46:09 -0000

How much consideration was given to scalability issues in your initial design?

What decisions, from your initial design, presented the largest hurdles to scaling?

What issues will you ensure are taken into consideration in your next version 0 design?

Re: Help, I’m clueless about Web Service scalability

sidereal — Tue, 07 Oct 2008 15:03:34 -0000

1) At what point in the lifecycle of your product do you start to focus on high scalability. How willing should you be to compromise performance or development time to reach it before it's strictly necessary?

2) Is true horizontal scalability ever achievable or will there always be some shared resources?

3) Is designing for scalability a reactive or proactive process?

4) Are we too eager to abandon the benefits of normalization to achieve scalability?

5) Traffic patterns, especially for smaller sites, can be extremely volatile and unpredictable. How to you design for that?

6) What's the time horizon on scalability becoming a commodity that's provided by hosting providers alongside power and ethernet?

Re: Help, I’m clueless about Web Service scalability

Ted Murphy — Tue, 07 Oct 2008 14:39:14 -0000

Ask them for one scary example in their past where they screwed up and the app hit the wall. And ask them how they fixed it.

Re: Help, I’m clueless about Web Service scalability

Niall Kennedy — Tue, 07 Oct 2008 14:06:58 -0000

What are your current bottlenecks? If you could install a new piece of software or rack a new piece of hardware tomorrow to solve key pain points, what would it do?

Re: Help, I’m clueless about Web Service scalability

jonas — Tue, 07 Oct 2008 13:27:29 -0000

( http://highscalability.com/... )

Re: Help, I’m clueless about Web Service scalability

jonas — Tue, 07 Oct 2008 13:26:45 -0000

follow this blog if you want to know more :)

http://highscalability.com/

Re: Help, I’m clueless about Web Service scalability

Pete Austin — Tue, 07 Oct 2008 12:31:52 -0000

You need to decide how scalable you want it to be, because the architecture gets trickier with each power of 10 and the bugs subtler. You can give up on getting any sensible advice on the InterWeb for a start. With massive systems, the principle is to design your service as a cluster of sites which are as independent as possible in normal use, but which provide disaster-recovery for each other. The bottlenecks are in things like assigning a user to a site and passing messages between sites.

Re: Help, I’m clueless about Web Service scalability

rtwomey — Tue, 07 Oct 2008 10:05:02 -0000

@paul: One easy thing you can do is high-availability failover where a separate server takes over the IP address (and other characteristics) of your failed server. Check out www.linux-ha.org.

Re: Help, I’m clueless about Web Service scalability

paul — Tue, 07 Oct 2008 06:31:02 -0000

How do you provide guaranteed failover when ultimately your service has one IP address, cached inside the world's network of DNS servers? Anyone know, by the way?

Re: Help, I’m clueless about Web Service scalability

Matt M. — Mon, 06 Oct 2008 21:02:40 -0000

?: It seems like a lot of problems with scalability start with the current Database Server architecture. Should a new startup looking to scale start with an alternate? Are there production ready open source or commercially supported alternatives? What would it take for an alternate to be considered "safe" like a SQL Database Server?

Re: Help, I’m clueless about Web Service scalability

Stephen Pierzchala — Mon, 06 Oct 2008 17:31:10 -0000

Everyone has the internal component covered. I come at it from the outside.

How do you measure and monitor the external performance of your Web API to proactively deal with loading, connectivity, and application issues?

What is the issue you run up against most: bandwidth bottlenecks or application loads?

smp

Re: Help, I’m clueless about Web Service scalability

Geoffrey Wiseman — Mon, 06 Oct 2008 17:19:33 -0000

There's a few areas I'd want to cover:
- Where does scalability cross paths with standards (WS-*, WSDL, UDDI) vs. simplicity (REST, POX+HTTP).
- Do these people think that ESB has a place in their view of scalable services?
- How should you look at databases differently in a service-oriented model? (There's all sorts of sub-topics here; do services share a database, or have their own, does it vary? 2PC vs. other kinds of synchronization? Clustered caches vs. RDBMS?)
- What mechanisms are important to keep instances in synch and sharing work without tripping over each other or creating new bottlenecks (messaging, database, clustered caches, etc.)?
- Are location-transparency/routing and directory/discovery services important to scalable SOA, or is this simply the job for DNS and load balancers?

There's tons of interesting things to talk about here, really.

Re: Help, I’m clueless about Web Service scalability

alistairc — Mon, 06 Oct 2008 15:48:01 -0000

Robert,

Services are fascinating, for a couple of reasons. Sure, there are technical "service scaling" issues, which the other folks on the panel will know all about. Matt has great stories on the scaling of Akismet. But far more interesting are the human scaling issues. I always find more thoughtful discussions there.

First, there are terms-of-use issues. When you get to a certain size, you need to have policies for appropriate use. You wind up creating competitors, and an ecosystem emerges around your service. Look at Twitter, and the emergence of complementary products like Summize. Then look at what happened when Notchup exploited Linkedin to grab a bunch of users. How you monitor use and enforce terms of use is a big question, and it goes far beyond simple APIs and scaing.

Second, there's the fact that we're building human APIs. APIs and web services are typically focused on letting machines talk to one another. But by tying realtime activity feeds to our mobile devices, or location-based services that report our coordinates, we're plugging humans into applications, Amazon Turk style. As humans start to interact with applications via web services, through mobile devices and so on, a whole new set of scaling issues emerge.

Since you're the higher-level, human-angle participant on the panel, I'd elevate things beyond bits and bytes and into humans, policies, exploitation, and startup ecosystems.

Not sure that helps... sounds like it'll be a great panel!

Re: Help, I’m clueless about Web Service scalability

malbrecht — Mon, 06 Oct 2008 15:24:07 -0000

I have had my share of building scalable systems in the past -- and the one thing that always came back to me was the question: is the chosen application architecture adequate to support the needs?

There are some basic topologies that everybody uses:

a) "the sink" uses the database as repository of every message -- write it first, let it be read second; parallelize/cluster the db server and you easily crank up the volume. This model is very popular amongst web 2.0 systems. It is limited though, as adding a db server only brings 0.75 more power.

b) "the network" uses pipes to distribute messages between writers and readers -- very popular in the telco industry, where speed is master and geography plays an important role. Parallelize writers and readers and you get a linear scale that depends alone on the number of servers you put in. Big disadvantage is that distributed persistence needs to be consolidated at some point -- can generate some difficult to tame data flows.

Now, ask yourself: which model was chosen by google? and which by twitter?

Re: Help, I’m clueless about Web Service scalability

Tim Shisler — Mon, 06 Oct 2008 15:20:52 -0000

Very true Robert. It was more of a gut reaction that I should've thought more about. Thanks for replying.

Re: Help, I’m clueless about Web Service scalability

Robert Scoble — Mon, 06 Oct 2008 14:11:57 -0000

Tim: >>How can you complain about folks not being prepared to send in a job resume and turn around the next day and say you can go into something underprepared and be okay about it?

Who is going in unprepared. Seems this blog post just prepared me in a BIG way for Thursday!

Re: Help, I’m clueless about Web Service scalability

svetainiu kurimas — Mon, 06 Oct 2008 13:33:47 -0000

Thanks for post like this!!!

Re: Help, I’m clueless about Web Service scalability

Guest — Mon, 06 Oct 2008 12:41:34 -0000

Lots of the questions posed above are rather direct. If your looking to get a conversation going amongst them, I'd ask questions which encourage some "give-and-take" rather than give direction to their answers. You want to hear what these very smart dudes think is most important/relevant about the topic. Being modest, I'd start by assuming they would ask better questions of each other than I could ask of them - after all, we are interviewing them because of their successful expertise.

Questions:

"Individually looking at each others past projects like iLike and Friendfeed, what items in their development do you think were key to the projects success? In a similar vein, what would you have done differently and how would it have made things better?"

"In hindsight, were there key players beneath the surface in these projects which played a large part in making them a success, or was the project so well defined that everyone came together equally in making it a success? If there were key people, what did they provide to the project that you couldn't provide personally?"

Re: Help, I’m clueless about Web Service scalability

Molly Scofield — Mon, 06 Oct 2008 12:22:27 -0000

Great points raised by all. I'd also love to know, in the face of rapid scale, what success looks like to these guys.

Fast and big can be great (and usually commands attention) but may or may not be a good indicator of success. Likewise, how do they measure success and then revise/redeploy on the fly in response to the data.

Re: Help, I’m clueless about Web Service scalability

Tim Shisler — Mon, 06 Oct 2008 12:06:22 -0000

How can you complain about folks not being prepared to send in a job resume and turn around the next day and say you can go into something underprepared and be okay about it?

I know there is a stark difference, but really, come on. If you're allowed to ask help, why not give folks the benefit of the doubt if they flub a resume and ask them why?

Re: Help, I’m clueless about Web Service scalability

Girish — Mon, 06 Oct 2008 11:37:56 -0000

For someone that is serious about rejecting job applications if they have spelling errors, you sure don't care as much about your own blog concerning the same issue now, do ya? ;)

Admonishion? You mean admonition?

Re: Help, I’m clueless about Web Service scalability

Robert Sanchez — Mon, 06 Oct 2008 11:09:13 -0000

Hi Robert,

Sounds like an amazingly interesting webinar. I guess the question I would ask is, how exactly do you test your efficiency and scalability before hand, so that you can be modestly prepared for that overnight 6 million user count?

Also, will this be available after the conference? I registered on Fast Company, but I will have to working as the conference is live. Thank you Scobleizer