You are not answering the question that was asked. ianl's post was directly in reference to web application development, not systems programming. Of course you wouldn't do that with PHP. That's like saying "yeah, it's a great knife, but it can't change my oil." Well...sure. Nobody's saying it can. But it does what we're actually talking about just fine, and I am skeptical of ianl's claims that there are missing pieces of the infrastructure for web development.
(BTW: Puppet not playing nicely with PHP is new to me as I use it regularly. And I don't think you get much with Mongrel that nginx/FPM don't do well enough.)
These outages are very rough. Clearly a lot of the Internet is building out on AWS, and not using multiple zones correctly in the first place. But AWS can have multi-zone problems too as we see here. Nobody is perfect.
But what people forget is: AWS has a world class team of engineers first fixing the problem, and second making sure it will never happen again. Same with Heroku, EngineYard, etc.
Host stuff on dedicated boxes racked up somewhere and you will not go down with everyone else. But my dedicated boxes on ServerBeach go down for the same reasons: hard drive failure, power outages, hurricanes, etc. And I don't have anyone to help me bring them back up, nor the interest or capacity to build out redundant services myself.
My Heroku apps are down, but I can rest easy knowing that they will bring them back up with out an action on my part.
The cloud might not be perfect but the baseline is already very good and should only get better. All without you changing your business applications. Economy of scale is what the cloud is about.
The cloud might not be perfect but the baseline is already very good and should only get better.
Do we have reason to believe that it will only get better? I think it's possible the complexity of the systems we are building and the traffic they encounter will outpace our ability to manage them. Not saying I think it's the most likely outcome, but I don't feel as confident as you.
Food for thought for sure. True, nothing can get better forever...
But do we believe in "economy of scale" for computer and Internet systems in this age? Google, Amazon, Facebook, etc. have already proven to me that they have enough human and financial capital to architect and run systems that show economies of scale.
It's a bit scary to think about what it will mean when this runs out, but for now I personally feel confident that things are getting much better, and will continue to do so.
There were a lot of great longer videos that were hosted on Google Video leading up to their acquisition of Youtube. Now, I won't argue that the majority of the videos there are worthwhile, but it seems to me that a company whose self-professed goal is to "organize the world's information" really shouldn't be in the business of deleting large swaths of it. Google isn't exactly hurting for storage capacity; what harm could come from rolling the videos hosted there over to Youtube? It certainly couldn't hurt the average quality of Youtube.
I'd hope that the company that runs Google Books, that saw the value in ReCaptcha, that has cached the majority of the public-facing internet, that has mapped most regions of the world, and that provides satellite images of most of its surface wouldn't erase that data simply because they'd like to reuse the harddrives.
I have to agree. Two weeks notice? WTF, Google. The evil meter is dancing.
If you want people to trust you with their data, you should demonstrate more responsibility. And just because the hosting's been "free", doesn't mean you haven't made your pound of flesh off the advertising (speaking generally, at least).
Here's a thought: Shove it through YouTube. Heck, just keep the data around and convert and cache when an item is hit (those who really want it will wait out / return after the conversion). I suppose you need to be able to tie back to Video accounts for ownership, DMCA, etc. Still, there should be a better solution. And a couple of weeks' notice simply isn't fair. What if someone's tied up? On vacation? They're hosed.
Put it this way: Demonstrating yourself to be a source of "public" data loss, is rather bad PR.
They gave notice years ago that they were phasing it out. You haven't been able to upload anything in years. Frankly, I'm surprised it took them this long.
Well, then, I guess that compensates somewhat. Still, I'd suggest announcing the hard date 3 or more months in advance (I'd prefer 6). People become busy and complacent. And people may not quickly realize that a work they value has been abandoned by its original poster.
On the upside, they are "allowing" downloads. I guess I should acknowledge that as a serious positive.
I may have become sensitized by years of management that had similar attitudes towards changes. Push, push, push... until suddenly, we're abandoning this, NOW.
Different settings, but not entirely dissimilar feelings evoked.
There might also be abandoned worthwhile content that we'll never get the chance to see ever again. I'm with those saying Google should transfer the videos automatically across to YouTube.
In all seriousness though, I thought google video had shut down years ago. It must have just been a phase in its shutdown, but I could have sworn it was already gone.
Edit: not a good comment for HN. More: it was a great service, and it had a good run. And mad respect for Google for running it well but shutting it down gracefully.
I don't consider 13 days notice to be graceful. Imagine someone going on holidays for two weeks and coming back to find out their beloved video is lost forever.
Google isn't obligated to give a fair amount of notice but it'd be a nice thing to do. Don't be evil.
The TechCrunch article says people have until May 13 to download the videos. You just can't stream them after April 29.
Still, this is a pretty bad job on Google's part IMHO. More notice would have been good. (What if people were traveling or were sick for a month or something?) Plus Google should have an option to automatically transfer videos to Youtube.
Can't understand why Google's messing this up so badly.
In terms of public relations (when it comes to the "unwashed masses"), Google has always been a very reactive company. This is at the core of some of their most intractable problems / failure to perform (e.g. "social").
I see your point, but who is uploading videos to Google then deleting the source? These services are for sharing and streaming, not archiving.
I don't work for Google but I do work on large scale data hosting. When you shut a service down you have multiple phases. Step 1, stop capturing new data. Google Video did this years ago and anyone truly invested in the service knew what was up back then. Step 2, pick a date to stop serving existing data. Step 3, wait for a very long time for more requests to get data back. It is crazy how and why these trickle in years after the fact. Step 4, delete data. I bet Google wont actually delete data for a very long time if ever.
Maybe step 2 is aggressive, but in my experience 90 days notice doesn't make a difference. Nobody does anything until the last few days if ever. You just have to draw a line in the sand.
It is very hard to shut a service down with any sort of usage, let alone Google scale usage. But very important to do from time to time. This is what I respect.
I saw the first word of your comment, and thought, "downvote"
Then I read the edit and thought, "downvote, then punch this guy in the mouth"
Hey, you know what's respectful? Moving the videos to youtube. Or emailing download links to their users. Or giving them more than thirteen days notice. What are you, a MBA? How can you possibly consider this a good decision?
from my understanding, S3 optimizes throughput but not latency. I'm not surprised that his performance serving from S3 is perfectly fine, but I think the "answer" is to skip straight to CloudFront. CloudFront can mirror directly from your appserver, or from S3, it doesn't matter.
update: did a little googling, and found only anecdotal evidence that latency to S3 is fine but not awesome:
OKCupid seems to do a way, way better job at data analysis than the freakonomics guys, who sadly see no need to be correct in the analyses used in their popular books.
I don't think you intended this as a backhanded compliment, but it riles me up to see levitt mentioned as anything other than a pop Econ sellout. Reading anything by either Levitt or Krugman is guaranteed to enrage me. You know they can do better, but they traded rigor for money and fame.
You know they can do better, but they traded rigor for money and fame.
This is a relatively uncommon viewpoint. I thank you for sharing it. One should automatically suspect the popular interpreters of any field of trading "rigor for money and fame." If you have any blog posts on this subject, I'd be curious for the URLs. The Freakonomics guys have a bit of popular recognition which could be leveraged for marketing an OkCupid book. That said, it's likely that OkCupid has enough popular recognition to leverage for book sales on its own.
Detailed discussion of blatant, blatant errors on their article refuting climate change. Makes you wonder if they knew their contrarian position was weak, and still put it in the book for controversy and the headlines it brings.
http://climateprogress.org/2009/10/12/superfreakonomics-erro...
Browser technology is on a tear recently, and terminals are ripe for innovation. For a newborn project, this is an impressive mashup.
The number of dismissive people in this thread shows how disruptive this idea is!