This reminds me of the countless conversation I had with people after a crisis. What can we do to prevent this from happening again? What process can we put in place? What restriction needed to be tightened up?

And that's how processes are born.

> And that's how processes are born.

Not necessarily. Processes are implemented by people, so they can break at any time.

The correct solution is more code, or less bad code.

And how do you get less bad code? Magic dust or process? My money would be on the latter, as in http://www.fastcompany.com/28121/they-write-right-stuff.

A newsfeed with ranking is a bit far off a space shuttle.

> So what do you do to avoid this in the future?

It's HN... there's no SLA, there's no postmortems, there's no doing things better in the future. pg just runs this site out of the good of his heart, we should be lucky the volunteers run it for us at all.

"pg just runs this site out of the good of his heart"

Don't think it's a "good of his heart situation". HN provides a benefit to YC and YC companies and attracts people to YC. As another example Fred Wilson has a very popular blog AVC and has said many times that he considers it "his secret weapon" (or something like that) because the value it provides over his competition.

> there's no postmortems

1. Press [Home] key.

2. Read postmortem.

3. ???

> It's HN... there's no SLA, there's no postmortems

I didn't mean to imply that there were. I was just curious.

> there's no doing things better in the future. pg just runs this site out of the good of his heart, we should be lucky the volunteers run it for us at all.

Since there are multiple volunteers, I think that the site always feels important to at least one of them. I imagine that some of them have gone more than a week without giving a shit about HN, but not all of them at once. So I think there is doing things better in the future. In fact, HN keeps getting improvements behind the scenes, to keep it running, keep it interesting, and keep it from getting overrun with trolls.

> pg just runs this site out of the good of his heart

Hilarious. I would have believed you if you appended "and his wallet"

One core. One HD. Bandwidth is trivial with no images. How much do you think this site costs to run?

He means that PG runs the sight because it makes business sense, not out of charity.

You didn't understand what I said. I was implying that this site is a money-maker for pg, not that it costs him money.

His time. Maybe the core and HD are Very Nice ones, too, of course.

How much do you think the time of YC partners is worth?

I don't think parent did mean that he had a "right" to expect some level of quality or anything.

It's just that we, as programmers, tend to take measures so that silly bugs do not happen anymore or that, at least, we leave big clues as to what went wrong.

In a project I had a similar issue: I was wrapping lists inside immutable lists but, due to a silly bug, I kept wrapping immutable lists inside immutable lists at every save made. So saved files would grow bigger and bigger.

And I did fix the bug and also added a big fat warning logs in case too many nested lists were detected.

pg might just as well have now added something preventing infinite recursion inside the comment tree or some WARN logging telling when a generate page is getting too big, etc.

I'd still find it very interesting to know what pg did, if any, to dodge / minimize / make it easier to determine if such an issue happens in the future.