whether YC will do something or not is beyond my concern, but as a general remark, could such feedback loop encourage applicants to fit into criteria to pass that particular system instead of their actual goals (building successful startup)?
The same remark can be applied to other fields; recruitment, record companies, auditions to cast a role, men/women relationships, .... what's common here is that, although you can tell success or failure retrospectively, you cannot nail down the systematic criteria beforehand.
Just to jump in here, there's also a measure of agility that is required. Having any sort of formalized system based on criteria assumes by its nature that those criteria are always going to be true. Anybody who's been watching the tech world for more than five years knows that's not true.
Not to mention the issues that come up once the system is changed. If the system is changed next year, and some companies that failed under the old system would pass under the new system, wouldn't that be a huge problem?
You're taking people at the top of their game, with knowledge in the industry and technology, and having them make a fine-tuned judgment call.All this for a dynamically changing business environment. If the decision is of a well-considered and thoughtful nature at all, that's going to be very hard to systematize. The more you formalize it the more you lose your special sauce. There are qualitative scorecard-type solutions, but those by nature are personal opinions and would have to remain proprietary.
The same remark can be applied to other fields; recruitment, record companies, auditions to cast a role, men/women relationships, .... what's common here is that, although you can tell success or failure retrospectively, you cannot nail down the systematic criteria beforehand.