TLDR: The huge bill was a result of an improper way the application was coded. They contacted Google/Firebase who were gracious enough to waive off the bill.
You can, cloudwatch can shut things down on reaching threshold or scale-down.
With cloud you really have to optimize your code/architecture as you pay for what you use.
With your own servers that is not an issue unless you overload them, then you can have an outage.
EDIT: I thought Firebase is amazon for some reason, it is not. So in general Google panel doesn't have that functionality (from my experience with it), they have billing alerts but I never saw anything that would shut off things on reaching that alert.
You can limit by API requests usually but not by credit amount so if you use multitude of API's it can get tricky and they likely didn't limit anything.
That's very nice of them. However I'd have to read all of firebase's small print before i would consider using it.
Does their SLA guarantee data availability even if google dcide to spin it off or sunset the way they did with google+, reader and such?
Does it matter what their fine print says if you have to bang your head against a buggy automated bot process to get support things done? Switched over to MS appcenter recently(they added support for cosmosdb + authentication). Their customer care is so nice by comparison
Former Firebase PM: firebase.google.com/terms has all the details.
TL;DR: minimum 1 year deprecation policy, and you've got access to the data at any time (e.g. do a backup and get everything as JSON, or download an entire store bucket and transfer it to S3).
Glad to hear that Google has done something with it.
Also that's one of the reasons I try to use the realtime database, and not firestore. But they still charge for bandwidth there (most if it is consumed by downloading the SSL certificates from the clients).
This must be the first time I hear about Google responding timely and adequately to a specific problem of a customer. Could the publicity have something to do with it?
More likely because google responds timely and adequately to customers all the time but nobody tweets about it or writes a medium post that gets shared.
Their report reads like they had it coming big time:
> The app was running, all the supporters were able to support and the comments on social networks were that the app made it really simple to do support. We were very proud :)
> We didn’t want to release any new feature with that many users on the site, so we decided to merge a version with Angular V.6 […]. The site started to load slower, for some users it took them more than 30 seconds to load the page. That was weird. Our team was not comfortable with that and we couldn’t understand what was causing it and now we had our code with a completly new version of Angular, and probably many other bugs in production.
Am I reading this right: Their site was running well. They didn’t want to interrupt it by adding new features (potential bugs) so they casually update the framework and push the new version of their website to production hastily? And instead of rolling back the release they double down and optimize their code without knowing the source of the slowdown. Like WTF? Apparently they hadn’t even opened the browser’s network console to check if/what requests cause the slowdown. How did these people get $25K grant money?
Rgerding grant money post says: NXTP Labs acceleration program - so venture capital basically.
Looks like incompetence really, using a framework and unable to pinpoint the bottleneck and deciding that maybe upgrading version will somehow fix their bad code.
Not a completely pointless idea as framework might indeed have been changed enough to force them into using itself correctly, but still lot of questions.
I see. Thanks. Hadn't realised theres a link to the actual post-mortem. I thought that part was underlined for emphasis.
Really should've just linked to that hackernoon post though IMO, contains a lot more details thats very helpful in understanding what actually happened.
