For me "big red button" is "Emergency Power Off", which I guess is also a viable response to some showstopper bugs, until you have time to fix, sometimes.
I used turned on heroku maintenance mode for a couple of apps I didn't care to patch right away, according to its description it prevents requests from reaching dynos so should be good for preventing access to existing data/stored keys in the meantime.
