Why is forgetting important? Things can either have an end time where they are no longer applicable or things we thought were true turn out to be false but it's still useful to see where we went wrong.

I imagine humans are limited by the # of synapses we have so it's useful to forget but maybe machines can move the useless stuff to deep storage until it's dug out, in the same way certain things can trigger a deep memory in humans.

Which is cheaper, among these two logically equivalent things: reducing one weight, or increasing every other weight?

Do you remove dead code? Get rid of clutter? Ever try to change a habit?

Yeah, I delete dead code but it's important to remember why I wrote it that way in the first place and why I'm deleting it now. Doomed to repeat past mistakes and all that.