I started at precalculus and worked my way back until I found a video that didn't assume I already knew something I don't already know.

Could be that you know a lot of maths at a higher level than that, but your education just has a lot of holes in it.

If only that were true. :) I only know cross multiplication and basic arithmetic, excepting long division.

We all start somewhere! One of the beauties of math is it’s hierarchical structure. Once you figure out where you’re at, it’s just a matter of progressing further up the hierarchy of abstraction (spoken by someone who has only made it to Calculus in the hierarchy)

I really want to make a site that expresses this like a skill tree you'd find in an RPG or in Civilisation.

Duolingo has something like this. I rather like it.

That was my experience. I took through calculus in high school, but that was a lot of years ago. I discovered my trig and logs were very weak, and started back at algebra 2 and worked my way forward. A lot of it was review, but a lot of it felt completely new to me.

I'm in the middle of a very similar experience.

I run into this every time I help my kids with their math homework- even for subjects I thought I had learned comprehensively their curriculum covers branches/approaches I've never seen before, down to elementary school work.