Yeah, the annoyance to others is a big thing. I've had success getting stuff that is quiet and sits in my pocket. If you search on Amazon for ADHD fidget toys for adults, you'll find quite a few options. The one that works the best for me, is a little brass cube that has gears on each side. You can spin the gears easily in your pocket without looking , and it is completely noiseless. I also use it during video meetings where I can hold it off camera.

Whatever you do, when you are in front of people, don't pick something that makes noise :-) it will annoy almost everyone, but very few people are willing to tell you.