First grow is tough, be proud you made it through and have something to show and sounds like you learned alot. If you get this fear in the future DO NOT worry. It is normal and fixable. Even if it gets tot he point where it can become systemic we can get you through to the end and help you through a bud washing process as well..
Been at this for a LONG time, guess you could say I'm old lol.. I dont know what your environment was beforehand but PM should not have really hit those buds but it happens..
Next time, there is always a next time if you grow long enough btw, h2o2 and parecatic acid are your bext friends while beating it down, keeping it at bay/eradicating it til harvest and a bud wash (you can find the basic recipe on here use the 3 bucket system as a norm and worse case a 2 bucket system h2o2 water rinse etc.. )If it rubbed off with your fingers like you touch it lightly its gone , its PM.. The spottiness you see usually indicates PM as it grows in clusters and spreads.. Trichomes pretty much grow in uniform.. But you should spray with a water /peroxide combo now.. wont hurt at all, wont effect taste at all, will kill all PM that is lingering.. youll see the next day the leaves are back to your normalness..
let me know if I can help and follow my stuff as I post .. lots of upgrades I have learned over the years.. Cannabis 101 ;)