All this stuff you mention are variables. Accuracy is a multivariable equation. That is, if you change one value, the other values may also need to change in order to get it back where you want it.
I always advise starting with ... the starter load for the round.
Velocity is a function of bullet mass. The heavier the bullet you use, the slower it will go. The slower it goes, the more it will drop due to gravity --- if 2 things go 100 yards, and one of them takes 10 seconds and one of them takes 5 seconds, the one going 10 seconds will fall much, much farther than the 5 second one because gravity is an acceleration! I prefer the lightest, fastest flying projectiles that are appropriate for my barrel. Bullet shape also matters but not as much on pistol as rifle. Still, aerodynamics and stability are crucial.
All that to say that for accuracy at long ranges in my 9mm Pistol (not your same gun, keep in mind!) I shoot a redline normal pressure (that is, not quite +P but close) 9mm with a 90 grain 380 projectile. Wife can make better groups with her pistol at 25 than you did with that rifle using whatever it was. Out of a rifle, those 90s would be pretty good consistency out to 100 yards easily. I can dig up the recipe if you want to try that, but I don't know what it would do in a rifle.
Also, remember the powder problem. Your tools have a "fixed" margin of error. That is, your scale is +- 0.1 grains or whatever, that is "fixed", for example. If you are using a mega hot uber powder that takes 3 grains for the load, that is 2.9 to 3.1 with the error thrown in. If you instead used a magnum powder that took 6 grains or something, that would be 5.9 to 6.1 with error. Which should tell you that your error is twice as good with the 6 grain powder load!
If you get really serious you can weigh each projectile before you use it, keeping only identicals. You would be amazed at the variation in cheap bullet weights.
