You have opened a can of PITA! But, doable. Problem is I haven't used what looks like 2.79 in years, and can't show you with that version. I'm using 2.93. What you have is influence from arm bones. Most likely bicep. Your project is easier than a cape, so that's good.
What I do is import the garment as geom, and matching body geom. In Blender 2.8x and up, you can then import the rig to match the gender/age. Select all parts, and last the rig, then Object> Parent> Armature Deform. The body and garment are now attached to the rig. Select the rig and go to Pose mode. Select a shoulder, and rotate it. Watch your mesh distort. The goal is to get all of the bones to move the garment as YOU want... or as close as possible!
You might first simply delete all arm bones from the garment, since you appear to want this to drape the body, and NOT follow the arms, as with my cape.
When painting this type of garment, you can mask off just the area you want to paint. Use the gradient brush, and experiment with linear and radial, and weight, size of brush, and strength.
Normalize after painting, and remember your best friend, the undo button!!!
But again, you probably want to just delete arm bones that influence your garment.
NOTE: The gradient brushes will paint front and backfaces at the same time, so no guessing as when using the regular brush.
Shiny, happy people make me puke!