I had that until I was 12 and started regularly masturbating. Seems like that was indeed the fix:
Patients are advised to stop exacerbating masturbation techniques and are encouraged to masturbate by moving the foreskin up and down so as to mimic more closely the action of sexual intercourse. After giving this advice Beaugé noted not once did he have to recommend circumcision.
Well… warning NSFW penis, but Wikipedia.
I had that until I was 12 and started regularly masturbating. Seems like that was indeed the fix: