I’m your same height and about 145lbs and have the XS. Sometimes I feel I should’ve gotten the S, and here’s why:
• Front feels shorter than back when wearing, and it barely hits the pants waistline.
• The side hand warmers are too high, and it takes a bit getting used to.
• The chest pockets and inside pockets are very small. I can’t fit my hand inside the inside pocket, therefor it’s kind of useless to put anything small in there.
• The leather tag is sowed over the seam that goes across the back, therefore it’s a bit uncomfortable to wear and it looks a bit awkward when new.
The sleeves are a good length for me on the XS, and would probably be too long on the S though.
I’ve probably used my jacket 5 times in the 2 1/2 years I’ve owned it (Miami weather), therefore I might sell it and give the S a try instead.