Thanks so much for all the replies - they've been really helpful.
I was chatting to someone on IG who wears a medium (and it looks great on him). He is 6ft and 180lb. I am 5'10" and 170lb. So I figured a medium would be the best choice for me.
But now I'm a little worried there won't be room for layering. I don't intend to wear thick layers, but I'd definitely want room for an UHF with a t-shirt underneath.
I worry a large will be a bit 'baggy' though and I'm also concerned about the arm length (I have short arms).
It's a tough call!
It's worth considering that if the jacket fits too tight it will most likely give less warmth as the alpaca lining will be compressed. Mine fits me perfect to layer over an UHF but probably a bit loose over a t-shirt. It's worth considering that this is a very warm jacket so I'd say it's unlikely that you'll wear it with thin layers underneath. I'd also say that the jacket has a very flattering cut which means that it probably won't look off if it's a tad big.