Probably Pink Moon, his last. Five Leaves Left, his debut, is my favourite and has a lot more instrumentation. Pink Moon is sparse. Not a fan of Bryter Layter for the most part.
i, too, prefer five leaves left. and since he only has three records, you probably can't go wrong starting with any one of them. pink moon is the most popular, though.
They are all excellent. i would go chronological order. The first two are more orchestral, compared to the stripped down demo feel of Pink Moon. Then go to Time of No Reply (first odds and sods album). Do Made To Love Magic very last as some of the different mixes are less than stellar.
Five Leaves Left - 2nd
Bryter Later - 3rd
That’s the order that worked for me and that’s also how I rank his albums