I used to go to this place near where my brother lived on 7th Ave called Psychic Records (413B 7th Ave, Brooklyn, NY 11215). It's super tiny, but that made it really manageable to poke around. I don't think it's open Mondays. I found some good things over the years, and it'll scratch your itch, but don't go in with high expectations.
Needles & Grooves
blenoxsmith