Does the phrase; "You can't judge a book by its cover." seem absurd to you?

Absurd as in we use it more to describe people, then we do books... Cause when I'm on the hunt for a new book the cover most definitely is what gets my attention. Then the blurb on the back tells me if its worth a read. And the only time the cover isn't important is when I have already read other books by the author.