Do you think it would be ethical to photocopy (or, to remove the expense of the copies from the equation, scan-in) an entire book that you borrowed from the library?
If I was going to read it only once, keep the book checked out for an appropriate amount of time, and not redistribute it, yes.
An audio book is more like a music album than [a video game or a movie]
Other than solely technologically, I vehemently disagree. The usage pattern of a book, either physical or audio, far more closely matches that of a movie or video game than a piece of music.
But once a publisher has chosen to publish it that way, your ethics are bound by their intentions.
Once a man has loaned you his slave, your ethics are bound to return the slave and not set him free.
Is that an extreme example? Yes. But it only takes one counterpoint to disprove a rule.