I think musicians generally write music for their audiences to listen to, and authors generally write books for their audiences to read. I'm not saying there's anything wrong with reading sheet music or listening to audio books, but I'm just saying, that in most cases, that was probably not the intent of the creator of said work.
The difference between reading music and listening to music is similar to the difference between reading the word "red" and looking at the color red.
The difference between reading a book and listening to a book is similar to the difference between reading the word "red" and hearing the word "red."