This is as far as I have got today. Random size OK. I would like to randomize proportion as well, will have a go tomorrow.
Edit 2: I got rid of the Max wiggle and left X wiggle only.
Books.blend (117.3 KB)
Edit 3!
A side note that is not so obvious, apart from the node setup to get the bottom of books flat on the shelves and the back flat on the wall (if you set the x wiggle to 0 ) I moved the origin of the book object. The instances scale from that origin.