I don't like charging time to the client if I'm not working on a client. If I was paying someone per hour for a job, I would want them to charge me based on how much work they put in.
In the end of the day, I just roughly subtract some time for my breaks (like going out for a smoke) so everything is fair and square.
Still, most other white collar jobs usually have downtimes, something you barely get doing consulting/assurance. And to be honest, most of the time wasted is because the client is doing some shit so I don't feel like I have to spent every paid second worrying about their stuff.