Golden Week is a week of different festivals, which usually means important things like banks etc end up being closed. Thats ok if you have enough cash or know someone that can help out. If you run out of money during that time, it can mean major headaches, so plan ahead if you fancy hitting that week. Also costs of flights will be more expensive at that time.
Fuji can be very strange when it comes to weather, hence being closed over the winter, since it can get brutal up there. From Spring onwards you can climb it but expect lots of people to be doing the same thing at peak time. Most don't climb it that much, so if you intend to get to the summit, only a few will make it.
April to August? Hrmm...How long did you plan on visiting for?