I'll try and tip toe around spoilers as much as possible, but having played the entire series thus far:
0 doesn't really reference the other Yakuza games in plot bar one little scene referencing Seijima and the flashback events that happened in 4 (basically 4 showed a flashback to Majima and Seijima's youth, and those scenes take place before 0). However this is explained fine in 0 so its not as if you'll be left scratching your head wondering what's going on.
Otherwise the references are relegated to characters that turn up. Some characters - like Nishiki and Dojima are main support characters, while other characters play smaller roles in the story such as (minor spoilers)
. However the game does throw a couple of nice curve balls in terms of a couple of unexpected cameo's (not saying who, but only fans of the series will get their significance in turning up). But these are mainly just cameo's in side stories, so while little Easter eggs like those will pass you by, unless you can play through 1-5 before touching 0 just to get a couple of little in jokes you'll be more than fine starting with 0.
Kiwami is mainly a straight up scene for scene remake of the original Yakuza with a few added scenes here and there (mainly with Nishiki, a couple with Majima etc.) HOWEVER there is one side mission plot that is a direct continuation from 0 (the Pocket Racer mini game) so its well worth playing 0 first, then Kiwami just for this little side story (its excellent).
Plus the way 0 ends, it basically feeds right into the beginning of Kiwami story wise, basically think of 0 as the "Rogue One" to Kiwami/Yakuza 1's "A New Hope".