you have to look at it this way.
during the major plot points they will likely be around, as you said they have a vested interest in those, but during the lighter less major story focused episodes are they really needed?
the show only runs 11 minutes. you already typically have the 4 main characters in the episodes, the simple fact is that those characters alone can typically fill up the 11 minutes the show is. adding two more characters can result in good things, but at the same time it can easily result in being just to much of a good thing and having the episode fall flat. We have already know of one time peridot was originally planned to have more of a presence in a episode but then was cut just because they couldn't fit everything with her there as well.
beyond all of that it's not like voice actor/actresses are free. Getting a large cast each episode tends to cost more money. once we get back to the main plot I am sure they will show up again, but I wouldn't be surprised if we got 5-10 more episode of the week type things going on here and if they only showed up once or twice during them.
