Having success in the voiceover business doesn't mean you need 100 clients. Sure, you may end up working with 100 clients throughout your career, but you only need about a dozen clients coming back to you regularly to make a good living doing voiceovers.
I've been a voice actor for a long time and some clients have come and gone, but I've managed to keep a steady pool of about 12 clients who consistently use my voiceover services regularly. These are the 12 golden tickets. Nourish them, treat them like family and overdeliver on every project. These are the clients that your success will depend on. Of course, you'll get plenty of one-hit wonders, but this small pool of active clients will keep your motor running.