For me, it really depends on the balance between the voice actor, the character, and the story.
If it's a voice actor I genuinely love, I can tolerate a pretty terrible story and force myself to finish it just because I want to hear them. But honestly, that's probably a one-time deal. If the story is bad enough, I'm not going back for another release just because they're in it.
There are also times when I end up dropping something halfway through because I really don't vibe with the character. Even if I like the voice actor, there's only so much I can tolerate if the character themselves just isn't my type.
On the other hand, if the story is genuinely good, I can become interested in a voice actor I was previously pretty neutral about. A great character and well-written story can definitely earn them bonus points from me.
But there's one hard limit: if I genuinely dislike someone's voice, even an amazing story probably won't save it for me. I just can't stay immersed if I don't enjoy listening to the voice itself.
So I'd say it has to be balanced to some extent. A great voice actor can carry a mediocre story for me, and a great story can make me appreciate an unfamiliar voice actor, but there's a certain threshold where either one can become a dealbreaker.