I've run into that argument many times. It was crap the first time and it's still crap now.
Here's the difference; The waifus don't exist. They're fictional characters that you can never touch and can only ever see 'virtually'. In contrast, your (hypothetical) GF/wife does exist in the real world and you can interact with her directly so if she gets railed by the mailman, that's pretty fucked up. Ergo; It's not 'NTR' to watch "vanilla" because 'she' doesn't fucking exist so you never had her/literally cannot lose her since reality=/=fiction. If you were going to argue a default fetish in this context, VOYEURISM was right there. Pfeh.
Anyway "vanilla" content is fine, but I don't like it because I find it boring, lame & a bit reeking of desperation but hey, if that's your thing that's cool. Point being, you don't have to adhere to some max delulu cope reason and lie to yourself & others to explain why vanilla isn't popular. It's just bland, that's all.