Anime used to be things made by nerds for nerds (like games). It blew up in Japan during the times where Japan's economy are projected to overtake USA. Those era is also during the space race between US and the Soviets. Hence why some of the earliest anime that became cult classics are in the science-fiction genre.
After the Japanese housing bubble popped and their economy crashed, the nation as a whole became a lot more conservative with their spending. This is the point where producing original shows becomes a "risky prospect" and the animation studios gravitate towards taking "safer" choices like adapting existing IPs. You know, like the current Hollywood and their endless sequels and adaptations. Now, keep this trend in mind.
Here's the thing. People is just a lot more stern back in the days (post-WW2 era). That's why the things they make and do, are generally also has a lot more depth, seriousness, and thought-provoking. Compared to nowadays (era of instant gratification) with all the internet brainrot, abysmally short attention span, and shitty reading comprehension. You can say that this is both a blessing and a curse of long and relatively stable peace.
In other words, because they're mainly doing adaptations of existing IPs, the quality of anime industry's storytelling as a whole, became tied to each generation's ability to write stories. Whether those stories themselves are good or bad. During this post-bubble economy, is when the industry start to focus on "guaranteed hits" that the "production comittee" (studio executives) already know will sell. Adapting existing acclaimed IPs from major manga publishers (things like Dragon Ball, City Hunter, etc). This is also the point where ecchi manga (fanservice-heavy stuff) become popular, to the frustration of many Japanese prudes who wanted it gone. But things are about to change yet again.
During what some considered to be the "Golden Age of Anime", adaptations of Visual Novel (VN) and Light Novel (LN) became highly popular. Things like Haruhi Suzumiya, Jun Maeda's touching stories, Steins; Gate, etc. entered the scene and became the dominant powerhouse. This is also the point where the industry gravitate towards adapting light novels.
Why? Again, for economic reasons AKA MONEY. By their nature, LN (a wall of text) is cheaper and faster to produce than manga (pictures with some texts). This is also the point where wish fulfillment shit like Mushoku Tensei start to appear and set up a bad example for the rest of the industry to follow. Keep in mind that the people who wrote these wish fulfillment shitshows, are the poors sods who experienced the economic crash I mentioned earlier.
These sad and angry group of people that can't find stable employment, are collectively called the "Lost Generation" by the Japanese society. They consumed a lot of pop culture stuff like manga, anime, games, etc. to stave off their depression and bleak future. Forming the template for stereotypical fat otaku that's into the "creepy idol stuff". The "losers of the society" that were scorned and looked down upon the their own society.
The wish fulfillment stories they write, merged with ecchi manga tropes (cause of course they're horny and can't get a girlfriend) and formed the template to all wish fulfillment slopfest we see today. However during this era, the industry is still mostly adapting good written stories (for the most part). This goes on for a while until things went from bad to worse.
After some time, anime blews up and becomes mainstream in the West, with modern classics like Attack on Titan and FMAB. Alas, a black sheep appears amongst them. One that's not only badly written on its own, but also dragged the whole industry to a much, much worse place. The poster child of all modern isekai shitty slopfest: Sword Art Online. Which only blew up because of a perfect storm of coincidences.
Popularity of LN adaptions is at an all-time high. The studio behind it (A-1 Pictures), is a powerhouse studio that's trying to do new things. SAO is popular amongst the Lost Generation otaku (because of the gaming theme). The emergence of Oculus Rift in the gaming scene and the false impression that VR games are within the "near future". In short, the "generational wave and excitement" coincides with the release of SAO, and the studio went hard with the production quality. The studio brings in stellar sound design, OST, voice acting, and animation that's relatively advanced for its time. All of it is commendable, except for the mediocre character writing of its source material. While you can turn a blind eye towards the first half of season 1 (Aincrad arc), the second half (Fairy Dance arc) is irredeemably and irrefutably horrendous. Worse, it starts off a bad trend that brought the decline of quality shows all over the whole industry. Decline that run down to the very rock bottom.
While it wasn't fully its own fault per se (more of a perfect shistorm of bad coincidences), what Sword Art Online represent, is a bad example where style matters over substance, and it is considered "acceptable enough". Alas, the cat is our of the bag, and the industry (as always) thirsts for any trends that will bring in MORE MONEY.
And that brings us to today, and the effects of the decay is as you've already seen. Many self-styled amateurs think that they can get away with garbage writing and strike gold. Ignoring the fact that they'll only generate bad publicity, and ruined their own future chances. Alas, their perception is somewhat justified because major LN publishers (like Kadokawa), keeps funding these slopfest. Traditional seasoned writers are semi-forced to incorporate isekai tropes by the publishers, lest they risk not getting their works published. The level of writing in the anime industry had plummeted, becoming lazy and stale. This is how we ended up with so many badly written stories with half-assed fanservice (some are even softcore porn). In short, I've seen hentai with better writing and it have uncensored action.
And that's why serious stuff goes out, and the power fantasy isekai slopfest for wankers / gooners is trending in. Thankfully, it (mostly) only decay shows in the romcom and fantasy genre (otaku's favorites), and we're seeing the resurgence of shows with good writing lately. The industry is slowly healing, albeit at a snail's pace.
To reiterate my words, it's a generational issue and monetary incentive. Thanks for coming to my TED Talk.
