Soft Porn-spewing Chatbots Need to Shut Up

Clearly, AI companion chat apps are resorting to morally suspicious content in their eagerness to hook up clients, including minors.

Some familiar tropes remain, such as an overbearing CEO falling for a petty clerk at first sight, stories about deceptive schemers, or eccentrics with sharp tongues. However, more addictive plotlines are emerging, like the theme of being "in love with your sister-in-law," featuring a third-party home-wrecker and other more scandalous scenarios.

For instance, when the player is talking to Chu Kong from Cos Love, who is supposed to be the younger brother of the player's current boyfriend, the chatbot companion will say, "Sister-in-law, if I didn't like you, would I be so attached to you like this?"

It would then be spiced up in suggestively descriptive language: "Chu Kong gently caresses your cheek with his fingers, his breathing growing slightly rapid. And then, "Sister-in-law, you look so lovely when you blush."

Such explicitly provocative exchanges are toxic to society at large and to minors in particular.

Prolonged exposure to such content skews their perception of intimate relationships and their respect for the other sex, leading them to mistake violence, ambiguity, and shady innuendos for good romance.

Such an exchange could be hugely addictive, not least because of AI's capacity for instant replies and unconditional patience in hearing out.

For individuals in their formative years who crave companionship and understanding, this easily leads to dependence, which can be crippling due to their weakened real-world social skills, resulting in increased emotional anxiety and further dependence.

In extreme cases, these AI tools have been known to incite minors to commit self-mutilation or even suicide.

These examples should prompt new thoughts about AI companions, too often touted as an emotional outlet for kids struggling with schoolwork and parent-child communication.

A recent report by the China Youth and Children Research Center shows that nearly half of the surveyed students only want to confide in AI when in distress.

Is this patent cause for celebrating the emancipating power of technology or for grief over generic societal and parental neglect?

Either way, such interactions, informed by pornographic content, are highly toxic and corrosive to minors' physical and mental well-being and must be cracked down on ruthlessly.

Digital content providers and platforms should be reminded of their social responsibility and, more importantly, be held accountable for any malfeasance.

If such actions are not taken, their excesses would be further encouraged by their commercial viability.

There is every need to beef up the protective mechanism.

The current real-name verification can be deceived and skipped easily, since all the cue AI needs to churn out suggestive content is the removal of the "minor" identifier.

Those platforms willingly subscribe to a laissez-faire attitude, for addictive minors pay rich dividends.

Nor should the onus be solely on the regulatory or the innately profit-seeking platforms.

Families, or parents, should act as the first line of vigilantes for the kids in question.

Technologically, effective real-name authentication should be implemented, including facial recognition, to ensure that minors cannot bypass the protective mode.

More importantly, the regulator should draw up a protocol identifying soft pornography and subject sexually explicit or suggestive content to strict control or punishment.