The global drive to protect children online is fueling a surge in artificial intelligence-based tools aimed at keeping harmful content out of young users’ reach.
In the U.K., the recently enacted Online Safety Act requires tech firms to shield minors from age-inappropriate material, bullying, hate speech, fraud and child sexual abuse content. Companies that fail to comply face fines of up to 10% of their annual worldwide revenue. A similar effort is advancing in the United States, where lawmakers are pushing the Kids Online Safety Act, which would hold social media firms accountable for protecting young users.
These regulatory pushes have triggered a rethink across the tech industry. Pornhub and other adult sites are blocking all visitors who refuse to undergo age checks, while mainstream platforms including Spotify, Reddit and X have rolled out verification systems to prevent minors from accessing explicit or unsuitable material.
Much of this activity has created momentum for digital identification providers, with U.K.-based Yoti emerging as a key player. Its AI tool estimates users’ ages from facial features with a claimed accuracy of within two years for people aged 13 to 24. The company has already partnered with the U.K. Post Office and hopes to benefit further as governments consider broader digital ID systems. Other companies such as Entrust, Persona and iProov also compete in the space.
“There is a race on for child safety technology and service providers to earn trust and confidence,” said Pete Kenyon, partner at law firm Cripps. He added that new rules have created a fresh market where firms are rushing to gain a foothold. Still, privacy advocates warn of risks tied to data collection and potential breaches. “Substantial privacy issues arise with this technology being used,” Kenyon said, stressing that trust will depend on strong safeguards.
Rani Govender, policy manager for child safety at the NSPCC, argued that effective tools already exist without compromising privacy. “Tech companies must make deliberate, ethical choices by choosing solutions that protect children from harm without compromising the privacy of users,” she said.
Hardware makers are also entering the space. Earlier this month, Finnish manufacturer HMD Global unveiled the Fusion X1, a smartphone designed with AI filters that block children from creating, viewing or sharing explicit material across apps and cameras. The device incorporates technology from U.K.-based cybersecurity firm SafeToNet. “We believe more needs to be done in this space,” said James Robinson, HMD’s vice president for family products, while noting the concept predated the Online Safety Act.
The NSPCC’s Govender warned that tech giants like Google and Meta must place greater focus on safeguarding minors. “For years, tech giants have stood by while harmful and illegal content spread across their platforms, leaving young people exposed and vulnerable,” she said. “That era of neglect must end.”
Related Readings:
