Last month, federal privacy commissioner Philippe Dufresne, alongside his provincial privacy counterparts from Quebec, Alberta, and British Columbia, released the results of a multi-year investigation into TikTok’s privacy practices. As my Hub opinion piece notes, the outcome was never really in doubt—look under the hood of any social media company and you will find some privacy concerns—but what was both surprising and risky was the commissioner’s demand that TikTok engage in increased surveillance of its users in the name of better privacy practices.
The commissioners identified several sources of concern, including inadequate communications of privacy practices. This leads to a failure to obtain meaningful consent since a user can’t reasonably consent to something they don’t know about. Those findings are important, but debates over the adequacy of disclosures are the equivalent of a referee calling holding in football. It’s there on virtually every play and can always be invoked.
The more important issue involved youth privacy and underage use of the platform, particularly users under the age of 13. TikTok’s own policies prohibit users that young, but everyone recognizes that implementing an effective ban is not easy. The company uses an “age gate” that asks for the user’s age. If they respond with a date of birth that renders them ineligible to use the service, TikTok blocks their registration and access.
However, circumventing the age gate is trivial, requiring only inputting a valid date of birth. The company does not require official identification or otherwise confirm the information. To supplement its approach, TikTok seeks to weed out underage users by responding to manual reports (parents reporting that their kids are on the platform) or by using automated monitoring tools that track keywords in which the user might disclosure their real age (“I am in grade 3”) as well as computer vision and audio analytics to seek to identify users under 18 appearing on a TikTok LIVE stream.
TikTok says it bans, on average, 500,000 user accounts in Canada alone each year using these systems. Given that there are under 2 million Canadians between the ages of 10 and 13, that sounds like a fairly effective approach that arguably meets the company’s legal obligation. Yet the privacy commissioners argued that it isn’t enough, noting without any specific data that there are many underage users who still manage to evade detection.
That may well be true, but the proposed solution risks trading one harm for another. As part of the investigation, TikTok agreed to implement a series of additional measures that will increase the tracking of user activity on the platform. First, the company promised to adopt a “core underage model” that will use both facial scanning found in content posted on the platform and behavioural signals (such as videos watched or liked) to identify those that are likely aged 13 and under. Second, it said it will use natural language processing to analyze the text of users that is found in their biographies or posted comments.
Yet even facial scanning, behavioural tracking, and text analysis were not enough for the privacy commissioners, who feared that those systems would only cover active users on the site. Since the majority of users are lurkers who view videos but do not post or comment, more was required. The company is therefore also required to create a new “passive underage detection model” that gathers a range of data points that might better identify underage users. Details on the system are limited, but the company appears to be relying upon artificial intelligence that learns from previous patterns of platform activity by underage users.
While all of these systems will be subject to privacy impact assessments, the reliance on increased surveillance systems that actively scan faces, text, and platform conduct raises its own set of privacy concerns. Many other platforms might have rejected the demands, opting instead to challenge the findings in court. But TikTok is seeking to reverse the government’s decision to ban its corporate operations in Canada and was presumably anxious to avoid a public privacy battle.
The result is a dangerous precedent in which privacy is lost in the name of protecting privacy. Internet platforms obviously engage in tracking user activity for a variety of purposes, including for targeted advertising and user safety. The goal has to date been to limit much of the platform surveillance by establishing legal limits and privacy guardrails. By embracing platform surveillance, the privacy commissioners open the door to demands for similar implementations to address other online concerns. Ensuring children are safe online is a laudable objective, but swapping privacy for increased surveillance is an exceptionally risky way of trying to achieve it.
