It’s not a stretch to say that every tech company and their mother launched chatbots powered by generative artificial intelligence this year. Take for example Google’s Bard, Anthropic’s Claude, Microsoft’s Copilot, and, of course, OpenAI’s ChatGPT. These chatbots are based on incredibly powerful large language models (LLM) and, in the short amount of time they’ve been accessible to the public, they’ve improved tremendously.

Such rapid refinement speaks to the power of LLMs and generative AI—but it’s also a testament to the behind-the-scenes work of hundreds of thousands of gig workers. These humans do a little-known job called data annotation, which entails rating and describing the inputs and outputs of AI models to help the chatbot “learn.” These tasks could involve reading and ranking AI-generated poetry, or labeling a menu’s text with what words correspond to food items, drinks, and prices.

Historically, companies have exported this labor overseas and paid workers pennies. But recently, they’ve been outsourcing data annotation work to a different and unexpected demographic—and it could lead to some unforeseen and even dangerous consequences.

First, there’s some Gen-Z slang you need to know: girlies are young, online, mainly female-identifying users—and Jackie Mitchell is definitely one of the girlies. As she chats with The Daily Beast on a video call, she sports a flawless deep red manicure that complements her auburn hair, with two loose strands tastefully framing her face. She’s wearing a sleeveless, cream-colored turtleneck and dangling pearl earrings—in other words, she looks like the opposite of a person who you’d assume trains AI chatbots.

“I really did not ever anticipate doing anything like this, but I’m really into side hustles,” Mitchell told The Daily Beast.

When she was a college student, she wanted a bit of money to pay for date nights, but with school limiting her free time she didn’t have a lot of options for “traditional” jobs. So she turned to remote, on-demand work that she could do on her own time to bring in extra cash. She began taking short 10-minute surveys for money. One of these surveys turned out to be a data annotation project with a message at the end advertising work on a site called Data Annotation Tech.

That was two years ago. She’s since graduated and found a full-time job at a nonprofit—but still manages to work in data annotation as one of many side hustles. Lately, she’s been trying to make an extra $100 a day for 100 days to afford the down payment on her first house; since Mitchell works full-time, she’s doubled down on remote gig work to meet her daily benchmark.

In September, Mitchell started posting “day in the life” videos to TikTok describing her side hustles—including data annotation.

While she doesn’t mention the site she uses by name, data annotation makes an appearance in dozens of Mitchell’s videos in the series—including one post that has been viewed more than 4.4 million times. Dozens of users have commented on these videos, saying they’ve signed up for data annotation or asking questions about the work. Meanwhile, videos with the hashtag #dataannotation (which Mitchell does not use) have another 4.6 million views on TikTok.

Brin, another creator on the platform, said that the so-called “Jackie Mitchell effect” is indeed real: One of Mitchell’s videos popped up on Brin’s “For You” page and inspired her to take an unpaid assessment through Data Annotation Tech.

“I watched it and I was like, hell yeah,” Brin, whose last name is omitted to protect her privacy, told The Daily Beast. “She’s doing what she needs to do to make things happen—I want to do that. And I think I went into the assessment five minutes after watching that video.”

Mitchell and Brin signed confidentiality agreements with Data Annotation Tech, so they can’t disclose any of the chatbots or projects they’ve essentially helped train. But they’re working with some of the biggest names in generative AI: Data Annotation Tech all appear to be owned by the same company called Surge AI, whose clients include OpenAI, Google, Microsoft, Meta, and Anthropic.

So, that means if you’ve used ChatGPT, Bard, or Bing in the past year to help you write your essays, answer questions, or even craft jokes, you’ve likely been chatting with a bot that’s been trained by a small army of TikTok girlies.

Doing It For the Girls
Mitchell said that she mainly completes projects where she writes, edits, or fact-checks a chatbot’s responses. She might have to compare two poems written by a chatbot to determine which of the two best answers a specific prompt, or take a response about the best seafood restaurants in Maryland and make sure the bot isn’t “hallucinating,” or fabricating parts of its response or making inaccurate statements.

There are other types of projects on the site, too, involving skills like computer coding and language translation. In theory, the bots can learn from human data annotation and craft responses that are more accurate, helpful, and safer.

In practice, data annotation can feel rote and monotonous, Brin said. “It’s nothing incredibly fulfilling or anything really complicated for that matter. It’s just trading your time for money—and sometimes you need that.”