The Financial Times (FT) reported on the 7th, citing multiple sources familiar with the matter, that ByteDance Ltd., the parent company of TikTok, is preparing a model on the scale of Anthropic's top-tier artificial intelligence (AI) model "Mitos."
According to the report, ByteDance Ltd. has begun pretraining a new AI model with 10 trillion parameters. That is about three times the size of Moonshot AI's "Kimi K3," which has 2.8 trillion parameters, and is the largest among Chinese AI models disclosed to date. Parameters are often likened to brain cells, and the larger the number, the smarter the model and the more complex thoughts and tasks it can handle.
Pretraining an AI model typically takes three to six months, after which, if all processes proceed smoothly, the model is released following fine-tuning. The exact model size will be determined during subsequent training.
Anthropic does not disclose parameter counts for its AI models, but the industry estimates the state-of-the-art "Mitos 5" at about 8 trillion and "Fable 5" at about 5 trillion parameters. Development is led by the 2,000-person AI research organization "Seed," headed by Wu Yonghui, formerly of Google DeepMind.
On ByteDance Ltd.'s push to develop an AI model comparable in scale to "Mitos 5," the FT said it "shows an ambition not only to catch up with American rivals but to surpass them in cutting-edge AI." Recently unveiled Chinese AI models from Moonshot AI and Alibaba have shown performance in various benchmarks comparable to the top models from OpenAI and Anthropic.
Unlike most Chinese AI corporations that focus on open-weight (public-weight) models, ByteDance Ltd. has developed closed models and has been aggressively expanding AI infrastructure investment accordingly. ByteDance Ltd.'s "S-Dance" model is regarded as one of the world's top models in video generation, and the consumer-facing "Doubao (豆包)" has 324 million monthly users, making it the most popular AI model in China.
According to sources, ByteDance Ltd. recently instructed staff not to use a training method that "distills" models developed by other AI corporations. The aim is to develop the model independently without referencing U.S. cutting-edge AI models. However, some analysis says that by sticking to this stance for over a year, ByteDance Ltd.'s development pace has slowed compared with rivals.
Zhang Yiming, the founder of ByteDance Ltd., asked staff to "not worry about falling behind in the short term and set a goal of securing 'world-class model capabilities' in the long term," according to the U.S. tech outlet The Information.