AI Companies Innovate Training Techniques

(Reuters) – Artificial intelligence companies like OpenAI are looking to tackle unexpected delays and challenges in developing larger language models by adopting more human-like thinking techniques for algorithms.

A dozen AI scientists, researchers, and investors shared their insights with Reuters, highlighting that these innovative techniques, which underlie OpenAI's recently released o1 model, could significantly reshape the AI arms race. The implications extend to the resources AI companies continually demand, such as energy and specific types of chips.

OpenAI declined to provide a comment on the matter. Since the viral launch of the ChatGPT chatbot two years ago, tech companies have publicly insisted that scaling current models with added data and computing power is the way forward for creating improved AI.

However, leading AI scientists are raising concerns about the limitations of the “bigger is better” approach. Ilya Sutskever, co-founder of AI labs Safe Superintelligence (SSI) and OpenAI, informed Reuters that outcomes from scaling up pre-training—an essential stage where an AI model is trained on vast amounts of unlabeled data to grasp language patterns—have plateaued.

Sutskever, a key proponent of large-scale AI advancements through extensive data and computing power in pre-training that led to ChatGPT, recently departed OpenAI to establish SSI. He remarked, “The 2010s were the age of scaling; now we're back in the age of wonder and discovery. Everyone is looking for the next breakthrough. Scaling the right thing matters more now than ever.” Sutskever refrained from detailing his team's approach but confirmed that SSI is pursuing an alternative scaling method.

AI researchers are also encountering delays and less-than-ideal results as they strive to launch large language models that can outmatch OpenAI's nearly two-year-old GPT-4 model. Running extensive training runs can cost millions, and because of the complexities in the systems, hardware failures during these runs are common; researchers often do not assess model performance until the runs are completed, which could take months.

Another challenge is that large language models require enormous quantities of data, and many models have exhausted all readily available data. Power shortages have further complicated the training process, which demands significant energy.

To navigate these difficulties, researchers are investigating what is termed “test-time compute,” a method that enhances already existing AI models during the inference phase, where the model is actively used. Instead of selecting one answer instantly, a model could generate and evaluate several possibilities in real-time, ultimately selecting the most suitable option.

This approach empowers models to allocate more processing power toward complex tasks like math and coding problems or operations necessitating human-like reasoning and decision-making. As Noam Brown, an OpenAI researcher involved in o1, explained at a recent TED AI conference, a bot thinking critically for just 20 seconds in a poker hand displayed performance enhancements equivalent to scaling the model by an astronomical 100,000 times and training it for 100,000 times longer.

OpenAI has adopted this test-time compute technique in their newly released o1 model, previously known as Q* and Strawberry, first reported in July by Reuters. The o1 model mimics human-like thinking with its multi-step problem-solving approach and utilizes curated data and feedback from PhDs and industry experts. The unique aspect of the o1 series includes additional training imposed atop base models like GPT-4, and OpenAI plans to extend this method to larger base models soon.

Concurrently, researchers at other prominent AI laboratories, including Anthropic, xAI, and Google DeepMind, are also developing their iterations of this technique, according to several insiders.

“We identify a lot of immediate opportunities to enhance these models swiftly,” stated Kevin Weil, chief product officer at OpenAI, at a tech conference in October. “By the time others catch up, we aim to be three steps ahead.”

Responses to inquiries from Google and xAI were not available, and Anthropic had no immediate comment.

The implications of this shift could markedly transform the competitive landscape for AI hardware, predominantly characterized by skyrocketing demand for Nvidia’s AI chips. Top venture capital investors, from Sequoia to Andreessen Horowitz, who have invested billions in developing expensive AI models across multiple labs, are recognizing this transition and assessing its effects on their substantial investments.

“This change will transition us from giant pre-training clusters to inference clouds, which comprise distributed, cloud-based servers for inference,” remarked Sonya Huang, a partner at Sequoia Capital, in discussions with Reuters.

Nvidia's AI chips, known for their cutting-edge features, have propelled the company's growth, eclipsing Apple as the world’s most valuable company in October. Though Nvidia has a stronghold on training chips, competition may emerge in the inference sector.

Regarding potential impacts on product demand, Nvidia pointed to recent presentations emphasizing the significance of the techniques used in the o1 model. CEO Jensen Huang mentioned rising demands for the company’s chips used in inference.

“We've identified a second scaling law, specifically during inference… All these elements have culminated in a heightened demand for Blackwell,” Huang stated at a conference in India last month, referring to Nvidia’s latest AI chip.