The Evolution from GPT-4 to GPT-5
The transition from GPT-4 to GPT-5 presents an intriguing opportunity to explore the evolution of artificial intelligence algorithms. In an era marked by increasing complexity and opacity within AI development, predicting the trajectory of these advancements becomes a formidable challenge. A practical approach involves focusing on key figures at OpenAI and examining research papers from leading laboratories to glean insights into what might lie ahead.
While Sam Altman’s promotional narratives may sometimes seem exaggerated, they offer valuable perspectives on the structured vision for AI’s future. This vision encompasses various capabilities, particularly in reasoning (including self-play, iterative cycles, trial and error, and System 2 thinking) and personalization (through techniques like retrieval-augmented generation and fine-tuning). The question remains: will GPT-5 bring this vision to fruition?
The Role of Agents in AI Development
A pivotal question surrounding GPT-5 is whether it will possess agent capabilities or remain a conventional language model, similar to its predecessors. This distinction is crucial for several reasons:
- The significance of agents in the realm of intelligence cannot be overstated.
- We know that a rudimentary version of this capability is achievable to some extent.
- OpenAI has been actively researching artificial intelligence agents.
To acquire tacit knowledge, humans engage in actions that require feedback loops, experimentation, tool usage, and a method for integrating these experiences with existing knowledge. This is akin to what AlphaZero accomplishes through targeted reasoning that transcends mere imitation learning. For an agent, reasoning serves as a means to an end, providing new explicit knowledge that AI agents can utilize to plan and act toward complex goals. This encapsulates the essence of intelligence and represents the ultimate form of AI.
This agent intelligence stands in stark contrast to current large language models (LLMs) like GPT-4, Claude 3, Gemini 1.5, and Llama 3, which struggle with executing plans. Early attempts at LLM-based agents, such as BabyAGI and AutoGPT, have demonstrated limitations in autonomy. Presently, the most advanced AI systems function as tools rather than fully autonomous agents.
The transition from AI tools to AI agents capable of reasoning, planning, and acting is a critical challenge. Can OpenAI bridge the gap between GPT-4, an AI tool, and GPT-5, a potential AI agent? Token Prediction Algorithms (TPAs), which encompass various modal models like DALL-E, Sora, and Voice Engine, may hold the key to achieving AI agent capabilities.
TPAs are exceptionally powerful, underpinning the entire landscape of modern generative AI. The premise is simple: a sufficiently capable TPA can develop intelligence. Models like GPT-4, Claude 3, Gemini 1.5, and Llama 3 are all TPAs. Even less conventional examples, such as Figure 01 (which processes video input to produce trajectory output) and Voyager (an AI player in Minecraft utilizing GPT-4), can be classified as TPAs. However, relying solely on TPAs may not be the optimal solution for every problem. For instance, DeepMind’s AlphaGo and AlphaZero are not TPAs; they represent a clever integration of reinforcement learning, search, and deep learning.
Enhancing Reasoning Capabilities
The anticipated advancements in GPT-5 may introduce unprecedented reasoning capabilities. Altman has suggested that GPT-5 will be generally smarter than its predecessors, indicating a significant enhancement in reasoning abilities. If human intelligence is distinguished from animal intelligence by our capacity for reasoning, then the ability to integrate existing knowledge with new information through logical rules (deduction or induction) is fundamental. This process enables us to construct mental models of the world and devise plans to achieve our goals.
AI companies have historically focused heavily on imitation learning, which involves training models on vast amounts of human-generated data from the internet. The theory posits that by exposing AI to centuries of human-created content, it will learn to reason like humans. However, this approach has proven insufficient.
Imitation learning has two significant limitations:
- Most knowledge available online is explicit (knowing what), while tacit knowledge (knowing how) cannot be effectively conveyed through language. For example, you may read an article but remain unaware of the numerous drafts that preceded it.
- Imitation is just one of many tools in the learning toolbox of human children. Children learn through experimentation, trial and error, and self-play, engaging with the world in diverse ways that update their knowledge and integrate new information.
The lack of these critical reasoning tools in LLMs poses a challenge. Yet, how did DeepMind’s AlphaGo Zero defeat AlphaGo 100-0 without using any human data? It achieved this by playing the game itself, employing a combination of deep reinforcement learning and search.
In addition to its robust trial-and-error mechanisms, AlphaGo Zero possesses a unique ability that even the most advanced LLMs lack today: the capacity to contemplate the next steps. This ability allows it to discern optimal choices by comparing and integrating new information with prior knowledge through search algorithms. Allocating computational resources based on the complexity of the problem is a skill humans naturally employ, a concept explored by Daniel Kahneman in Thinking, Fast and Slow as System 2 thinking. Researchers like Yoshua Bengio and Yann LeCun are actively working to endow AI with similar System 2 thinking capabilities.
Self-play, iterative cycles, trial and error, and System 2 thinking represent promising research avenues for narrowing the gap between AI and human reasoning. The existence of AI systems equipped with these capabilities, such as AlphaGo Zero, AlphaZero, and MuZero, contrasts sharply with the limitations of contemporary AI systems like GPT-4. The challenge lies in the complexity of the real world, which is far more intricate than a game board, characterized by incomplete information, poorly defined rules, and an almost infinite range of actions.
Bridging the gap between reasoning in game-playing AI and reasoning in real-world AI is the essence of ongoing research projects. Evidence suggests that OpenAI is particularly focused on transcending imitation learning by integrating the strengths of search and reinforcement learning with LLMs. This speculation leads to the concept of Q*.
To move beyond imitation learning, it is essential to combine it with search, self-play, and reinforcement learning. GPT-5 is expected to be a pure LLM with significantly enhanced reasoning capabilities, drawing from reinforcement learning models akin to Q*.
The Importance of Personalization
Personalization is crucial for fostering a closer relationship between users and AI, empowering users to shape their interactions. Currently, users cannot fully customize ChatGPT to become the assistant they envision. Techniques such as system prompts, fine-tuning, and retrieval-augmented generation (RAG) allow users to guide the chatbot’s behavior, but they fall short in terms of how well AI understands users and the extent of user control over AI interactions.
If AI companies wish to prevent users from migrating to open-source alternatives, they must find a satisfactory compromise that balances powerful functionality with privacy protection. Is there a viable middle ground between robust capabilities and safeguarding user data? As AI scales, cloud processing becomes essential, yet OpenAI has not yet made personalization a focal point for GPT-5. One reason is the anticipated size and computational intensity of the model, which complicates local processing and data privacy concerns—most businesses are reluctant to send sensitive data to OpenAI.
In addition to privacy and on-device processing, another factor poised to unlock new levels of personalization is the implementation of extensive context windows. Recent advancements have shown that reasoning on such large input prompts can be prohibitively costly, as the cost increases quadratically with each additional word. This phenomenon is known as the “quadratic attention bottleneck.” However, recent research from Google and Meta suggests that this bottleneck may no longer pose a significant barrier.
Scaling Laws and the Future of GPT-5
Model Size Trends
The trend in model size has seen significant growth from GPT to GPT-4. The progression is as follows:
- GPT (2018): 117 million parameters
- GPT-2 (2019): 1.5 billion parameters
- GPT-3 (2020): 175 billion parameters
- GPT-4 (2023): Estimated at 1.8 trillion parameters
As we look ahead to GPT-5, it is expected to continue utilizing the Mixture of Experts (MoE) architecture, enhancing performance and reasoning efficiency by activating different specialized models to process inputs.
Predictions for Parameter Count
While predictions suggest that GPT-5’s parameter count could range from 2 to 5 trillion, the exact number remains uncertain, influenced by various factors, including the size of the training dataset and available computational resources.
The Role of Training Data
According to Chinchilla scaling laws, larger models necessitate more training data to avoid underfitting and ensure improved performance. Data indicates that even with overfitting, models like Llama 3 can continue to learn and enhance performance as long as sufficient data is available.
GPT-4 was trained on approximately 12-13 trillion tokens, providing a reference point for GPT-5. If GPT-5’s model size is similar to GPT-4, OpenAI may need to substantially increase the data volume, potentially scaling up to 100 trillion tokens to achieve performance improvements.
Data Collection Strategies
OpenAI’s data collection strategies may include:
- Utilizing the Whisper model to transcribe YouTube videos, despite potential violations of YouTube’s terms of service.
- Leveraging synthetic data, which has become a common and necessary practice in the AI field, especially as the pool of available human-generated internet data diminishes.
Computational Resources
The number of GPUs available for training significantly influences model performance. More GPU resources enable the training of larger models on the same dataset and allow for more training iterations, enhancing performance until a plateau is reached.
OpenAI has access to thousands of H100 GPUs from Azure, providing ample floating-point operations (FLOP) for training the next generation of models, which is crucial for enhancing performance.
Optimizing MoE Architecture
OpenAI may have discovered further optimizations for the Mixture of Experts (MoE) architecture, allowing for the fitting of more parameters without increasing training or inference costs.
Alberto Romero’s Estimates for GPT-5 Scale
Assuming OpenAI uses 25,000 H100 GPUs to train GPT-5, as opposed to the 25,000 A100 GPUs suggested by some analysts. H100s are 2 to 4 times faster than A100s when training large language models, with similar costs. If GPT-5’s training period spans 4 to 6 months, its parameter scale could range from 7 to 11 trillion, significantly exceeding previous estimates.
Considering the existing parallel configuration’s ability to allocate model weights during inference, GPT-5’s parameter scale could reach 10 to 15 trillion, potentially ten times that of GPT-4. OpenAI may also choose to optimize the model for efficiency, making it more cost-effective.
As OpenAI continues to improve GPT-4, some of the newly available computational resources may be reallocated to enhance GPT-4’s efficiency or reduce costs, potentially even offering it for free as a replacement for GPT-3.5. This strategy could attract users who are aware of ChatGPT’s existence but are hesitant to pay or are unaware of the significant differences between the free version of 3.5 and the paid version of 4.
In conclusion, the journey from GPT-4 to GPT-5 encapsulates a fascinating exploration of the future of AI. As we anticipate the unveiling of GPT-5, the potential for enhanced reasoning capabilities, agent functionalities, and personalized user experiences promises to redefine our interaction with artificial intelligence. The evolution of AI continues to be a dynamic and transformative force, shaping the way we understand and engage with technology.
