GPT-5 and the arc of progress

<p>If you want a video version of this, check out the <a href="https://www.youtube.com/watch?v=oOtHtK7YK2g" target="_blank">last 20 minutes of the livestream reaction</a> (edit, fixed link) I did with Will Brown of Prime Intellect and Swyx of Smol AI & Latent Space.</p><p>GPT-5 was set up to fail on some of the narratives it was expected to satisfy. The two central themes it had to decide between were the AGI (or superintelligence) narrative that Sam Altman & co. have been using to fundraise and the fact that ChatGPT is one of the fastest-growing consumer technologies of all time. </p><p>To fulfill both, GPT-5 needed to be AGI while also being cheap enough to serve as the most-used AI system in the world. Business and technological realities made it inevitable that GPT-5’s primary impact would be to solidify OpenAI’s market position, even if it raises a lot of eyebrows for the long-term trajectory of AI.</p><p>The reactions online capture this as well. The OpenAI live streams have historically catered to AI insiders, but the product speaks entirely to a different audience. The people discussing this release on Twitter will be disappointed in a first reaction, but 99% of people using ChatGPT are going to be so happy about the upgrade. Confusingly enough, this includes many of the critics. GPT-5 is a good AI system. It’s right in line with best-in-class across pretty much every evaluation, while being cheap enough to serve the whole world. </p><p>OpenAI is largely fixing its product offering with an announcement that was hyped to be one of the biggest AI news cycles of the year. AI news being loud is defined by <em>narratives</em> being different more-so than technology being better. OpenAI releasing an open model again will likely be pinpointed as just as important a day for the arc of AI as the GPT-5 release. In many ways GPT-5 was set up to fail and that is very off-putting for those expecting maximum AI progress in the near term.</p><p>I’m not going to dwell on it, but oh boy, that was a messy release. GPT-5 being announced and rolled out like this is very odd. Countless <a href="https://x.com/connerdelights/status/1953503460768592236" target="_blank">plots were mislabeled</a>, live demos had bugs, and the <a href="https://x.com/captainbullshi/status/1953540488549863753?s=46" target="_blank">early rollout is doing some weird stuff</a>. This reinforces how OpenAI was torn about the release and backed into a corner with their messaging. They knew they needed to improve the experience with strong competition in the industry, but releasing GPT-5 needed to make a splash after how long they’ve waited (and already parked the GPT 4.5 name).</p><p>The core question we track in this post is: What does it mean for the next 6-18 months of AI progress if GPT-5 is just as good as all the best models out there, e.g., Claude Sonnet for coding or o3 for search, funneled into one, super cheap package? </p><p>If AGI was a real goal, the main factor on progress would be raw performance. GPT-5 shows that AI is on a somewhat more traditional technological path, where there isn’t one key factor, it is a mix of performance, price, product, and everything in between. </p><p><p>Interconnects is a reader-supported publication. Consider becoming a subscriber.</p></p><p>GPT-5’s performance</p><p>There are a few places that we can see that GPT-5 represents a solid step on the performance trend line, but nothing like a step change. First, on <a href="https://x.com/lmarena_ai/status/1953504958378356941" target="_blank">LMArena</a>, GPT-5 is fantastic, sweeping the board to #1 on all categories. The last model to claim #1 in pretty much every category was <a href="https://www.interconnects.ai/p/gemini-25-pro-googles-second-ai-chance" target="_blank">Gemini 2.5 Pro</a> — and that was the biggest step change in Elo since GPT-4 Turbo skyrocketed past the first Claude.</p><p>Second, GPT-5 is the top model on the <a href="https://x.com/ArtificialAnlys/status/1953507703105757293" target="_blank">ArtificialAnalysis</a> composite benchmark.</p><p>These two, LMArena & ArtificialAnalysis, represent two coarse evaluations — community vibes and raw benchmarks. Both of these can be gamed, but are still correlated with real-world use. You can also see in OpenAI’s shared results how much the <strong><em>smaller versions</em></strong> improve on the likes of GPT-4.1 mini and o4-mini.</p><p>In many ways, the march of progress on evals has felt slowed for a while because model releases are so frequent and each individual step is smaller. Lots of small steps make for big change. The overall trend line is still very positive, and multiple companies are filling in the shape of it. </p><p>My post on “<a href="https://www.interconnects.ai/p/summertime-outlook-o3s-novelty-coming" target="_blank">what comes next</a>” from earlier this summer all but called this type of release, where the numbers aren’t shocking but the real world use cases are great, becoming more common.</p><p>This is a different path for the industry and will take a different form of messaging than we’re used to. More releases are going to look like <a href="https://www.interconnects.ai/p/claude-4-and-anthropics-bet-on-code" target="_blank">Anthropic’s Claude 4</a>, where the benchmark gains are minor and the real world gains are a big step. There are plenty of more implications for policy, evaluation, and transparency that come with this. It is going to take much more nuance to understand if the pace of progress is continuing, especially as critics of AI are going to seize the opportunity of evaluations flatlining to say that AI is no longer working.</p><p>To say it succinctly: Abilities will develop more slowly than products.</p><p>The product overhang is being <em>extended</em> with each release. We’re still building untapped value with AI models and systems faster than we’re capturing it.</p><p>Another way to see this incremental push out in models or systems is through OpenAI’s update to the famous <a href="https://metr.org/" target="_blank">METR</a> plot of time to completion for humans of various tasks AI systems can solve 50% of the time. GPT-5 is leading, but also just in line with trends.</p><p>All of this is to say comprehensively that AI progress is very alive and well, as long as you don’t subscribe to the exponential takeoff in ability. Those arguments are very strained by this GPT-5 release.</p><p>Yes, AI progress on intelligence and “raw ability” is certainly going to continue at a solid pace for a long time, but how will this translate into recursive self-improvement?</p><p>GPT-5’s details</p><p>If you’re reading closely, you may have noticed that this post uses the word <em>system</em> instead of model. All of the leading chat systems have been adding more components onto them like safety checkers and so on, but this is the first one to use different architectures and weights for the primary generation of content across similar queries. GPT-5 is the first in what is to come, mostly to better balance cost and give better user experiences. From the system card:</p><p>GPT‑5 is a <strong>unified system</strong> with a smart and fast model that answers most questions, a deeper reasoning model for harder problems, and a real-time router that quickly decides which model to use based on conversation type, complexity, tool needs, and explicit intent (for example, if you say “think hard about this” in the prompt). The router is continuously trained on real signals, including when users switch models, preference rates for responses, and measured correctness, improving over time.</p><p>Along with this, they shipped many product improvements, such as how the model has a 400K context window in the API with <a href="https://x.com/jxmnop/status/1953511687190982919" target="_blank">great performance</a>, <a href="https://x.com/polynoamial/status/1953517966978322545" target="_blank">reduced hallucinations</a>, and <a href="https://x.com/OpenAI/status/1953534071772262511" target="_blank">new personalities</a>. </p><p>Primarily, I worry as a power user about the router. I sense that for now I’ll default to GPT-5 Thinking, and sometimes upgrade to Pro mode, while downgrading to standard GPT-5 only for benign queries (depending on its search behavior — if it is search-heavy like o3 without thinking, then it should still work well). </p><p>Thankfully, the thinking mode has a “get an early answer” button, so I don’t see any reason to start elsewhere. If I need an answer fast, I’ll get one. If not, I want the best responses possible.</p><p>As for prices, here’s a comparison. GPT-5’s top-level model is cheaper than Claude Sonnet and far better than any OpenAI model has been before at coding — one of the core details of this release. Matching Gemini Pro’s pricing when considering Google’s infrastructure advantage is a substantial accomplishment.</p><p>* <strong>OpenAI — GPT-5 (API sizes)</strong></p><p>* <strong>GPT-5:</strong> input <strong>$1.25</strong>, output <strong>$10.00</strong>. (<a href="https://openai.com/gpt-5/" target="_blank">OpenAI</a>)</p><p>* <strong>GPT-5 mini:</strong> input <strong>$0.25</strong>, output <strong>$2.00</strong>. (<a href="https://openai.com/gpt-5/" target="_blank">OpenAI</a>)</p><p>* <strong>GPT-5 nano:</strong> input <strong>$0.05</strong>, output <strong>$0.40</strong>. (<a href="https://openai.com/gpt-5/" target="_blank">OpenAI</a>)</p><p>* <strong>OpenAI — o3 (reasoning)</strong></p><p>* <strong>o3:</strong> input <strong>$2.00</strong>, output <strong>$8.00</strong>. (<a href="https://platform.openai.com/docs/models/o3?utm_source=chatgpt.com" target="_blank">OpenAI Platform</a>)</p><p>* <strong>o3-mini:</strong> input <strong>$1.10</strong>, output <strong>$4.40</strong>. (cached input <strong>$0.55</strong>) (<a href="https://platform.openai.com/docs/pricing?utm_source=chatgpt.com" target="_blank">OpenAI Platform</a>)</p><p>* <strong>Anthropic — Claude 4 family</strong></p><p>* <strong>Claude Sonnet 4:</strong> input <strong>$3.00</strong>, output <strong>$15.00</strong>. (<a href="https://www.anthropic.com/pricing" target="_blank">Anthropic</a>)</p><p>* <strong>Claude Opus 4.1:</strong> input <strong>$15.00</strong>, output <strong>$75.00</strong>. (<a href="https://www.anthropic.com/pricing" target="_blank">Anthropic</a>)</p><p>* <strong>Google — Gemini 2.5</strong></p><p>* <strong>Gemini 2.5 Pro:</strong> input <strong>$1.25</strong> (≤200k prompt) / <strong>$2.50</strong> (>200k); output <strong>$10.00</strong> (≤200k) / <strong>$15.00</strong> (>200k). (<a href="https://ai.google.dev/gemini-api/docs/pricing" target="_blank">Google AI for Developers</a>)</p><p>* <strong>Gemini 2.5 Flash:</strong> input <strong>$0.30</strong> (text/image/video) or <strong>$1.00</strong> (audio); output <strong>$2.50</strong> (includes thinking tokens). (<a href="https://ai.google.dev/gemini-api/docs/pricing" target="_blank">Google AI for Developers</a>)</p><p>* <strong>Gemini 2.5 Flash-Lite:</strong> input <strong>$0.10</strong> (text/image/video) or <strong>$0.30</strong> (audio); output <strong>$0.40</strong>. (<a href="https://ai.google.dev/gemini-api/docs/pricing" target="_blank">Google AI for Developers</a>)</p><p>Cheaper, thinking models that work well in applications are far more useful than scaling (as <a href="https://www.interconnects.ai/p/gpt-45-not-a-frontier-model" target="_blank">GPT-4.5 has shown us</a>).</p><p>GPT-5’s impact</p><p>It seems like most people in all walks of life are going to love this model — from AI researchers all the way to people who are learning of ChatGPT for the first time today. This is very in line with my expectations for how AI will proceed, as a long, steady march of progress. </p><p>The fact that the models are getting way cheaper rather than way more expensive definitely signals that we cannot just brute-force scale our way to much stronger systems. Scaling helps, but it is now one of many considerations, and all the laboratories are showing us that much bigger models have diminishing returns in value to customers. At the same time, models being cheaper could be just what we need for <a href="https://en.wikipedia.org/wiki/Jevons_paradox" target="_blank">Jevons paradox</a> to kick in and provide another boost in AI adoption.</p><p>Many people will claim that the GPT-5 release was a flop and the bubble will pop for AI. This is downstream of the industry generally making totally unrealistic promises. As someone whose core through-line when covering frontier models is tracking the pace of progress, I translate this as “AI capabilities on benchmarks will proceed a bit more slowly, but we aren’t reaching any clear walls in performance.” The AI performance hills we’re climbing up as an industry do put up some more resistance as the obvious low hanging fruit is gone, but we have the tools to overcome it consistently for the next 6 to 18 months. </p><p>For companies that have been fundraising on promises of AGI, such as Anthropic and OpenAI, closing the next rounds could be harder. Of course, this depends on whether the messaging of the rounds was a key part of the fundraising. </p><p>This fundraising inspires capital expenditures across the industry, e.g. TSMC developing the next node for NVIDIA to build new chips, and so on. The AGI narrative and the fundraising it has enabled have been good for the U.S. in terms of building out valuable, raw infrastructure. </p><p>This could be the beginning of the money train slowing down, but that’s very different from a derailment and a stock market crash. As raw infrastructure spend slows, there will be even more pressure to deliver valuable products to users. A key trend for 2025 has been many of those appearing — Deep Research and Claude Code being the paradigms that everyone has copied. </p><p>GPT-5 makes these applications better and makes it easier and cheaper for the next viral AI products to hit the market. I’m still excited for what is to come. </p><p>But first, I’m going to sign off and go play with GPT-5. It’s a good day to build something for the fun of it. As I use it more, I’ll have more to say.</p><p>Extra GPT-5 links</p><p>For more specifics on the model from people who got early access, I recommend <a href="https://marginalrevolution.com/marginalrevolution/2025/08/gpt-5-short-and-enthusiastic-review.html" target="_blank">Tyler Cowen</a>, <a href="https://every.to/vibe-check/gpt-5" target="_blank">Every.to</a>, or <a href="https://simonwillison.net/2025/Aug/7/gpt-5/" target="_blank">Simon Willison</a> (or Swyx soon, on <a href="https://substack.com/profile/89230629-latentspace" target="_blank">Latent.Space</a>).</p><p>Livestream link: <a href="https://openai.com/gpt-5/" target="_blank">https://openai.com/gpt-5/</a> Research blog post: <a href="https://openai.com/index/introducing-gpt-5/" target="_blank">https://openai.com/index/introducing-gpt-5/</a> Developer blog post: <a href="https://openai.com/index/introducing-gpt-5-for-developers" target="_blank">https://openai.com/index/introducing-gpt-5-for-developers</a> Enterprise blog post: <a href="https://openai.com/index/gpt-5-new-era-of-work" target="_blank">https://openai.com/index/gpt-5-new-era-of-work</a> GPT-5 landing page: <a href="https://openai.com/gpt-5/" target="_blank">https://openai.com/gpt-5/</a> System Card: <a href="https://openai.com/index/gpt-5-system-card/" target="_blank">https://openai.com/index/gpt-5-system-card/</a> Coding examples: <a href="https://openai.github.io/gpt-5-coding-examples/" target="_blank">https://openai.github.io/gpt-5-coding-examples/</a>What would you say if you could talk to a future OpenAI model <a href="https://progress.openai.com/" target="_blank">https://progress.openai.com/</a></p><p>Finally, I’ll plug again the video I did with Will Brown and Swyx:</p><p> Send me the most interesting things you find on GPT-5!</p> <br /><br />This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit <a href="https://www.interconnects.ai/subscribe?utm_medium=podcast&#38;utm_campaign=CTA_2">www.interconnects.ai/subscribe</a>