ChatGPT Releases GPT-4o

<p>ChatGPT has released its newest model called GPT-4o&period;<&sol;p>&NewLine;<p>GPT-4o &lpar;&OpenCurlyDoubleQuote;o” for &OpenCurlyDoubleQuote;omni”&rpar; is a step towards much more natural human-computer interaction—it accepts as input any combination of text&comma; audio&comma; image&comma; and video and generates any combination of text&comma; audio&comma; and image outputs&period; It can respond to audio inputs in as little as 232 milliseconds&comma; with an average of 320 milliseconds&comma; which is similar to <a class&equals;"transition ease-curve-a duration-250 underline-offset-&lbrack;0&period;125rem&rsqb; underline decoration-gray-40 dark&colon;decoration-gray-60 hover&colon;decoration-copy-primary" href&equals;"https&colon;&sol;&sol;www&period;pnas&period;org&sol;doi&sol;10&period;1073&sol;pnas&period;0903616106" target&equals;"&lowbar;blank" rel&equals;"noopener noreferrer">human response time<span class&equals;"sr-only">&lpar;opens in a new window&rpar;<&sol;span><&sol;a> in a conversation&period; It matches GPT-4 Turbo performance on text in English and code&comma; with significant improvement on text in non-English languages&comma; while also being much faster and 50&percnt; cheaper in the API&period; GPT-4o is especially better at vision and audio understanding compared to existing models&period;<&sol;p>&NewLine;<p>Prior to GPT-4o&comma; you could use <a class&equals;"transition ease-curve-a duration-250 underline-offset-&lbrack;0&period;125rem&rsqb; underline decoration-gray-40 dark&colon;decoration-gray-60 hover&colon;decoration-copy-primary" href&equals;"https&colon;&sol;&sol;openai&period;com&sol;index&sol;chatgpt-can-now-see-hear-and-speak"><u>Voice Mode<&sol;u><&sol;a> to talk to ChatGPT with latencies of 2&period;8 seconds &lpar;GPT-3&period;5&rpar; and 5&period;4 seconds &lpar;GPT-4&rpar; on average&period; To achieve this&comma; Voice Mode is a pipeline of three separate models&colon; one simple model transcribes audio to text&comma; GPT-3&period;5 or GPT-4 takes in text and outputs text&comma; and a third simple model converts that text back to audio&period; This process means that the main source of intelligence&comma; GPT-4&comma; loses a lot of information—it can’t directly observe tone&comma; multiple speakers&comma; or background noises&comma; and it can’t output laughter&comma; singing&comma; or express emotion&period;<&sol;p>&NewLine;<p>With GPT-4o&comma; the company trained a single new model end-to-end across text&comma; vision&comma; and audio&comma; meaning that all inputs and outputs are processed by the same neural network&period; Because GPT-4o is our first model combining all of these modalities&comma; we are still just scratching the surface of exploring what the model can do and its limitations&period;<&sol;p>&NewLine;<p>GPT-4o’s text and image capabilities are starting to roll out now in ChatGPT&period; We are making GPT-4o available in the free tier&comma; and to Plus users with up to 5x higher message limits&period; We&&num;8217&semi;ll roll out a new version of Voice Mode with GPT-4o in alpha within ChatGPT Plus in the coming weeks&period;<&sol;p>&NewLine;<p>Developers can also now access GPT-4o in the API as a text and vision model&period; GPT-4o is 2x faster&comma; half the price&comma; and has 5x higher rate limits compared to GPT-4 Turbo&period; We plan to launch support for GPT-4o&&num;8217&semi;s new audio and video capabilities to a small group of trusted partners in the API in the coming weeks&period;<&sol;p>&NewLine;