<p><b>SANTA CLARA</b>— <a href="https://www.servicenow.com/">ServiceNow</a>, <a href="http://www.huggingface.co/" target="_blank" rel="noopener">Hugging Face</a>, and <a href="https://www.nvidia.com/en-us/" target="_blank" rel="noopener">NVIDIA</a>, have announced the release of <a href="https://huggingface.co/bigcode/starcoder2-15b/" target="_blank" rel="noopener">StarCoder2</a>, a family of openâaccess large language models (LLMs) for code generation that sets new standards for performance, transparency, and costâeffectiveness.</p>
<p>StarCoder2 was developed by the <a href="https://www.bigcode-project.org/" target="_blank" rel="noopener">BigCode</a> community, stewarded by<a href="https://www.servicenow.com/"> ServiceNow</a>, the leading digital workflow company, and <a href="https://huggingface.co/" target="_blank" rel="noopener">Hugging Face</a>, the mostâused openâsource platform where the machine learning community collaborates on models, datasets and applications.</p>
<p>Trained on 619 programming languages, StarCoder2 can be further trained and embedded in enterprise applications to perform specialized tasks such as application source code generation, workflow generation, text summarization, and more. Developers can use its code completion, advanced code summarization, code snippets retrieval, and other capabilities to accelerate innovation and improve productivity.</p>
<p>StarCoder2 offers three model sizes: a 3 billionâparameter model trained by ServiceNow, a 7 billionâparameter model trained by Hugging Face, and a 15 billionâparameter model built by NVIDIA with NVIDIA NeMo and trained on NVIDIA accelerated infrastructure. The smaller variants provide powerful performance while saving on compute costs, as fewer parameters require less computing during inference. In fact, the new StarCoder2 3 billionâparameter model also matches the performance of the original StarCoder 15 billionâparameter model.</p>
<p>“StarCoder2 stands as a testament to the combined power of open scientific collaboration and responsible AI practices with an ethical data supply chain,” emphasized Harm de Vries, lead of ServiceNow&#8217;s StarCoder2 development team, and coâlead of BigCode. &#8220;The stateâofâtheâart openâaccess model improves on prior generative AI performance to increase developer productivity and provides developers equal access to the benefits of code generation AI, which in turn enables organizations of any size to more easily meet their full business potential.”</p>
<p>&#8220;The joint efforts led by Hugging Face, ServiceNow and NVIDIA enable the release of powerful base models that empower the community to build a wide range of applications more efficiently with full data and training transparency,&#8221; said Leandro von Werra, machine learning engineer at Hugging Face and coâlead of BigCode. “StarCoder2 is a testament to the potential of openâsource and open science as we work toward democratizing responsible AI.&#8221;</p>
<p>&#8220;Since every software ecosystem has a proprietary programming language, code LLMs can drive breakthroughs in efficiency and innovation in every industry,” said Jonathan Cohen, vice president of applied research at NVIDIA. “NVIDIA’s collaboration with ServiceNow and Hugging Face introduces secure, responsibly developed models, and supports broader access to accountable generative AI that we hope will benefit the global community.”</p>
<p><b>FineâTuning Advances Capabilities with BusinessâSpecific Data</b></p>
<p>StarCoder2 models share a stateâofâtheâart architecture and carefully curated data sources from BigCode that prioritize transparency and <a href="https://arxiv.org/abs/2312.03872" target="_blank" rel="noopener">open governance</a> to enable responsible innovation at scale.</p>
<p>The foundation of StarCoder2 is a new code dataset called <a href="https://huggingface.co/datasets/bigcode/the-stack-v2/" target="_blank" rel="noopener">The Stack v2</a> which is more than 7x larger than <a href="https://huggingface.co/datasets/bigcode/the-stack" target="_blank" rel="noopener">The Stack v1</a>. In addition to the advanced data set, new training techniques help the model understand lowâresource programming languages (such as COBOL), mathematics, and program source code discussions.</p>
<p>StarCoder2 advances the potential of future AIâdriven coding applications, including textâtoâcode and textâtoâworkflow capabilities. With broader, deeper programming training, it provides repository context, enabling accurate, contextâaware predictions. These advancements serve seasoned software engineers and citizen developers alike, accelerating business value and digital transformation.</p>
<p>Users can fineâtune the openâaccess models with industry or organizationâspecific data using openâsource tools such as NVIDIA NeMo or Hugging Face TRL.</p>
<p>Organizations have already fineâtuned the foundational StarCoder model to create specialized taskâspecific capabilities for their businesses.</p>
<p>ServiceNow’s textâtoâcode Now LLM was purposeâbuilt on a specialized version of the 15 billionâparameter StarCoder LLM, fineâtuned and trained for ServiceNow workflow patterns, useâcases, and processes. Hugging Face also used the model to create its StarChat assistant.</p>
<p><b>BigCode Fosters Open Scientific Collaboration in AI</b></p>
<p>BigCode represents an open scientific collaboration jointly led by Hugging Face and ServiceNow. Its mission centers on the responsible development of LLMs for code.</p>
<p>The BigCode community actively participated in the technical aspects of the StarCoder2 project through working groups and task forces, leveraging ServiceNow’s Fast LLM framework to train the 3 billionâparameter model, Hugging Face’s nanotron framework for the 7 billionâparameter model, and the endâtoâend NVIDIA NeMo cloudânative framework and <a href="https://developer.nvidia.com/blog/nvidia-tensorrt-llm-supercharges-large-language-model-inference-on-nvidia-h100-gpus/" target="_blank" rel="noopener">NVIDIA TensorRTâLLM</a> software to train and optimize the 15 billionâparameter model.</p>
<p>Fostering responsible innovation is at the core of BigCode’s purpose, demonstrated through its open governance, transparent supply chain, use of openâsource software, and the ability for developers to opt data out for training. StarCoder2 was built using responsibly sourced data under license from the digital commons of <a href="https://www.softwareheritage.org/2024/02/28/responsible-ai-with-starcoder2/" target="_blank" rel="noopener">Software Heritage</a>, hosted by <a href="https://www.inria.fr/en/inria-ecosystem" target="_blank" rel="noopener">Inria</a>.</p>
<p>“StarCoder2 is the first code generation AI model developed using the Software Heritage source code archive and built to align with our policies for responsible development of models for code,&#8221; stated Roberto Di Cosmo, Director at Software Heritage. &#8220;The collaboration of ServiceNow, Hugging Face, and NVIDIA exemplifies a shared commitment to ethical AI development, advancing technology for the greater good.&#8221;</p>
<p>StarCoder2, as with its predecessor, will be made available under the BigCode Open RAILâM license, allowing royaltyâfree access and use. Furthermore, the supporting code for the models resides on the BigCode project’s GitHub page.</p>
<p>All StarCoder2 models will also be <a href="https://huggingface.co/bigcode" target="_blank" rel="noopener">available for download</a> from Hugging Face and the StarCoder2 15B model is available on <a href="https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/starcoder2-15b" target="_blank" rel="noopener">NVIDIA AI Foundation models</a> for developers to experiment with directly from their browser, or through an API endpoint.</p>

MOUNTAIN VIEW - Intuit Inc., operator of the financial technology platform that makes Intuit TurboTax, Credit Karma, QuickBooks,…
The Santa Clara County District Attorney’s Office has charged a Sunnyvale apartment manager with possessing…
SAN FRANCISCO -- Wispr, the voice-to-text AI that turns speech into clear, polished writing in every…
SAN FRANCISCO -- Numeric, an AI accounting automation platform, has raised a $51 million Series…
Apple has announced 45 finalists for this year’s App Store Awards, recognizing the best apps…
The University of California (UC) and the California Nurses Association (CNA) have reached a tentative…