Event Date
Presented by: Dr. Hongyi Wang, Assistant Professor of Computer Science, Rutgers University
The recent surge in “open-weight” Large Language Models (LLMs), such as LLaMA, Qwen, and DeepSeek, provides diverse options for AI practitioners and researchers. However, most LLMs have only released partial artifacts, such as final model weights or inference code, while technical reports increasingly limit their scope to high-level design choices and superficial statistics. These limitations hinder progress in the field by reducing transparency in LLM training and forcing teams to rediscover critical details of the training process.
In this talk, I will introduce LLM360, an initiative to fully open-source LLMs, advocating for the availability of all training code, data, model checkpoints, and intermediate results to the community. The goal of LLM360 is to foster open and collaborative AI research by making the entire LLM training process transparent and reproducible for everyone. I will discuss in detail the LLMs we have pre-trained and are currently training from scratch, which achieve leading benchmark performance. Additionally, I will present our newest generation of fully open and transparent LLM pre-training datasets, TxT360, as well as the next generation of LLMs that are under development on top of TxT360.
Bio
Hongyi Wang is a tenure-track Assistant Professor in Fall 2025. He spent two years as a postdoctoral fellow at CMU, working with Prof. Eric Xing. His research focuses on large-scale machine learning algorithms and systems. He obtained his Ph.D. from the Department of Computer Sciences at the University of Wisconsin-Madison. Dr. Wang has received several accolades, including the Rising Stars Award from the Conference on Parsimony and Learning in 2024, the NAACL 2024 Best Demo Award runner-up, and the Baidu Best Paper Award at the SpicyFL workshop at NeurIPS 2020.