Brandon Duderstadt
2024
Tracking the perspectives of interacting language models
Hayden Helm
|
Brandon Duderstadt
|
Youngser Park
|
Carey Priebe
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Large language models (LLMs) are capable of producing high quality information at unprecedented rates. As these models continue to entrench themselves in society, the content they produce will become increasingly pervasive in databases that are, in turn, incorporated into the pre-training data, fine-tuning data, retrieval data, etc. of other language models. In this paper we formalize the idea of a communication network of LLMs and introduce a method for representing the perspective of individual models within a collection of LLMs. Given these tools we systematically study information diffusion in the communication network of LLMs in various simulated settings.
2023
GPT4All: An Ecosystem of Open Source Compressed Language Models
Yuvanesh Anand
|
Zach Nussbaum
|
Adam Treat
|
Aaron Miller
|
Richard Guo
|
Benjamin Schmidt
|
Brandon Duderstadt
|
Andriy Mulyar
Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023)
Large language models (LLMs) have recently achieved human-level performance on a range of professional and academic benchmarks.The accessibility of these models has lagged behind their performance.State-of-the-art LLMs require costly infrastructure; are only accessible via rate-limited, geo-locked, and censored web interfaces; and lack publicly available code and technical reports.In this paper, we tell the story of GPT4All, a popular open source repository that aims to democratize access to LLMs.We outline the technical details of the original GPT4All model family, as well as the evolution of the GPT4All project from a single model into a fully fledged open source ecosystem.It is our hope that this paper acts as both a technical overview of the original GPT4All models as well as a case study on the subsequent growth of the GPT4All open source ecosystem.
Towards Explainable and Accessible AI
Brandon Duderstadt
|
Yuvanesh Anand
Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023)
Large language models (LLMs) have recently achieved human-level performance on a range of professional and academic benchmarks. Unfortunately, the explainability and accessibility of these models has lagged behind their performance. State-of-the-art LLMs require costly infrastructure, are only accessible via rate-limited, geo-locked, and censored web interfaces, and lack publicly available code and technical reports. Moreover, the lack of tooling for understanding the massive datasets used to train and produced by LLMs presents a critical challenge for explainability research. This talk will be an overview of Nomic AI’s efforts to address these challenges through its two core initiatives: GPT4All and Atlas
Search
Co-authors
- Yuvanesh Anand 2
- Hayden Helm 1
- Youngser Park 1
- Carey Priebe 1
- Zach Nussbaum 1
- show all...