Openish ML
Notes on open machine learning, mostly as a source for my openml.fyi newsletter
Openish ML
Prof. Emily M. Bender(she/her) (@emilymbender@dair-community.social)
dair-community.social
The rise of statistical NLP in the early 2000s was another such cycle. But major methodological shifts come with major cultural shifts as well. Statistical NLP introduced a culture laser-focused on benchmarks and saw the end of a small, “high trust” research community.
Part of a long thread on the culture of ML, which I think is critical to understand whether/how Open ML might succeed/fail.
My TED Talk is Out!
heathermeeker.com
Building Open Models Responsibly in the Gemini Era
opensource.googleblog.com
I appreciate the attention to detail here, and it’s generally a really careful, thoughtful post, but you then throw away pretty much all the good effort by settling on a term that the vast majority of people are not going to be able to tell apart from open source. “publicly available”, “free to use”… lots of other, non-confusing options right there.
When Less is More: Investigating Data Pruning for Pretraining LLMs at Scale
arxiv.org
Data Futures Lab - Mozilla Foundation
foundation.mozilla.org
Data Refusal From Below: A Framework for Understanding, Evaluating, and Envisioning Refusal as Design | ACM Journal on Responsible Computing
Jonathan Zongdl.acm.org
Today Google released Gemini with a 60-page report in which they repeatedly say the training data is key ("We find that data quality is critical to a highly-performing model"), while providing almost no information about how it was made, how it was filtered, or its contents.