Data as a Sufficient but Not Necessary Condition for Machine Learning
Abstract
For decades we believed that algorithms were the key to prediction. From the Perceptron to Support Vector Machines, from Gaussian Mixture Models to Random Forests, the competition was about algorithms — until Banko and Brill (2001) demonstrated empirically that "there is no data like more data": with enough data, consistent learners converge toward the same Bayes-optimal limit, so the choice of algorithm matters less and less. The hunt for data began, and it was believed that "whoever owns the data, owns the market." Now, however, foundation models show us that — at least in the language, speech, and computer vision domains — data is no longer even the most valuable asset. Drawing on my experiments performed for Mitra, Amazon's frontier tabular foundation model, this talk illuminates why data is only the tip of the iceberg — even for tabular data — and what future research directions this implies for the field of computer science.
Bio
Dr. Gerald Friedland is a Principal Scientist at AWS and adjunct professor at the University of California, Berkeley.
Dr. Friedland has published about 200 scientific publications, including two books and is a part-time Lecturer at the EECS department of the University of California, Berkeley. He is the recipient of several research and industry recognitions, among them the European Academic Software Award and the Multimedia Entrepreneur Award by the German Federal Department of Economics. He also led the team that won the ACM Multimedia Grand Challenge in 2009. Dr. Friedland received his doctorate (summa cum laude) and master’s degree in computer science from Freie Universitaet Berlin, Germany, in 2002 and 2006, respectively.
Gerald is also an avid software and hardware developer and is known for open-source contributions, such as SIOX (Simple Interactive Object Extraction), which has become the open-source standard algorithm for interactive image cut and paste used in applications like GIMP or Inkscape.