Yann LeCun Quotes

[AI progres is very dependent on Moore's law.] The one thing that allowed big progress in computer vision with ConvNets is the availability of GPUs with performance over 1 Tflops.

The "ungovernability" (and openness) of Linux and the Internet is precisely what has made their success.
The same will be true of open weight AI foundation models.

[Large language models] require enormous amounts of data to reach a level of intelligence that is not that great in the end. And they can't really reason. They can't plan anything other than things they’ve been trained on. So they're not a road towards what people call “AGI.”

AI is going to bring a new renaissance for humanity, a new form of enlightenment, if you want, because AI is going to amplify everybody's intelligence.

Don't get fooled by people who claim to have a solution to Artificial General Intelligence, who claim to have AI systems that work "just like the human brain", or who claim to have figured out how the brain works (well, except if it's Geoff Hinton making the claim). Ask them what error rate they get on MNIST or ImageNet.

Many of the papers that make it passed the review process are [good but boring] papers that bring an improvement to a well-established technique... Truly innovative papers rarely make it, largely because reviewers are unlikely to understand the point or foresee the potential of it.

Every reasonable ML technique has some sort of mathematical guarantee. For example, neural nets have a finite VC dimension, hence they are consistent and have generalization bounds... every single bound is terrible and useless in practice. As long as your method minimizes some sort of objective function and has a finite capacity (or is properly regularized), you are on solid theoretical grounds.

Infrastructure wants to be open.
Foundation models are becoming an infrastructure and will inevitably become commoditized.
Long term, the money is in the application layer

[Can a commercial entity] produce Wikipedia? No. Wikipedia is crowdsourced because it works. So it's going to be the same for AI systems, they're going to have to be trained, or at least fine-tuned, with the help of everyone around the world. And people will only do this if they can contribute to a widely-available open platform.

The analogy I've been using is the fact that perhaps an equivalent event in the history of humanity to what might be provided by Generalization of AI assistant is the invention of the printing press. It made everybody smarter.

The direction of history is that the more data we get, the more our methods rely on learning. Ultimately, the task use learning end to end. That's what happened for speech, handwriting, and object recognition. It's bound to happen for NLP.

My problem with sticking too close to nature is that it's like "cargo-cult" science... I don't use neural nets because they look like the brain. I use them because they are a convenient way to construct parameterized non-linear functions with good properties. But I did get inspiration from the architecture of the visual cortex to build convolutional nets.

I try to stay away from all methods that require sampling. I must have an allergy of some sort. That said, I am neither Bayesian nor anti-Bayesian... I think Bayesian methods are really cool conceptually in some cases... but I really don't have much faith in things like non-parametric Bayesian methods...

The vast majority of human knowledge is not expressed in text... LLMs do not have that, because they don't have access to it. And so they can make really stupid mistakes. That’s where hallucinations come from.

I don't like to call [human intelligence] AGI because human intelligence is not general at all.