I love the expression “food for thought” as a concrete, mysterious cognitive capability humans experience but LLMs have no equivalent for. Definitio… - Andrej Karpathy

" "

I love the expression “food for thought” as a concrete, mysterious cognitive capability humans experience but LLMs have no equivalent for.

Definition: “something worth thinking about or considering, like a mental meal that nourishes your mind with ideas, insights, or issues that require deeper reflection. It's used for topics that challenge your perspective, offer new understanding, or make you ponder important questions, acting as intellectual stimulation.”

So in LLM speak it’s a sequence of tokens such that when used as prompt for chain of thought, the samples are rewarding to attend over, via some yet undiscovered intrinsic reward function. Obsessed with what form it takes. Food for thought.

English
Collect this quote
All quotes from Andrej Karpathy Similar Quotes by other Originators
Go Premium

Support Quotewise while enjoying an ad-free experience and premium features.

View Plans

Additional quotes by Andrej Karpathy

I don't know what labs are doing to these poor LLMs during RL but they are mortally terrified of exceptions, in any infinitesimally likely case. Exceptions are a normal part of life and healthy dev process. Sign my LLM welfare petition for improved rewards in cases of exceptions.
Sources available for members. Sign up free to see where this quote comes from
Find Similar
The future expands the variance of human condition a lot more than it drags its mean. This is an empirical observation with interesting extrapolations.

The past is well-approximated as a population of farmers, living similar lives w.r.t. upbringing, knowledge, activities, ideals, aspirations, etc.

The future trends to include all of:
- the transhumanists who "ascend" with neuralinks etc., and the Amish living ~19th century life.
- those who "worship" ideals of religion, technology, knowledge, wealth, fitness, community, nature, art, ...
- those exploring externally into the stars, those exploring internally into minds (drugs++), or those who disappear into digital VR worlds
- those who date a different partner every day and those who are monogamous for life
- those who travel broadly and those who stay in one location their entire life
- those in megacities and those off-the-grid

For almost any question about a dimension of human condition, the answer trends not to any specific thing but to "all of the above". And to an extreme diversity of memetics. At least, this feels like the outcome in free societies that trend to abundance. I don't know what it feels like to live in such a society but it's interesting to think about.
Sources available for members. Sign up free to see where this quote comes from
Find Similar