I've been asked on various occasions, "Why don't we, as an origins of life community, get together, think what a killer experiment is, and then go and build a or something, where we go and do the experiment?" And the answer to that is... [W]e can't agree with each other about what experiment would you do? ...[I]t is intrinsically a lot more complex, precisely because it's a continuum. We don't know. We don't agree about what environment, we don't agree about what kind of chemistry or biochemistry. We can't join these things up, and so it seems to me a much healthier environment is to be deliberately multiple about it. Not to say, "Ok, this particular world view is going to dominate." I think we have to have multiple views until we know more.

We define Experimentation as the implementation of the Lean Startup methodology of testing assumptions and constantly experimenting with controlled risks.

You do an experiment because your own philosophy makes you want to know the result. It’s too hard, and life is too short, to spend your time doing something because someone else has said it’s important. You must feel the thing yourself…

From now on, everything you do in this book, and after, should be viewed as an experiment. This has been a profound shift for people who worry that “starting a business” is this big daunting thing. Experiments are supposed to fail. And should they fail, you just take what you’ve learned and try again a little bit differently.

Part Three, that part of formal scientific method called experimentation, is sometimes thought of by romantics as all of science itself because that’s the only part with much visual surface. They see lots of test tubes and bizarre equipment and people running around making discoveries. They do not see the experiment as part of a larger intellectual process and so they often confuse experiments with demonstrations, which look the same. A man conducting a gee-whiz science show with fifty thousand dollars’ worth of Frankenstein equipment is not doing anything scientific if he knows beforehand what the results of his efforts are going to be. A motorcycle mechanic, on the other hand, who honks the horn to see if the battery works is informally conducting a true scientific experiment. He is testing a hypothesis by putting the question to nature. The TV scientist who mutters sadly, “The experiment is a failure; we have failed to achieve what we had hoped for,” is suffering mainly from a bad scriptwriter. An experiment is never a failure solely because it fails to achieve predicted results. An experiment is a failure only when it also fails adequately to test the hypothesis in question, when the data it produces don’t prove anything one way or another.

Should philosophy guide experiments, or should experiments guide philosophy?

Just as scientific experimentation is informed by theory, startup experimentation is guided by the startup’s vision. The goal of every startup experiment is to discover how to build a sustainable business around that vision.

That then, that I wish for, as to systems, is this, that men, in the first place, would forbear to establish any theory, till they have consulted with (though not a fully competent number of experiments, such as may afford them all the phænomena to be explicated by that theory, yet) a considerable number of experiments, in proportion to the comprehensiveness of the theory to be erected on them. And, in the next place, I would have such kind of supestructures looked upon only as temporary ones; which though they may be preferred before any others, as being the least imperfect, or, if you please, the best in their kind that we yet have, yet are they not entirely to be acquiesced in, as absolutely perfect, or uncapable of improving alterations.

Some things may have been tried before their time, but if these things don't violate the laws of physics they are likely to prove possible the next time around. Engineering is a series of failures with an occasional success. At least the kind where you are really looking at new technology. You tend to try things. You try things that are extrapolations of what has happened before. A lot of them don't work. Occasionally, you hit one that does. That's the way we make progress. Failures are not something to be avoided. You want to have them happen as quickly as you can so you can make progress rapidly. But, I've known technical people who are very competent but who would avoid doing a critical experiment. They would kind of work around the problem and do the things where the results were relatively straightforward, but they hated to do that experiment that might tell if their whole approach was right or wrong. And these people were relatively nonproductive. Then I've known other people who weren't perhaps as bright as the ones that worked around the problem who jumped right at the heart of the matter. They turn out to be the most productive. So my single piece of advice is don't delay making the critical test. That's the one that will tell you if you are right or wrong and where to go next.

I have long believed that an experimentalist should not be unduly inhibited by theoretical untidyness. If he insists in having every last theoretical T crossed before he starts his research the chances are that he will never do a significant experiment. And the more significant and fundamental the experiment the more theoretical uncertainty may be tolerated. By contrast, the more important and difficult the experiment the more that experimental care is warranted. There is no point in attempting a half-hearted experiment with an inadequate apparatus.

It is known to all persons who are conversant in experimental philosophy, that there are many little attentions and precautions necessary to be observed in the conducting of experiments, which cannot well be described in words, but which it is needless to describe, since practice will necessarily suggest them; though, like all other arts in which the hands and fingers are made use of, it is only much practice that can enable a person to go through complex experiments, of this or any kind, with ease and readiness.

The system problem is essentially the problem of the limitation of analytical procedures in science. This used to be expressed by half-metaphysical statements, such as emergent evolution or ‘the whole is more than the sum of its parts,’ but has a clear operational meaning.

Experiment with crazy ideas, fail & iterate--it's a prerequisite to your success. Don't fear trying an experiment where you don't know the outcome, relish the idea of discovery.

Experimenters are the shocktroops of science… An experiment is a question which science poses to Nature, and a measurement is the recording of Nature’s answer. But before an experiment can be performed, it must be planned – the question to nature must be formulated before being posed. Before the result of a measurement can be used, it must be interpreted – Nature’s answer must be understood properly. These two tasks are those of theorists, who find himself always more and more dependent on the tools of abstract mathematics.