That thing is called free, which exists solely by the necessity of its own nature, and of which the action is determined by itself alone. On the othe… - Benedictus de Spinoza

According to the seventeenth-century way of thinking, an atheist was by definition a decadent. If there was no God (or, at least, no providential, rewarding-and-punishing God of the sort worshipped in all the traditional religions), the reasoning went, then everything is permitted. So a non-beliver would be expected to indulge in all manner of sensual stimulation... to lie, cheat, and steal...
Spinoza, according to all seventeenth-century interpreters, rejected all the traditional ideas about God; he was indesputably a heretic. Yet his manner of living was humble and apparently free of vice. Then, as now, the philosopher seemed a living oxymoron: he was an ascetic sensualist, a spiritual materialist, a sociable hermit, a secular saint. How could his life have been so good, the critics asked, when his philosophy was so bad?

The philosopher John Toland, in the early 18th century, insisted that the terms 'Spinozism' and 'pantheism' are synonymous. Toland says that "Moses was, to be sure, a Pantheist, or, if you please, in more current terms, a Spinosist", while Spinoza's pantheism was taken for granted by Moses Mendelssohn, Gotthold Lessing and Friedrich Jacobi, in their famous Pantheismusstreit of 1785. More recently, this interpretation also appears in both the scholarly literature and popular representations of Spinoza's thought. In the recently published Cambridge Dictionary of Philosophy we read that "Spinoza is the most distinguished pantheist in Western philosophy". But the problem with calling Spinoza a 'pantheist' is that pantheism is still a kind of theism.

...The materialist is primarily an observer; and he will probably be such in ethics also; that is, he will have no ethics, except the emotion produced upon him by the march of the world. If he is an esprit fort and really disinterested, he will love life; as we all love perfect vitality, or what strikes us as such, in gulls and porpoises. This, I think, is the ethical sentiment psychologically consonant with a vigorous materialism: sympathy with the movement of things, interest in the rising wave, delight at the foam it bursts into, before it sinks again. Nature does not distinguish the better from the worse, but the lover of nature does. He calls better what, being analogous to his own life, enhances his vitality and probably possesses some vitality of its own. This is the ethical feeling of Spinoza, the greatest of modern naturalists in philosophy; and we shall see how Lucretius, in spite of his fidelity to the ascetic Epicurus, is carried by his poetic ecstasy in the same direction.