This post is also a part of a forum discussion.
Fred Hoyle commented that it makes more sense to be unorthodox than orthodox in science. I fully agree with that. If a thousand people are looking in one direction and you are looking in another, more will be found in the orthodox direction but you have a much higher chance of discovering something yourself.
The reason why scientists and wannabes goes “oh no, not again” whenever a pseudo-scientific idea crops up is that there are so many of them, the idea is so nebulous, and the proponents are almost incapable of admitting to themselves that they might be wrong no matter the argument or method. This reaction is perfectly natural but in excess it is damaging. It doesn’t help that to successfully promote a scientific theory you have to be pretty thick-skulled, and that doesn’t go well with admitting defeat. Like other human beings scientist tend to cling to their convictions until they die.
I would claim that the crackpot has more of the scientific nature than the hangers-on that use science as a religion, defending science from people daring to but its body of work in doubt. Not for nothing does science use terms like theory and model, while terms like law and proof have been quietly shown the back door. However following the crackpot is hardly the road to success either. That the majority is always wrong does not imply that the minority is right. One of the roles of science is as myth-maker. Why are we here? Where do we come from? Where do we go? This is a strong crackpot attractor, but science also has a practical and pragmatic role. Will this work? What can we use this for? This is what separates crack science from crackpot science.
I think cold fusion was a very instructive example. The research had all the indications of bad science, and was proved to be nonsense. But the huge story in the middle of it is not nonsense, the idea of (nearly) unlimited energy (nearly) for free. There is a distinct possibility that a breakthrough could abruptly cure all our energy woes. It breaks with our conditioning, the idea of getting something valuable for free. Though when something gets free it isn’t valuable any more, and soon will be taken for granted. Worse, this is dangerous territory for any self-respecting scientist as it is hard to separate good science from bad, and it will be crowded with bad science. This will work when the alchemical mix is right, or under favourable phases of the moon.
Much of the bad science has been epicycles created by scientists themselves (“epicycle” is commonly used as a disparaging term for extra crutches used for a scientific theory to work). That includes the mystical ether. 19th century science needed ether to be able to explain light as to their knowledge waves always travelled through a medium, like water waves, and thus there had to be some medium between us and the light-giving sun.
When I was younger and more starry-eyed I considered a pursuit in astrophysics and I am glad I didn’t. It is hard to do experimental cosmology and in my opinion astronomy is currently in disarray, full of dark energy and dark matter and epicycles upon epicycles. This on the other hand is a good career opportunity for a scientist. When something is obviously broken you can make a name for youself if you can fix it (or in this case fixing parts of it).