The earliest explanations of the world, and thus the earliest “science”, were probably myths, which explained the world by positing supernatural beings, such as spirits, who made it behave as it did. The next small step after that towards what we recognize as science was taken by the Greeks, who tried to explain the world in a way that was less an amalgamation of different stories and more of a theory created explicitly for the purpose of understanding. Of course their theories leaned heavily on projecting agency onto the world as well, but they were also more systematic in the way they attempted to explain everything by appealing only to a small number of general principles (at least in the beginning). By their own standards I consider those explanations to be fairly successful. Obviously they weren’t able to make predictions with them, but they were able to shoehorn the observed phenomena into being described by some story about the elements or the Platonic solids. And for them that was sufficient, because it allowed them to at least posit some reason for events to happen as they did, which makes the world an understandable and more regular place.
And for a long time Greek science was considered the pinnacle of what we could know about the world. Even when new phenomena were discovered people would simply find a way to fit them into the Greek model. Eventually though this period of relative scientific inactivity came to an end, people began to challenge the correctness of Aristotle, and new theories were put forward to explain the world. Honestly I don’t know where the impetus to shake off the old science came from; perhaps it was the development of new mathematics which made precise scientific theories possible where previously they had been inconceivable, or perhaps it was as a byproduct of the beginnings of the industrial revolution, where engineering advances required more precise predictions in order to determine whether a machine that was under consideration would perform as expected. Given our modern attitudes such a scientific revolution wouldn’t be hard to motivate; all we would have to do would be to point out that the new theories predicted more and more accurately than the Aristotelian ones did. But that is an attitude that came later, as a result of that scientific revolution. To argue against Aristotelian science the new scientists needed something else, something that would convince their learned colleagues who were not particularly inclined to care about practicality. Thus they argued that Aristotelian science was arbitrary, and so in a sense irrational. And they presented their new theories as deduced from rationally indubitable principles, and thus as necessary facts about the world.
Some of the early scientists who argued for the superiority of the new science in that way were part of the rationalist movement. The rationalists thought that everything could be understood by the application of pure reason, an attitude that seemed plausible at the time because many believed that god guaranteed that fact. Either because god was a perfectly rational creator, and thus that everything must thus have some reason behind it, and so opening up the possibility that the reason might be uncovered by the human intellect, or because god charitably designed humanity with that capability. Descartes is a good example of such a rationalist. Descartes was “originally” a scientist rather than a philosopher, although the distinction didn’t mean much at the time; he was focused primarily on devising laws for things such as optics and motion. But Descartes was not happy to just devise laws that seemed accurate; he referred to such laws as hypothetical reasoning, meaning that they might be the case but that they weren’t confirmed. Descartes wanted his system to follow from purely rational principles, and not just to be the best fit to the evidence. And because of that he was led to more philosophical endeavors, which is unavoidable if you want your physics to follow from some kind of metaphysics. There, Descartes hoped, he had found principles, such that bodies are defined by extension, from which everything else would follow and which were completely rational.
But did Descartes and the other rationalists really accomplish what they hoped, setting aside for the moment the errors in their systems? Obviously they thought that the principles that they had developed as a starting point were “rational”, but from a modern perspective they seem as arbitrary as the principles of Aristotelian physics, just in a different way. Consider the idea that bodies are defined by extension. If that principle is rational then how can we consistently entertain the idea of point particles, which have location but no extension? And yet we do entertain them, quite productively. What we have done then is simply move the arbitrariness around. Aristotelian physics was claimed to be arbitrary because the rules the world was supposed to operate by did not themselves fit into any larger scheme. But the systems of the rationalists were equally arbitrary because they principles they picked as a starting point were not themselves guaranteed, as was claimed, but were simply highly intuitive to the rationalists. But, despite that, surely they were correct in rejecting Aristotelian physics, and investigations in a “rationalist” vein continue today; although people aren’t looking for more “rational” principles scientists are always looking for more fundamental laws which will explain the laws that we have already established (and more fundamental laws for those, and so on). How then can the rationalist scientific project, or some successor to it, be so successful given that we know it is flawed?
Let us return to the beginning then and ask whether Aristotelian science was really flawed because of how arbitrary it was. Certainly that seems to be its fault, especially if is brought to our attention that people would explain the fact that certain substances put people to sleep by saying that they had the property of putting people to sleep (actually they used a Latin word to name that property, but it amounts to the same thing). And that is no explanation at all. But, on the other hand, Aristotelian science had its virtues as well. Its primary virtue is that it proceeded to its laws (at least initially) by observation alone, just as modern scientists are supposed to. An Aristotelian scientist would go out into the world, observe what occurred, and then label and systematize it. And that is what modern scientists do too. Sure we are able to explain why certain substances put people to sleep now, but ultimately, at the level of the most basic particles, we still attribute properties to them, such as the property of the disposition to engage in certain interactions with other particles, without any “reason” for doing so. Rather we simply point out that those properties explain our observations extremely well, and are hence justified in that way. And that is what the Aristotelian scientist was doing when he described substances as having the power of putting people to sleep, just at a much coarser level.
What Aristotelian science was really suffering from was dogma. It simply wasn’t open to revision of any kind with respect to the main body of the theory, and thus scientists were restricted to adding more and more special cases, and unexplained properties, in order to make it work. Thus what the rationalists did right was rejecting the existing dogma and trying something new. But the rationalists were not against dogma. Indeed if the rationalist project had gained a greater foothold in science they would just have created their own dogmas. Instead of unquestionable Aristotelian principles there would have been unquestionable rational principles, and scientists would not have to hammer their explanations into being compatible with and derivable from those principles. Indeed I can see a foreshadowing of that possible outcome in Descartes own work. Originally he “deduced” that the Earth must move. However he realized that making that claim would be bad for his career and, showing a somewhat typical lack of intellectual character, he fiddled with his deductions until he could conclude that the Earth didn’t move. Not only is that absurd for a system which is supposed to be deductive (you can’t fiddle with a deduction, if you revise it you must do so by rejecting some of the principles you started with), but it illustrates how future scientists would have fiddled with their deductions until they were compatible with the rational principles. Fortunately rationalism didn’t get a grip on the scientific community, probably because new theories soon came along, such as Newton’s, which were more successful and which left the idea that they were derived from purely rational principles somewhat by the wayside.
The fact that modern scientific investigations always seem to look for the underpinnings of what are currently held to be basic laws we can attribute not to an aftereffect of the rationalist program, specifically as a kind of looking for more rational and simpler principles, but rather to an ongoing attempt at rejecting dogma. Specifically scientists look at the current laws and attempt to either show them to be wrong, or to show them to be a manifestation of other laws, and thus rejecting them as primitive. And so if we attack what seems arbitrary it is not because being arbitrary is essentially bad or irrational, but because it might be a dogma which we might productively challenge by replacing it with some more complicated and more accurate description of the world.