Believing true things is good and hence rationality, the most reliable method of believing true things while not believing false things, is good. (By good I mean of at least practical value, no matter what your goals are.) But most people are not rational. Most people would rather believe what makes them comfortable than what is true. Most people do not reject or suitably modify their beliefs when a contradiction in them is revealed to them. Most people are not swayed by a rational argument to change their beliefs. Most people then are irrational, at least sometimes. But if rationality is good why is irrationality so widespread?
One possibility is that we are hard-wired to be irrational. That evolution was moving us away from being moved almost completely by desire, like animals are, to being rational beings, but before that shift was complete we became smart enough that we were able to escape most selection pressures. And thus evolution wasn’t able to take us as a species as far as it could have. Although this is probably true enough it can’t be the complete story. Because of our evolutionary heritage none of us are perfectly rational. But we are developed enough that we can choose to be more rational than we are naturally disposed to be, by trying to judge possibilities based as much as possible on evidence alone, and resisting our natural inclinations to believe or disbelieve. However, most people don’t make that choice, and that still needs explaining.
I think the answer lies in the fact that true beliefs, and thus rationality, are instrumental. Which is a fancy way of saying that they are good because they help us in getting what we want. But this means that they are necessarily secondary to our other desires. And I hypothesize that the desires of many people conflict with rationality. I think that many people want the universe to be a nice place, for their lives to be important on a grand scale, to be immortal, etc. Rationality unfortunately is not going to give these people what they want. Pure rationality leads to the conclusion that the universe simply isn’t as desired. Now obviously in these cases there is nothing you can do to change the universe to be more like you desire. Thus you must either give up those desires, give up rationality, or live with the fact that the universe simply isn’t as you would like it to be. I think most people resolve this dilemma by giving up rationality. Which is not to say that they embrace irrationality, they are rational when they can be, because rationality is effective. But they fully embrace the occasional failure of rationality in order to preserve the belief that the universe is as they would like it to be.
Clearly there is a sharp divide then between those who treat rationality only as a means to an end, and thus are willing to abandon it when they need to, and thus who see rationality, and truth, as an end in itself. To value rationality and truth themselves is to desire your beliefs to be true and be arrived at by rational processes. And thus having untrue beliefs, or beliefs formed irrationality would make such a person unhappy. Obviously then such a person will have to give up expecting or wanting things to be a certain way, otherwise they will be deeply unhappy. Because to be rational may be to give up things being that way, and to fully accept rationality may be to live believing the nature of the world is deeply unsatisfying. Of course there is always the possibility for some cognitive dissonance to set in, where people who desire to be completely rational construct irrational arguments for the positions they want to be true, and then fail to realize that they are in fact deeply flawed arguments. This may seem to leave them better off, but cognitive dissonance isn’t exactly a blessing, because no amount of cognitive dissonance can prevent the person from eventually seeing some contradictions within their views. And if they dwell on those contradictions eventually they will see that the arguments they have constructed are flawed; thus cognitive dissonance hasn’t resolved the problem it has only postponed it. Perhaps in some ways that might be an improvement for such a person’s state of mind, but on the other hand if a person truly values rationality then they should be trying to avoid cognitive dissonance whenever possible.
But what about limited rationality? What about someone who is rational when it comes to some matters, but abandons rationality when it comes to some others, because they value the truth and they have a strong desire for the universe to work a certain way. In the past this may have been a viable option, where areas of inquiry were relatively separate. For example, there was a time where it was widely believed that scientific and religious truths wouldn’t come into conflict. However nowadays that isn’t the case. Nowadays if a position has any practical content science probably has something to say about it. Nowadays to believe in the existence of a god who interferes in worldly affairs, in spirits that hang around after death, in paranormal powers, etc is to contradict the claims of science, which describes the world working in a way that excludes the meddling of the previously mentioned.
Now I am not trying to condemn being irrational here. Perhaps many people are better off being irrational then embracing truths they would find unpleasant. So long as that irrationality doesn’t interfere too much in practical matters maybe no one is the worse for it. Although I value rationality over believing what is comfortable I don’t think that rationality is necessarily “Good”, meaning that people are somehow intrinsically failing if they are irrational, as many past philosophers have thought. I simply seek to understand the phenomena.