Ultimately justification must bottom out somewhere. If there wasn’t a foundation for justification then either our claims could never be justified (because it would require an infinite number of steps to justify them), or circular justification would have to be permitted. Obviously circular justification cannot be permitted (because with it anything except an internally contradictory claim can be justified). But it is equally clear that there can be no indubitable foundation for justification, there are no necessarily true axioms that all our claims can be founded upon. But the idea that we need such an indubitable foundation is an outdated one, which stems from confusing justification with a process that brings us to necessarily true knowledge. Justification only has to track the beliefs that it is rational or reasonable to have, not those that are necessarily true. And so we can permit the foundation of justification to be less than certain. Indeed the best position to take with respect to justification is that it rests ultimately on facts that are rational or reasonable to trust tentatively, but which may possibly be shown to be false. Let us call those facts evidence.
But of course simply labeling this foundation with the word evidence doesn’t help too much, since we aren’t trying to stick to the usual meaning of the word. What exactly counts as evidence is probably as nuanced as anything else in epistemology, but it is obvious that there are two major requirements: it must be able to be overturned, and it must not be a judgment. It may seem strange to require claims that serve as the foundation of justification to be able to be overturned. After all, wouldn’t it be better to base justification on facts that are known to be true without any room for doubt? But, unfortunately, we don’t have direct access to the truth, and probably never will. Thus whatever we take as evidence could at least possibly be wrong, without our being aware that it is wrong. But if it is possible for false evidence to be revealed as false then we can at least hold out the hope that if we are wrong we will eventually be made aware of the fact. Or, from a different angle, it is rational to use beliefs that can be overturned as the foundation for justification because by doing so we make our epistemic position improvable over time; although we don’t know how close we are to the truth to start with we know that if our evidence is wrong there is at least the possibility of becoming aware of that fact. Thus given enough time (an infinite amount of time to be precise) we can expect all false evidence to be revealed as false, and hence that our epistemic position will become as good as it can possibly be. On the other hand if we use beliefs that cannot be overturned as a basis for justification then our epistemic position has no hope of improving, and while we might possibly be right we have no way to know if we are right. Hence it is more rational to base justification on beliefs that can be overturned.
Hopefully I don’t need to say any more about falsifability, in this day and age it is like beating a dead horse. The more interesting component to evidence is that is must not be a judgment. In the sense I am using it here a judgment is a conclusion reached from other data. To say why we can’t use judgments as evidence let me first just come out and say what evidence is. For us evidence is the data provided by our senses (which are falsifiable by disagreement between two senses and disagreement with someone else about what we are sensing). Let us now consider a judgment, such as that a particular painting is beautiful. This judgment could be falsifiable; we could believe beauty to be an objective property, and so if many people disagreed with us we would believe that we were in error about the painting. But even if this were the case we still couldn’t use such judgments as evidence. Practically judgments are not guaranteed to coincide, there is not necessarily going to be a consensus as to which paintings are beautiful, and so justification based on them is not necessarily, or even probably, going to be truth tracking. And practically the judgments we make reflect our psychological features and not features of the world. More importantly however is the theoretical objection to using judgments as evidence. The theoretical objection is that judgments reflect the application of concepts. And a concept is not meant to reflect a basic feature of the world (it is not the case that there is some beauty-stuff that is added to the painting in addition to the colors and shapes), a concept is thought of as tracking some kind of arrangement of the basic features (a natural kind). And so concepts, while revisable, are revisable in different ways from evidence, which tracks the basic features themselves. Thinking of something as beautiful is not to say anything about what kind of thing beauty is. Or, in other words, a concept uninvestigated doesn’t have the right kind of content to serve as the basis for justification when reasoning about various things, by themselves concepts only tell us how we think about them. On the other hand evidence is supposed to serve as a claim about how things are (although what evidence we have does say something about us as well). As a closing remark I should note that it isn’t necessarily easy to tell where judgment begins and evidence ends (made more complicated by the fact that a judgment may serve much the same role as evidence does in justification when we have evidence to believe that our judgment reliably tracks certain facts). But this is just one further aspect of the falsifability of evidence, what we thought of as evidence may turn out not to be. And we can determine what is a judgment by learning about how we receive information about the world (how our senses work); evidence is then about the features of the world that we know we receive information about through our senses because those features play a causal role in their operation, anything else (beauty, right and wrong, causation) is a matter of judgment.