The demands of fair machine learning are often expressed in probabilistic terms; yet, most of the systems of concern are deterministic in the sense that whether a given subject will receive a given score on the basis of their traits is, for all intents and purposes, either zero or one. What, then, can justify this probabilistic talk? FIU Philosophy Assistant Professor Clinton Castro argues that it can be justified by attending to morally salient aspects of data-driven decision systems (as opposed to, say, the epistemic limitations of particular agents or the identification of some hidden source of stochasticity) and provide a framework for characterizing fair machine learning in probabilistic terms. Castro's framework identifies the statistical reference classes used in fairness measures with what John Rawls called representative individuals, hypothetical persons who are representative of social positions. He then address the question of how to determine in a principled way which social positions should be represented. Castro identifies, motivates and critically evaluates three possible approaches. The first is causal: a group should be represented if it appears in a causal explanation of inequality. The second is psychological: a group should be represented if individuals subjectively identify with it. The third is moralized: a group should be represented if membership to that group alters themoral evaluation of the inequality, independently of the satisfaction of the two previous conditions