Arguably the most common criterion for job performance in the I/O literature is manager ratings. The rational is that managers have good knowledge regarding the performance of their immediate subordinates. Overall job performance is typically rated as a single rating or several items combined into an index. Intuitively, there are a number of threats to managers rating many employees, for example, fatigue and ability to differentiate between employees’ performance. Roughly speaking, the ratings could get affected in two ways by increasing the number employees to rate: 1) the average score could become lower/higher (i.e., performance reference drift) and 2) the rating variance could increase/decrease. From a selection perspective, 1) is not necessarily a direct threat to estimation of predictive validity, because different rating levels can be addressed statistically. However, 2) is more difficult to handle, and it could create restriction of range in the criterion variable, resulting in underestimation of predictive validity. In the present study, 56 managers rated job performance on six dimensions for 1 to 60 employees (total n=1164). Results showed that the higher number of employees rated, the lower the average total rating (r=.38, p<.01). However, the variance of the ratings seemed unaffected (r=.16, ns). These results suggest that the number of employees that managers rate is not an immediate threat to the validity estimates in a study. However, it may be a problem in situations where the absolute values of the ratings are of import rather than the relative values.