Adverse Impact Reduction and Job Performance Optimization via Pareto-Optimal Weighting: A Shrinkage Formula and Regularization Technique Using Machine Learning
Abstract
In personnel selection practice, one useful technique for reducing adverse impact and enhancing diversity is the Pareto-optimal weighting approach of De Corte et al. (2007). This approach produces a series of hiring solutions that characterize a diversity–job performance trade-off and can lead to more optimal selection outcomes (sometimes doubling the number of job offers for minority applicants without changing the job performance outcomes of personnel selection). Despite these advantages, recent research has identified a potential problem with the Pareto-weighting technique—Pareto solutions suffer from shrinkage upon cross-validation. To address the problem of shrinkage in the Pareto trade-off curve (i.e., diversity shrinkage and validity shrinkage), we offer two contributions. First, a shrinkage approximation formula is introduced (similar to a formula for adjusted &R^2&, but applicable to the entire Pareto trade-off curve). Second, we derive a novel technique for the regularization of Pareto-optimal predictor weights (borrowed from the field of machine learning), which is designed to produce predictor weights that are less vulnerable to shrinkage (similar to ridge regression and adapted from the elastic net technique). Both approaches—the proposed Pareto shrinkage formula and the proposed regularization technique—are then evaluated using Monte Carlo simulation. Recommendations are provided for approximating potential diversity–performance trade-off curves in personnel selection, while accounting for shrinkage.
