Evaluating the fairness in the performance of machine learning methods

Abstract

Machine learning plays an increasingly important role in our lives, tackling both prevalent and specialized but high-risk problems. Motivated by legislation, responsibility to ensure transparency and accountability of machine learning methods and needs to maintain public's trust on the algorithms used in our lives, researchers have paid much attention to the fairness issue in machine learning. There are many methods developed to measure, reduce and even eliminate the fairness issue for both general and specific settings or algorithms. In this project, we focus on fairness in classification machine learning problems in healthcare which is one critical field of the application of machine learning. We found a general way to detect the fairness issue in the performance of machine learning methods and found the general solutions to address the issue in all the dimensions of data, method and metrics. We also introduced fairness threshold to help reduce the fairness issue without retraining the model and performance boundary to help analyze the effect of the methods we tried.