Sepsis is a major public health issue and over one-third of people who die in U.S. hospitals have sepsis. Early detection and antibiotic treatment of sepsis can both decrease the mortality of patients and cut down the healthcare expenses.
Previous methods formalize sepsis prediction problem in binary classification framework and we classify them into two categories namely, instance-based methods and longitudinal-based methods. Instance-based methods utilize traditional machine learning algorithm such as logistic regression and family of gradient boosting tree but they achieve lower accuracy as they ignore longitudinal dependency among several patients' outcomes. Longitudinal-based methods like recurrent neural networks (RNN) fulfilled drawback by utilizing longitudinal information. But the aforementioned methods all failed to optimize with respect to a given score function which is designed by physicians to give penalty or reward balanced between accuracy and early prediction.
To tackle this problem, we formalize early sepsis prediction in an actor-critic style reinforcement learning framework where patients' longitudinal measurements can be seen as states, discrete action as prediction is predicted based on the current state and reward is given by user-defined score function. In particular, we implement two key components actor and critic using RNN. Experiments on about 40,000 patients provided by The 20th PhysioNet/Computing Challenge in Cardiology showed the effectiveness of our method which achieved -0.15 average score per patient on validation dataset.