A tutorial on evaluating time-varying discrimination accuracy for survival models used in dynamic decision-making.

Many medical decisions involve the use of dynamic information collected onindividual patients toward predicting likely transitions in their future healthstatus. If accurate predictions are developed, then a prognostic mode canidentify patients at greatest risk for future adverse events, and may be usedclinically to define populations appropriate for targeted intervention. Inpractice, a prognostic model is often used to guide decisions at multiple timepoints over the course of disease, and classification performance, i.e.sensitivity and specificity, for distinguishing high-risk versus low-riskindividuals may vary over time as an individual's disease status and prognosticinformation change. In this tutorial, we detail contemporary statisticalmethods that can characterize the time-varying accuracy of prognostic survivalmodels when used for dynamic decision-making. Although statistical methods forevaluating prognostic models with simple binary outcomes are well established,methods appropriate for survival outcomes are less well known and requiretime-dependent extensions of sensitivity and specificity to fully characterizelongitudinal biomarkers or models. The methods we review are particularlyimportant in that they allow for appropriate handling of censored outcomescommonly encountered with event-time data. We highlight the importance ofdetermining whether clinical interest is in predicting cumulative (orprevalent) cases over a fixed future time interval versus predicting incidentcases over a range of follow-up time, and whether patient information is staticor updated over time. We discuss implementation of time-dependent ROCapproaches using relevant R statistical software packages. The statisticalsummaries are illustrated using a liver prognostic model to guidetransplantation in primary biliary cirrhosis.