Evaluation of predictive model performance of an existing model in the presence of missing data

In medical research, the Brier score (BS) and the area under the receiver operating characteristic (ROC) curves (AUC) are two common metrics used to evaluate prediction models of a binary outcome, such as using biomarkers to predict the risk of developing a disease in the future. The assessment of an existing prediction models using data with missing covariate values is challenging. In this article, we propose inverse probability weighted (IPW) and augmented inverse probability weighted (AIPW) estimates of AUC and BS to handle the missing data. An alternative approach uses multiple imputation (MI), which requires a model for the distribution of the missing variable. We evaluated the performance of IPW and AIPW in comparison with MI in simulation studies under missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR) scenarios. When there are missing observations in the data, MI and IPW can be used to obtain unbiased estimates of BS and AUC if the imputation model for the missing variable or the model for the missingness is correctly specified. MI is more efficient than IPW. Our simulation results suggest that AIPW can be more efficient than IPW, and also achieves double robustness from miss-specification of either the missingness model or the imputation model. The outcome variable should be included in the model for the missing variable under all scenarios, while it only needs to be included in missingness model if the missingness depends on the outcome. We illustrate these methods using an example from prostate cancer

Evaluation of predictive model performance of an existing model in the presence of missing data

Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/168241/1/sim8978_am.pdfhttp://deepblue.lib.umich.edu/bitstream/2027.42/168241/2/sim8978.pd

BMI1, stem cell factor acting as novel serum-biomarker for Caucasian and African-American prostate cancer.

Lack of reliable predictive biomarkers is a stumbling block in the management of prostate cancer (CaP). Prostate-specific antigen (PSA) widely used in clinics has several caveats as a CaP biomarker. African-American CaP patients have poor prognosis than Caucasians, and notably the serum-PSA does not perform well in this group. Further, some men with low serum-PSA remain unnoticed for CaP until they develop disease. Thus, there is a need to identify a reliable diagnostic and predictive biomarker of CaP. Here, we show that BMI1 stem-cell protein is secretory and could be explored for biomarker use in CaP patients.Semi-quantitative analysis of BMI1 was performed in prostatic tissues of TRAMP (autochthonous transgenic mouse model), human CaP patients, and in cell-based models representing normal and different CaP phenotypes in African-American and Caucasian men, by employing immunohistochemistry, immunoblotting and Slot-blotting. Quantitative analysis of BMI1 and PSA were performed in blood and culture-media of siRNA-transfected and non-transfected cells by employing ELISA. BMI1 protein is (i) secreted by CaP cells, (ii) increased in the apical region of epithelial cells and stromal region in prostatic tumors, and (iii) detected in human blood. BMI1 is detectable in blood of CaP patients in an order of increasing tumor stage, exhibit a positive correlation with serum-PSA and importantly is detectable in patients which exhibit low serum-PSA. The clinical significance of BMI1 as a biomarker could be ascertained from observation that CaP cells secrete this protein in higher levels than cells representative of benign prostatic hyperplasia (BPH).BMI1 could be developed as a dual bio-marker (serum and biopsy) for the diagnosis and prognosis of CaP in Caucasian and African-American men. Though compelling these data warrant further investigation in a cohort of African-American patients

Kallikrein markers performance in pretreatment blood to predict early prostate cancer recurrence and metastasis after radical prostatectomy among very high-risk men

Background: To assess whether a prespecified statistical model based on the four kallikrein markers measured in blood—total, free, and intact prostate-specific antigen (PSA), together with human kallikrein-related peptidase 2 (hK2)—or any individual marker measured in pretreatment serum were associated with biochemical recurrence-free (BCR) or metastasis-free survival after radical prostatectomy (RP) in a subgroup of men with very high-risk disease. Methods: We identified 106 men treated at Mayo Clinic from 2004 to 2008 with pathological Gleason grade group 4 to 5 or seminal vesicle invasion at RP. Univariable and multivariable Cox models were used to test the association between standard predictors (Kattan nomogram and GPSM [Gleason, PSA, seminal vesicle and margin status] score), kallikrein panel, and individual kallikrein markers with the outcomes. Results: BCR and metastasis occurred in 67 and 30 patients, respectively. The median follow-up for patients who did not develop a BCR was 10.3 years (interquartile range = 8.2-11.8). In this high-risk group, neither Kattan risk, GPSM score, or the kallikrein panel model was associated with either outcome. However, after adjusting for Kattan risk and GPSM score, separately, preoperative intact PSA was associated with both outcomes while hK2 was associated with metastasis-free survival. Conclusions: Conventional risk prediction tools were poor discriminators for risk of adverse outcomes after RP (Kattan risk and GPSM risk) in patients with very high-risk disease. Further studies are needed to define the role of individual kallikrein marker forms in the blood to predict adverse prostate cancer outcomes after RP in this high-risk setting

BMI1 Polycomb Group Protein Acts as a Master Switch for Growth and Death of Tumor Cells: Regulates TCF4-Transcriptional Factor-Induced BCL2 Signaling

<div><p>For advanced prostate cancer (CaP), the progression of tumors to the state of chemoresistance and paucity of knowledge about the mechanism of chemoresistance are major stumbling blocks in the management of this disease. Here, we provide compelling evidence that BMI1 polycomb group protein and a stem cell factor plays a crucial role in determining the fate of tumors vis-à-vis chemotherapy. We show that progressive increase in the levels of BMI1 occurs during the progression of CaP disease in humans. We show that BMI1-rich tumor cells are non-responsive to chemotherapy whereas BMI1-silenced tumor cells are responsive to therapy. By employing microarray, ChIP, immunoblot and Luciferase reporter assays, we identified a unique mechanism through which BMI1 rescues tumor cells from chemotherapy. We found that BMI1 regulates (i) activity of TCF4 transcriptional factor and (ii) binding of TCF4 to the promoter region of anti-apoptotic <i>BCL2</i> gene. Notably, an increased TCF4 occupancy on <i>BCL2</i> gene was observed in prostatic tissues exhibiting high BMI1 levels. Using tumor cells other than CaP, we also showed that regulation of TCF4-mediated BCL2 by BMI1 is universal. It is noteworthy that forced expression of BMI1 was observed to drive normal cells to hyperproliferative mode. We show that targeting BMI1 improves the outcome of docetaxel therapy in animal models bearing chemoresistant prostatic tumors. We suggest that BMI1 could be exploited as a potential molecular target for therapeutics to treat chemoresistant tumors.</p></div

BMI1 induces growth of normal primary prostate cells (PrEC) by abolishing senescence and regulates the expression of proliferation-associated genes in CaP cells.

<p><b>(A–B)</b> While PrEC replicated for 5 passages and entered into senescence, BMI1-rich counterparts replicated and avoided senescence upto 8th passages. (A) <b>Inset</b> 400X of magnified areas show senescent morphology features i.e. globular shape and (B) indicate staining for ß-galactosidase. <b>(C)</b> Scattered Plot for qPCR array. The dots indicate gene expression on a log-scale representing the change in BMI1 silenced-LNCaP cells. Fold change (2∧- Delta Ct) is the normalized expression (2∧-Delta Ct) in the BMI-silenced cells divided by the normalized expression of Control. <b>(D and E)</b> Immunoblots represents the effect of BMI1-silencing and BMI1-overexpression on the expression of Cyclin-D1, BCL2 and p16 proteins in cells. The equal loading of protein was confirmed by ß-actin.</p

BMI1 protein levels are increased during the progression of CaP disease in human patients and BMI1 induces CaP cell proliferation.

<p><b>(A)</b> Immunoblot represents BMI1 protein levels in normal, dysplasia and tumor prostatic tissues as assessed by immunoblotting <b>(B)</b> in representative photomicrographs point to BMI1-positive immunostaining in neoplastic and non-neoplastic regions of prostatic specimens. Magnification 40X. <b>(C)</b> Immunoblot represents the effect of androgen on BMI1 expression in cells assessed by immunoblotting. (Di–Dii; Ei–Eii and Fi–Fii) Histograms represent the growth, rate of proliferation and clonogenic proliferation of BMI1-silenced and -overexpressed CaP cells measured by MTT, 3[H]thymidine uptake and soft-agar colony formation assays. Each histogram represents mean ± S.E. of three independent experiments, * indicates p<0.05. Equal loading of protein for immunoblotting was confirmed by ß-actin.</p

BMI1 confers chemoresistance to tumor cells.

<p>Rate of proliferation and apoptosis in cells were determined by 3[H]thymidine uptake and flow cytometery respectively. <b>(A–D)</b> Histograms represents the rate of proliferation in <b>(A–B)</b> LNCaP and <b>(C–D)</b> PC3 cells harboring varied BMI1 levels and treated with chemotherapeutic agents. Vehicle treated cells served as control. <b>(Ei and Fi)</b> immunoblots represent the levels of BMI1, Cyclin-D1 and BCL2 proteins in docetaxel-resistant, and BMI1-silenced docetaxel-resistant cells. (Eii and Fii) Histograms showing the rate of proliferation in docetaxel-resistant, and BMI1-silenced docetaxel-resistant cells. For immunoblot analyses (Figure Ei and Fi), equal loading of proteins was confirmed by ß-actin. (A–D, Eii and Fii) Each bar represents mean ± SE of three independent experiments, * represents P<0.05. <b>(G–H)</b> represents quantitative estimation of apoptosis in BMI1-silencing chemoresistant cells. The lower right quadrant of the FL1/FL2 plot (Annexin V-FITC) represent early apoptosis and the upper right quadrant (labeled with AnnexinV-FITC and PI) represent late apoptosis.</p