Analyze ROC Curves and AUC Scores to Evaluate Classifier Discrimination Power

You are a statistical learning expert specializing in classifier evaluation. Perform a thorough ROC curve and AUC analysis based on the following information. **Model Performance Data:** - Model(s) evaluated: [LIST_MODEL_NAMES_WITH_AUC — e.g., "Logistic Regression: AUC=0.87, Random Forest: AUC=0.92"] - Problem type: [BINARY/MULTI-CLASS] - Positive class definition: [POSITIVE_CLASS_LABEL] - Class distribution: [MAJORITY_CLASS_PERCENT]% / [MINORITY_CLASS_PERCENT]% - Domain: [DOMAIN — e.g., medical diagnosis, fraud detection, customer churn] **If available, paste ROC data points or describe the curve shape:** [ROC_DATA_OR_DESCRIPTION] **Please provide the following comprehensive analysis:** 1. **AUC interpretation:** Explain what each model's AUC score means in practical terms. Translate the AUC into a probabilistic interpretation (e.g., "there is an X% chance the model ranks a random positive instance higher than a random negative instance"). 2. **AUC benchmarking:** Compare the AUC scores against standard benchmarks: 0.5 (random), 0.7-0.8 (acceptable), 0.8-0.9 (excellent), 0.9+ (outstanding). Contextualize what constitutes a "good" AUC in the [DOMAIN] domain specifically. 3. **Model comparison:** If multiple models are provided, determine whether the AUC differences are practically significant. Suggest the DeLong test or bootstrap method for statistical comparison and provide a Python code snippet to execute it. 4. **ROC curve shape analysis:** Based on the curve description or data, analyze: (a) Does the curve hug the top-left corner? (b) Is there a sharp elbow suggesting a natural threshold? (c) Are there flat regions indicating poor discrimination at certain thresholds? 5. **Optimal threshold selection:** Recommend methods for choosing the best operating point on the ROC curve, including: Youden's J statistic, cost-sensitive threshold selection for [DOMAIN], and the point closest to (0,1). Provide formulas and a Python implementation. 6. **ROC limitations:** Discuss when ROC/AUC can be misleading, particularly with the class distribution of [MAJORITY_CLASS_PERCENT]%/[MINORITY_CLASS_PERCENT]%. Recommend Precision-Recall curves as a complementary analysis and explain when PR curves are more informative. 7. **Multi-class extension (if applicable):** Explain one-vs-rest and one-vs-one ROC strategies and which is more appropriate for [PROBLEM_TYPE] with [NUM_CLASSES] classes. 8. **Executive summary:** Provide a 3-sentence summary suitable for a non-technical stakeholder explaining the model's discrimination ability. Include Python code snippets using scikit-learn and matplotlib where relevant.