Develop a Train-Test Split Strategy for Reliable Model Evaluation

You are a machine learning engineer specializing in model evaluation and validation methodology. Help me design a robust train-test split strategy for my dataset. **Dataset Information:** - Total samples: [TOTAL_SAMPLES] - Number of features: [NUM_FEATURES] - Problem type: [PROBLEM_TYPE e.g., binary classification, regression, multi-class] - Class distribution (if classification): [CLASS_DISTRIBUTION e.g., 90/10 imbalanced] - Temporal component: [YES/NO — does data have a time dimension?] - Data collection method: [COLLECTION_METHOD e.g., random survey, time-series logs, web scraping] **Please address the following:** 1. **Split Ratio Recommendation:** Recommend the optimal train/validation/test split ratio for my dataset size and problem type. Justify the ratio with statistical reasoning, and explain at what sample size thresholds different ratios become appropriate. 2. **Splitting Methodology:** Based on my data characteristics, recommend the appropriate splitting method: - Simple random split - Stratified split - Temporal/chronological split - Group-based split Explain why the recommended method prevents data leakage and ensures generalizability. 3. **Imbalanced Data Handling:** If my classes are imbalanced, detail how the split strategy should account for this. Include stratification techniques and discuss whether oversampling/undersampling should occur before or after splitting. 4. **Reproducibility Protocol:** Provide a checklist for ensuring reproducible splits, including random seed management, data versioning, and documentation practices. 5. **Implementation Code:** Write Python code using scikit-learn that implements the recommended strategy with proper stratification, random seeds, and validation checks (e.g., verifying no overlap between sets, checking class distributions post-split). 6. **Red Flags Checklist:** List 5 warning signs that indicate a flawed split strategy and how to diagnose each one.