Tune XGBoost Hyperparameters for Maximum Model Performance

You are an expert machine learning engineer with extensive experience tuning gradient boosting models in production. Help me develop a systematic XGBoost hyperparameter tuning strategy. **Project Details:** - Task: [CLASSIFICATION/REGRESSION] - Dataset size: [NUMBER_OF_ROWS] rows × [NUMBER_OF_FEATURES] features - Target variable: [DESCRIBE_TARGET, e.g., binary churn label, continuous price] - Class balance (if classification): [BALANCED/IMBALANCED — specify ratio] - Current baseline performance: [METRIC_NAME: VALUE, e.g., AUC: 0.82] - Computational resources: [CPU_ONLY/GPU_AVAILABLE, approximate time budget] - Overfitting observed: [YES/NO — describe gap between train and validation scores] **Please provide:** 1. **Parameter Priority Ranking:** Rank the top 10 most impactful XGBoost parameters for my specific scenario and explain WHY each matters. Group them into tiers: Tier 1 (tune first), Tier 2 (tune second), Tier 3 (fine-tune last). 2. **Recommended Search Ranges:** For each parameter, provide specific search ranges tailored to my dataset size and problem type. Format as a Python dictionary ready for use with Optuna or scikit-learn's search utilities. Include: - n_estimators, max_depth, learning_rate, min_child_weight - subsample, colsample_bytree, colsample_bylevel - gamma, reg_alpha (L1), reg_lambda (L2) - scale_pos_weight (if imbalanced) 3. **Tuning Strategy:** Design a phased tuning approach: - Phase 1: Fix learning_rate, tune tree structure parameters - Phase 2: Tune regularization parameters - Phase 3: Lower learning_rate and increase n_estimators Provide the rationale for this ordering. 4. **Search Method Recommendation:** Compare Random Search, Bayesian Optimization (Optuna), and Hyperband for my setup. Recommend one with justification and provide a ready-to-run Python code template. 5. **Overfitting Mitigation:** Provide 5 specific strategies to reduce the train-validation gap using XGBoost parameters, early stopping configuration, and data-level techniques. 6. **Validation & Final Evaluation:** Describe how to properly evaluate the tuned model, including how to avoid optimistic bias from hyperparameter search and when to use a held-out test set versus nested cross-validation.