Group Regression With Tailored Penalties for High‐Dimensional Data With or Without Predefined Groups

ABSTRACT In high‐dimensional regression, covariates often have a natural grouping structure, and it is crucial to perform analyses at the group level rather than the individual covariate level. Group regression is an effective approach for leveraging inherent grouping structures among covariates, enhancing interpretability, and improving analyses. Penalized group regression methods such as group lasso impose structured sparsity or group‐level penalties to obtain robust and interpretable results. However, there are two major limitations with such existing methods: first, they are developed based on a one‐penalty‐type‐fits‐all approach, which can be restrictive and suboptimal in practice, and second they only work if groups of covariates are specified in advance. To address these issues, we provide a general framework for high‐dimensional group regression and propose methods that allow different types of penalties for groups depending on group structures. We develop a novel group correlation learning method to identify groups of covariates in situations where no groups are prespecified. We provide extensive theoretical results for our methods, including upper bounds for estimation and prediction errors along with asymptotic rates. We also obtain the exact distributions of our group‐penalized estimators for statistical inference. We provide an R package called HDGR for the implementation of the proposed methods.