Hierarchical structures in Bayesian Statistics

The amazing growth of computational power, storage capacity and data sources opens new exciting frontiers in the processing and analysis of data. This brings up new challenges when modeling phenomena with complex dependence structures, as both statisticians and applied researchers must deal with high dimensional problems and provide accurate inference with a reasonable computational effort. As a consequence, a tradeoff among complexity of the model, its interpretability and accuracy, and availability of efficient algorithms is a difficult, yet crucial, issue in modern data analysis. A theoretical understanding of widely applied methodologies and algorithms is therefore vital to provide convincing guarantees for the quality of the inference. A natural framework to address the above issues is provided by Bayesian inference. Indeed it combines principled modeling and coherent learning methodology with the availability of sampling schemes and other computational algorithms. In particular, this thesis will focus mostly (though not exclusively) on discrete Bayesian Nonparametric models, which allow for extremely flexible learning mechanisms that can capture complex features of the phenomenon of interest. However, the presence of an infinite dimensional parameter space makes the mathematical and methodological investigation more demanding. Within this framework, this thesis follows three distinct, but related, directions: (i) modelling complex dependence structures (e.g. time series, multi-samples data...) via a Bayesian nonparametric approach, (ii) mathematical investigation of the resulting inferential procedures, complemented by the proposal of methods for measuring and tuning dependence and proving frequentist asymptotic properties, (iii) rigorous analysis of the computational algorithms employed for posterior inference with the aforementioned structures, with a focus on high dimensional problems. A unifying thread shared by all these lines of research is the study of the specific probabilistic structure considered: indeed, the choice of a particular dependence structure (more specifically hierarchical models), which is often selected through modelling considerations (prior information, domain-specific knowledge, etc.), requires the investigation of the associated inferential and computational properties. Indeed, different specifications may have significantly different levels of analytical tractability and the performance of routinely used MCMC algorithms (e.g. gradient-based methods, Gibbs samplers) may greatly vary. Foundations of Bayesian learning are discussed in the first Chapter, with a focus on the predictive viewpoint; the relevance of hierarchical structures is also emphasized. Chapters 2 and 3 discuss various classes of hierarchical models, based on different nonparametric priors; both theoretical and methodological aspects are presented. The last Chapter, finally, deals with the computational challenges arising in high dimensional hierarchical models.