Advances in Bayesian cross-validation and tensor modelling

The first chapter presents a work I undertook with my supervisor Giacomo Zanella. We propose a novel estimator for the log pointwise predictive density(lppd) of a Bayesian model. The naive method to calculate this quantity would require running n markov chain Monte Carlo (MCMC) chains, resulting in an unfeasible computational cost. A classical approach to overcome such a problem is to leverage importance sampling, which although solves the computational hurdle results in a estimator that can be potentially very unstable. We propose to generate the samples from a particular mixture of the leave-one-out posteriors within the typical importance sampling framework. We provide theoretical proofs of the stability of our estimator for a broad family of models also in the challenging asymptotic regime of infinite dimensionality of the data, as well as both synthetic and real data applications displaying the superior performance of our estimator compared to competitors in the literature. The second chapter presents a project I undertook under the supervision of Giacomo Zanella and Omiros Papaspiliopoulos. In this work, we focus on studying how the local convergence rate of alternating least squares(ALS) algorithms behaves as we increase the size of the problem in the context of matrix completion. We first show that, under full design, classical ALS is non-scalable. Studying the geometric properties of the optimization landscape, we propose a modification to the classical ALS algorithm, which we term ALS with Gram Calibration, and we show that such an algorithm is scalable under full design. We then provide empirical evidence that such a behavior is maintained in various sparsity scenarios. The third and fourth chapters present the works I conducted during the visiting student period at Duke University under the supervision of David Dunson and Peter Hoff, respectively. Both works concentrate on Bayesian formulations of the Candecomp/Parafac (CP) decomposition. In the third chapter, which focuses on modelling dynamically evolving binary networks, we leverage the CP decomposition as a building block to propose a novel non-parametric tensor decomposition. We prove that such a decomposition is flexible enough to represent any underlying tensor, and we also show that our prior has full support. We then provide empirical evidence of our model capabilities, both for a synthetic design and a real dataset from ecology. In the fourth chapter, we develop a Bayesian hierarchical CP model with multiplicative error and apply it to a dataset of excitation-emission matrices (EEMs) from different sources of the Neuse River. Compared to classical optimization-only procedures, our proposed model allows our proposed model allows the borrowing of information across sources and the incorporation of available knowledge through the prior. Moreover, the multiplicative error term explicitly models the positivity of the data. We show some very promising initial results, with future research looking to extend those and leverage the generative capabilities of our model in other tasks. The fifth chapter presents a I carried out with Christoph Feinauer, Barthelemy Meynard-Piganeau and Carlo Lucibello. It focuses on protein inverse folding, in which the task is to generate a sequence of amino acids that will fold into a desired three-dimensional functioning protein. Such a problem is highly complex, also because such a mapping is notoriously many-to-one, meaning that many sequences fold into the same three-dimensional structure in nature. Typical deep-learning approaches, though, focus solely on mapping the native sequence to the structure, failing to model this diversity. We hence propose a novel deep learning architecture, which we term InvMSAFold , that explicitly models this diversity. We show the benefits of this modelling choice with various experiments, demonstrating how our work could be helpful in many protein-engineering pipelines.