Clustering Categorical Data based on Finite Mixtures

Clustering techniques are often performed to reduce the dimension of very large datasets, whose direct analysis using techniques such as regression can be computationally infeasible. The clustering of non-independent categorical variables in particular poses distinct difficulties due to its lack of a well-defined distance metric, while at the same time existing techniques tend to model variable correlations based on the latent group membership, requiring strong assumptions of conditional independence and low-to-moderate variable correlations.

This thesis proposes a joint model clustering approach for this data type based on finite mixture models, with a heavy focus on clustering based on correlation. In the case of row-clustering data with non-independent columns, we seek to model the dependency structure of columns using either pairwise joint models with misspecified likelihood, or full joint models. In the case of column clustering ordinal datasets with non-independent columns, this thesis proposes an anchor model approach where each column in the same group modelled by only one “anchor” column within the group. The result is a proper likelihood model which allows for straightforward parameter estimation while not being overly restrictive on the modelling space.