Evolving Convolutional Neural Networks for Image and Text Classification Tasks

In the Machine Learning field, Convolutional Neural Networks (CNNs) have successfully been applied to image and text classification tasks. Innovative research in CNN design, such as the usage of deeper models and data augmentation, has resulted in significantly increased performance. However, the resultant growth in CNN complexity from these innovations has required a corresponding increase in expert skills to design them. Furthermore, CNN architectures and data augmentation techniques designed for a classification task in one domain may not generalise to other classification tasks in a different domain, meaning the difficult and costly design process must be repeated. Evolutionary Deep Learning Neural Architecture Search (EDLNAS) has been used to automatically evolve performant CNNs for various classification tasks, including image and text classification. However, the approach suffers from high computation costs, often resulting in algorithms needing to focus on a single task for a single domain, thereby missing the opportunity to achieve algorithm generalisability. This thesis aims to investigate and improve upon the efficiency, effectiveness, and generalisability of EDLNAS of CNNs for image and text classification tasks without using data augmentation. Achieving this goal can mitigate three significant problems: the high-computational cost of EDLNAS algorithms, the generalisability problem, and the problem of collecting additional training data or implementing bespoke data augmentation techniques that require costly expert domain knowledge. This thesis proposes a new, configurable, and extensible version of Cellular Encoding (CE), including supporting network structures designed to represent and construct novel CNN architectures. The encoding successfully represents various novel CNN architectures of varying width and depth, suitable for image or text classification tasks. This thesis proposes a data augmentation-free EDLNAS method to automatically evolve novel CNN architectures for image and text classification tasks, using the proposed CE representation and predesigned state-of-the-art micro-architectures. The results show that peer-competitive CNNs can be evolved in less than one GPU day across image or text domains. Next, this thesis proposes to evolve micro-architectures to boost the performance in evolving CNN architectures for image and text classification tasks without data augmentation. The results show that this is achieved without increasing the number of GPU days required. This thesis proposes the first data augmentation-free cooperative coevolution algorithm to decompose a CNN architecture into four constituent sub-architectures, each evolved separately but in cooperation, to synthesise performant CNN architectures. A new population structure and novelty injection mechanism are proposed to create a better balance between exploration and exploitation. Furthermore, a persistent offline database is created, storing tens of thousands of evolved CNN architectures and associated performance metrics. The results show that highly competitive CNN architectures can be evolved in less than one GPU day, with the algorithm achieving a good balance between exploration and exploitation during the evolution process. Finally, this thesis proposes the first EDLNAS method that models a database of stored CNN architectures and performance metrics to create a deep neural network prediction model to predict the potential classification accuracy of a candidate CNN architecture, thereby removing the need to conduct back-propagation training during the evolutionary process. The method is able to evolve a large population of CNN architectures for both image and text classification tasks in seconds, making the evolutionary process near real-time while still achieving state-of-the-art peer-competitive results.