lensen2017Using.pdf (416.96 kB)

Using particle swarm optimisation and the silhouette metric to estimate the number of clusters, select features, and perform clustering

Download (416.96 kB)
conference contribution
posted on 06.10.2020 by Andrew Lensen, Bing Xue, Mengjie Zhang
© Springer International Publishing AG 2017. One of the most difficult problems in clustering, the task of grouping similar instances in a dataset, is automatically determining the number of clusters that should be created. When a dataset has a large number of attributes (features), this task becomes even more difficult due to the relationship between the number of features and the number of clusters produced. One method of addressing this is feature selection, the process of selecting a subset of features to be used. Evolutionary computation techniques have been used very effectively for solving clustering problems, but have seen little use for simultaneously performing the three tasks of clustering, feature selection, and determining the number of clusters. Furthermore, only a small number of existing methods exist, but they have a number of limitations that affect their performance and scalability. In this work, we introduce a number of novel techniques for improving the performance of these three tasks using particle swarm optimisation and statistical techniques. We conduct a series of experiments across a range of datasets with clustering problems of varying difficulty. The results show our proposed methods achieve significantly better clustering performance than existing methods, while only using a small number of features and automatically determining the number of clusters more accurately.

History

Preferred citation

Lensen, A., Xue, B. & Zhang, M. (2017, January). Using particle swarm optimisation and the silhouette metric to estimate the number of clusters, select features, and perform clustering. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 20th European Conference on the Applications of Evolutionary Computation (EvoApplications), Amsterdam, NETHERLANDS (10199 LNCS pp. 538-554). Springer International Publishing. https://doi.org/10.1007/978-3-319-55849-3_35

Conference name

20th European Conference on the Applications of Evolutionary Computation (EvoApplications)

Conference Place

Amsterdam, NETHERLANDS

Conference start date

19/04/2017

Conference finish date

21/04/2017

Title of proceedings

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volume

10199 LNCS

Series

Lecture Notes in Computer Science

Publication or Presentation Year

01/01/2017

Pagination

538-554

Publisher

Springer International Publishing

Publication status

Published

ISSN

0302-9743

eISSN

1611-3349

Exports