Please use this identifier to cite or link to this item:
Calculation of Dissimilarity Matrix for Mixed-type Data
|Issue Date: ||2018-07-03 17:23:30 (UTC+8)|
Clustering is a common method for data mining. It requires the information about the distance between observations. The way to define the distance becomes a big challenge due to the convenience of data collection. Datasets are in more complex structures, such as mixed-type. Two types of problems have arisen: how to measure the distances between categorical variables and how to measure the distances for mixed variables. The current study proposed an algorithm to define the distance of categorical variables by the ability of distinguishing other related variables. On the other hand, for continuous variables, first, variables were normalized and weighted Euclidean distances were calculated. Then, two distances we calculated above were combined to find a final distance.
Hierarchical clustering was used to verify the performance of proposed method, through some real-world data compared with the methods of the previous paper. The experiments results showed that the proposed method was comparable with other methods. The overall average performance was the best. The technique can be applied to all types of the data. In addition, by visualizing the proposed distance matrix from the heat maps, we found that the number of cluster patterns were the same as the level of class in the majority of our examples.
|Reference: ||1.A. Ahmad, L. Dey, “A k-mean clustering algorithm for mixed numeric and categorical data” , Data & Knowledge Engineering vol. 63, November 2007, pp.503-527.|
2.C. Stanfill, D. Waltz, “Toward memory-based reasoning” , Commun. ACM 29(12), 1986, pp. 1213-1228.
3.D.R. Wilson, T.R. Martinez, “Improved heterogeneous distance functions” , J. Artif. Intell. Res. 6, 1997, pp. 1-34.
4.D. Ienco, R. G. Pensa, and R. Meo, “From context to distance: Learning dissimilarity for categorical data clustering” , ACM Transactions on Knowledge Discovery from Data (TKDD), vol. 6, no. 1, March 2012.
5.J. C. Gower and P. Legendre, “Metric and Euclidean properties of dissimilarity coefficients” , J. Classification, vol. 3, no. 1, 1986, pp. 5–48.
6.L. Yu & H. LIU, “Feature selection for high-dimensional data: A fast correlation-based filter solution” . In Proceedings of ICML 2003. Washington, DC, USA, 2003, pp. 856–863.
7.L. Hubert, P. Arabie, “Comparing partitions” , Journal of Classification, vol 2, issue 1, December 1985, pp. 193-218.
8.M. Ring, F. Otto, M. Becker, T. Niebler, D. Landes, A. Hotho, “ConDist: A Context-Driven Categorical Distance Measure” , Machine Leraing and Knowledge Discovery in Databases, 2015 , pp. 251-266.
9.S. Boriah, V. Chandola, V. Kumar, “Similarity measures for categorical data: A comparative evaluation” , In: Proc. SIAM Int. Conference on Data Mining, 2008 , pp. 243–254.
10.S. C. Johnson, “Hierarchical clustering schemes” , Psychometrika, vol 32, no 3, September, 1967
11.V. Batagelj and M. Bren, “Comparing resemblance measures,” J. Classification, vol. 12, no. 1, 1995, pp. 73–90.
12.Z. Huang, “Extensions to the k-means algorithm for clustering large data sets with categorical values,” Data Mining and Knowledge Discovery, vol. 2, no. 3, 1998, pp. 283–304.
13.Z. Hubálek, “Coefficients of association and similarity, based on binary (presence–absence) data: An evaluation,” Biol. Rev., vol. 57, no. 4, 1982, pp. 669–689.
|Source URI: ||http://thesis.lib.nccu.edu.tw/record/#G0105354016|
|Data Type: ||thesis|
|Appears in Collections:||[統計學系] 學位論文|
Files in This Item:
All items in 政大典藏 are protected by copyright, with all rights reserved.