In the era of big data, the continuous generation of data from various fields has resulted in large and complex datasets. These datasets often come in diverse formats and structures, including unstructured or semi-structured data. Despite the wide availability of big data, high dimensionality remains a significant challenge for analysing and understanding the data for various purposes. Clustering analysis plays a crucial role in data analysis and visualization by uncovering hidden patterns and structures within datasets. However, several challenges hinder the effectiveness of clustering analysis, including data dimensionality, selection of appropriate clustering algorithms, determining the optimal number of clusters, interpreting the results, and handling outliers. This paper aims to explore these challenges and presents preferable visualization techniques that aid in visualizing and interpreting clustering results. By addressing these challenges, including the difficulty of handling outliers and the struggles with high-dimensional datasets, and employing effective visualization techniques, researchers and practitioners can enhance their understanding and utilization of clustering analysis in data analysis.
4 September 2024 in Journal of Advanced Research in Applied Sciences and Engineering Technology
High-dimensional data is collected from various sources, fields, and applications such as medicine, science, business and more to provide helpful information to others. Unfortunately, the complexity of high-dimensional data has made it difficult to interpret and understand. As a result, sophisticated data analysis is required to extract knowledge and information from it. This can be illustrated through a visualization presentation. However, overlap between data can occur during visualization as data increases. Indirectly, it can cause a cluttered visual presentation. As a result, it affects the visual perception of high-dimensional data patterns. High-dimensional data can be deeply explored using dimension arrangement and scaling to overcome it. The arrangement of dimensions is essential since the relationship between these dimensions can influence the existence of an efficient cluster. This dimension is arranged based on the correlation value. The dimension that is more related will be placed next to each other. While performing clusters, dimensions will be scaled in or out. These features are available through Star Coordinate (SC) technique. This paper aims to conduct an exploratory data analysis in the SC environment where users can visualize and interact in a low-dimensional data visualization space. This paper demonstrates data dimensions manipulation's importance in structuring the projected space layout using two data sets. As a conclusion, formation of clusters was crucial and manipulation of data dimensions were essential to structure the projected space layout. The proposed approach has helped users find significant cluster formations by randomizing the scaling and order of dimensions.