Internet of Things (IoT) refers to a wide variety of embedded devices connected to the Internet, enabling them to transmit and share information in smart environments with each other. The regular monitoring of IoT network traffic generated from IoT devices is important for their proper functioning and detection of malicious activities. One such crucial activity is the classification of IoT devices in the network traffic. It enables the administrator to monitor the activities of IoT devices which can be useful for proper implementation of Quality of Service, detect malicious IoT devices, etc. In the literature, various methods are proposed for IoT traffic classification using various machine learning algorithms. However, the accuracy of these machine learning algorithms depends on the data generated from various IoT devices, features extracted from network traffic, site at which IoT is deployed, etc. Moreover, the selection of features and machine learning algorithms are manual operations that are prone to error. Therefore, it is important to study the network traffic characteristics as well as suitable machine learning algorithms for accurate and optimized IoT traffic classification. In this article, we perform an in-depth comparative analysis of various popular machine learning algorithms using different effective features extracted from IoT network traffic. We utilize a public data set having 20 days of network traces generated from 20 popular IoT devices. Network traces are first processed to extract the significant features. We then selected state-of-the-art machine learning algorithms based on the recent survey papers for the IoT traffic classification. We then comparatively evaluated the performance of those machine learning algorithms on the basis of classification accuracy, speed, training time, etc. Finally, we provided a few suggestions for selecting the machine learning algorithm for different use cases based on the obtained results.