Aim. The aim of this paper is to present the results of a pilot validation of an artificial intelligence chatbot for mathematics tutoring. The subject of the analysis is the behaviour of the students in the course, in particular their approach to solving problems without external motivation through scoring or other assessment. Methods. The analysis of student behaviour was carried out on the basis of records in the application database. The subject of the analysis was not only the pupils' responses, but also the time taken to produce them and the overall trajectory through the course. Results. The results of the analysis show that students who choose to join the course, work intensively in the lessons. Most pupils solve more than one basic task. A significant proportion of pupils also use the prepared help and instructional videos. The analysis of the solution time allowed identification of a group of pupils who do not even read the problems and only confirm the results. Conclusion. The pilot validation confirmed that the chat-bot format is close to the learners, and they are able to use the courses created in this environment. Solution time analysis suggests that this parameter can be used to analyse learner behaviour and system control. Further analyses in this area are needed.