The use of intelligent crop recommendation systems has become crucial in the era of smart agriculture to increase yield and enhance resource utilization. In this study, we compared different machine learning (ML), and deep learning (DL) models utilizing structured tabular data for crop recommendation. During our experimentation, both ML and DL models achieved decent performance. However, their architectures are not suited for setting up conversational systems. To overcome this limitation, we converted the structured tabular data to descriptive textual data and utilized it to fine-tune Large Language Models (LLMs), including BERT and GPT-2. In comprehensive experiments, we demonstrated that GPT-2 achieved a higher accuracy of 99.55% than the best-performing ML and DL models, while maintaining precision of 99.58% and recall of 99.55%. We also demonstrated that GPT-2 not only keeps up competitive accuracy but also offers natural language interaction capabilities. Due to this capability, it is a viable option to be used for real-time agricultural decision support systems.