Quality of life (QoL) focuses on a person’s ability to live a fulfilling life. It helps in determining successful aging in the elderly population. Because of a scarcity of information regarding predictors of QoL in the elderly population in developing countries, this study was undertaken to assess the overall QoL and its predictors in the elderly population in an urban setting of Bangladesh. In this cross-sectional study, 275 participants were enrolled by systematic sampling at the outpatient department of a tertiary care hospital in Dhaka, Bangladesh. Data were collected by using a pretested semi-structured standard questionnaire for QoL (OPQOL-35). Pearson’s correlation was used to assess the association between QoL and sociodemographic factors. Multivariate linear regression was conducted to identify predictors of QoL, after controlling for potential confounders. The median age of the participants (n = 275) was 65 years (range, 60 to 85; 25th and 75th percentile, 60 and 68, respectively). The majority (78%) of them were married, and 20% were widowed or divorced. The median score of QoL was 113 (25th and 75th percentile, 101 and 124, respectively). Most of the participants had very poor (bad as can be) or poor (bad) QoL. Only 7.3% were observed to have “good” QoL (scores 140 or more). Females had significantly poorer QoL scores compared to males (p < 0.001). Age, gender, educational status, smoking, and sleep duration significantly predicted QoL in a multiple regression analysis. In conclusion, the QoL of the elderly individuals in Bangladesh was very poor or poor. Future research should focus on service-oriented interventions, especially targeting women. Counseling elderly people to enhance their quality of life may include improving sleep patterns, healthy diets, regular exercise, and caring for their mental health.