The utility and adoption of indoor localization applications have been limited due to the complex nature of the physical environment combined with an increasing requirement for more robust localization performance. Existing solutions to this problem are either too expensive or too dependent on infrastructure such as Wi-Fi access points. To address this problem, we propose APFiLoc—a low cost, smartphone-based framework for indoor localization. The key idea behind this framework is to obtain landmarks within the environment and to use the augmented particle filter to fuse them with measurements from smartphone sensors and map information. A clustering method based on distance constraints is developed to detect organic landmarks in an unsupervised way, and the least square support vector machine is used to classify seed landmarks. A series of real-world experiments were conducted in complex environments including multiple floors and the results show APFiLoc can achieve 80% accuracy (phone in the hand) and around 70% accuracy (phone in the pocket) of the error less than 2 m error without the assistance of infrastructure like Wi-Fi access points.