Human activity recognition (HAR) using inertial sensors is essential for health monitoring and wellness applications, yet robust classification in real-world adult scenarios remains challenging due to subject variability and activity transitions in smartphone sensing environments. This study investigated smartphone-based physical activity recognition using accelerometer and gyroscope signals under a cross-subject evaluation protocol. To reduce label ambiguity and improve generalization, the original activity set was grouped into a reduced 6-class taxonomy. We evaluated lightweight deep learning models, including a smartphone-only convolutional neural network (CNN) and a multimodal fusion model combining smartphone and smartwatch signals. Using GroupKFold cross-subject validation, the smartphone-only CNN achieved competitive performance with Macro-F1 ≈ 0.46, while multimodal fusion did not provide consistent improvements. We also examined temporal segmentation and showed that shorter windows (2.0 s) yield better results than longer windows. Sensor ablation confirmed the importance of gyroscope information, and per-class analysis indicated that dynamic activities could be recognized reliably, whereas stairs and static categories remained difficult. Overall, the results demonstrate the practicality of smartphone-based activity recognition using built-in smartphone sensors without external wearable devices for adult activity monitoring and provide recommendations for window length and sensor selection in cross-subject HAR.