Recently, programing by demonstration (PbD) becomes a white-hot research area with increasing demands in the robot manipulation area, especially in industrial applications. However, the key challenges of PbD, to date, has been the recognition of demonstrator's fine hand motions and the uncertainty of manipulation environments, which limit the efficiency and accuracy of PbD. In this study, inspired by humanoid manipulation, a novel PbD approach using the implicit muscular task planning strategy is presented to extract features from the arms' giant movement and the hands' fine motions during the demonstrator's operation. Furthermore, we integrate a deep reinforcement learning control method that further improves the manipulations' adaptive ability in the unknown or changeable environments. The experimental results show that our proposed approach can deal with relative complex assembly task from single demonstration with more than 67% success rate under about 4.2 mm fit tolerance.