Within an Internet of Multimedia Things, the risk of disclosing streamed video content, such as that arising from video surveillance, is of heightened concern. This leads to the encryption of that content. To reduce the overhead and the lack of flexibility arising from full encryption of the content, a good number of selective-encryption algorithms have been proposed in the last decade. Some of them have limitations, in terms of: significant delay due to computational cost, or excess memory utilization, or, despite being energy efficient, not providing a satisfactory level of confidentiality, due to their simplicity. To address such limitations, this paper presents a lightweight selective encryption scheme, in which encoder syntax elements are encrypted with the innovative EXPer (extended permutation with exclusive OR). The selected syntax elements are taken from the final stage of video encoding that is during the entropy coding stage. As a diagnostic tool, the Encryption Space Ratio measures encoding complexity of the video relative to the level of encryption so as to judge the success of the encryption process, according to entropy coder. A detailed comparative analysis of EXPer with other state-of-the-art encryption algorithms confirms that EXPer provides significant confidentiality with a small computational cost and a negligible encryption bitrate overhead. Thus, the results demonstrate that the proposed security scheme is a suitable choice for constrained devices in an Internet of Multimedia Things environment.