Understanding the contact dynamics of nonspherical particles beyond the microscale is crucial for accurately modeling colloidal and granular systems, where shape anisotropy dictates structural organization and transport properties. In this paper, we introduce an energy-conserving contact dynamics framework for arbitrary convex rigid-body particles, integrating vertex-boundary interactions in 2D with vertex-surface and edge-edge detection in 3D. This formulation enables continuous force evaluation and strictly prevents particle overlap while conserving total energy during translational and rotational motion. Simulations of polygonal and polyhedral particles confirm the framework's stability and demonstrate its capability to capture packing behavior, anisotropic diffusion, and equations of state. The framework establishes a robust and extensible foundation for investigating the nonequilibrium dynamics of complex nonspherical particle systems, with potential applications in colloidal self-assembly, granular flow, and hydrodynamics.