Polarization calibration units (PCU) are critical in calibrating the instrumental polarization induced by the polarimetry system itself and the astronomical telescope. However, azimuth errors of optical axes of polarization components in the PCU are one of the main limitations to the calibration accuracy. To address this issue, an optimization approach based on constrained nonlinear minimization to derive azimuth errors in the PCU is proposed in this paper, this method has the advantages of high calibration accuracy and fast calibration speed. Specifically, azimuth errors of the polarizer and the quarter-wave plate are set as two free parameters to be optimized, and then an objective function for the optimization is defined by using generated and measured Stokes parameters and response matrices obtained from the polarization calibration procedure. Finally, values of the two azimuth errors obtained by the optimization are compared with their preset ones, and the optimization method is validated in the aspects of simulation and experiment respectively. Experimental results reveal that the two azimuth errors can be successfully determined by the optimization, with an accuracy better than 2.79$'$ and 2.72$'$, respectively. Furthermore, the influence of azimuth errors on the Stokes parameters is theoretically analyzed. The optimization method presented in this paper is expected to be applied to the error calibration and development of polarization calibration devices in solar telescopes in our country.