After reviewing how the Dirac delta contributions to the electrostatic and magnetostatic fields of a point electric dipole and a point magnetic dipole are usually introduced, we present an alternative procedure for obtaining these terms based on a regularization prescription similar to that used in the computation of the transverse and longitudinal delta functions. We think this method may be useful for the students in other analogous calculations.