In nuclear power plants, the stress corrosion cracking (SCC) is the main reason to cause the cracks in the control rod drive mechanism (CRDM) nozzle with the J-groove attachment weld to the reactor pressure vessel (RPV) head. However, SCC is mainly driven by welding residual stresses. Thus 2D axisymmetric model is used to study the weld residual stress distribution in J-shaped groove. In order to obtain a simple, efficient and conservative method, the effect of simplification of heat source, simplification of weld geometry, weld material yield strength, phase transformation, and material hardening behavior on the welding residual stress is investigated. The research result shows that the double ellipsoid heat source and uniform heat source lead to the same weld residual stress distribution; the simplification of scale weld model to block weld model has little influence on the result, but the simplification to dumped weld model could produce relatively great variation on weld residual stress distribution; low yield strength could not obtain the conservative result; in ANSYS, the phase transformation has no effect on the stress result; the result by the isotropic hardening behavior is higher about two times than that by kinematic hardening behavior; for the welding simulation in engineering, the uniform heat source, block weld model, and isotropic hardening behavior are suggested.