Abstract: In order to improve the accuracy of the temperature measurement in bypass of the reactor coolant system in nuclear power plants, MATLAB optimization toolbox Lsqnonlin function by the nonlinear least square method is used to make a nonlinear curve fitting to build the relational function of resistance-temperature. Calculate and analysis the deviation between the temperature value through the relational function and the measurement value of the standard thermometer. Results show that the relational function of resistance-temperature established by the nonlinear least square method can improve the accuracy of the temperature measurement throughout the whole measurement range for one order of magnitude.