A model of double grains under plane stress state has been established. According to the double grain model, thermal stress induced by thermal cycling in welding fusion zone is numerically simulated by finite element method, and the microstructures before and after thermal cycling are observed. The effect of thermal stress on weld microstructure is discussed. Experimental and analysis results show that the difference between the coefficients of thermal expansion and elastic modulus for grains along different crystal direction can produce alternate thermal misfit stress and strain near boundaries under thermal cycling. At the temperature of upper and lower limit, thermal stress nearby grain boundary reaches maxima. Thermal stress induced changes in microstructure, which expressed by the sending dislocations from boundaries to matrix, piling up against the boundaries and the increasing of dislocation density.