Based on the parallel daily air temperature data of automatic and manual observations at 143 national benchmark stations from 2002 to 2010, systematic comparative analysis and objective evaluation of differences are made, especially focusing on the large differences and their causes. The impact of automatic observation on the homogeneity of temperature time series is evaluated using the penalized maximal t test (PMT) combined with the metadata of observation instrument changes. The results show that: (1) 51.29%, 54.14%, and 67.18% of daily average, highest, lowest temperatures obtained by AWS (automatic weather station) are greater than the values by manual observation and the percentage of difference between ±0.2℃ respectively accounts for 78.8%, 63.1%, and 60.9%. Average difference values of daily average, highest, lowest temperature are 0.05℃, 0.09℃ and 0.15℃, and the standard deviations are 0.14℃, 0.22℃ and 0.15℃, respectively. The difference, absolute difference value and standard deviation of all temperature elements have no apparent increasing or decreasing trend along with the observation time of AWS and their spatial distributions are different. (2) By the classification comparison and screening of the difference value, absolute difference value and standard deviation step by step, some differences are found greater at a few stations and the differences of the average, the highest and lowest temperatures collected from the same sensor are different as well. By the check of PMT, the inhomogeneous breakpoints are found in the monthly average temperature time series, monthly average maximum temperature time series of 35% station and monthly average minimum temperature time series of 25% stations among the 20 stations with largest absolute difference values of average, maximum and minimum temperatures. (3) The change of calibration error of temperature sensor is the important reason for difference jump between automatic and manual observations. The instrument failures, such as zero drift of electronic components of temperature sensor or data collector can lead temperature obtained by AWS to deviate greatly from the value of manual observation. The above two facts are the main causes for greater differences in temperature between automatic and manual observations, and also possible reasons for inhomogeneous breakpoints of temperature series because of observational instrument changes. Therefore, we suggest strengthening monitoring and quality control for automatic observation data, increasing the observation instrument calibration error correction and realizing the zero temperature compensation adopting methods of hardware and software compensation to reduce or eliminate the instrument error as much as possible and improve the accuracy of the automatic observation data.