Based on the provincial annual drought disaster losses and socio-economic data of China from 2004 to 2019, spatio-temporal variation characteristics are analyzed for the three loss indexes and their loss rates, including the crop area, population affected by drought and direct economic loss caused by droughts. Risk assessment of drought loss rates for different hazard bearing bodies are carried out by the information diffusion method so as to understand the drought risk pattern and provide reference for improving the ability of drought risk management. The results show that the annual disaster loss rates of the three indexes caused by drought tend to decrease in China during 2004 and 2019, in which the loss rates of crop area and population show decreasing trends significantly. This is the result of the increase of annual precipitation, decrease of annual drought cumulative intensity and days with moderate drought grade and above, as well as the enhancement of drought resistance capacity. The drought risk patterns of the three types of hazard bearers are different. When the annual rate of crop area affected by drought to sown area is greater than and equal to 20%, only Ningxia, Shanxi, Inner Mongolia and Gansu have a higher exceeding possibility level. When the rate of population affected by drought to total population is greater than and equal to 30% and the rate of direct economic losses to GDP is greater than and equal to 1%, the exceeding probability is small and there is no province with high or higher possibility grade. Under the level of 20-year return period of annual rate of crop area affected by drought to sown area, the serious drought area distributed in the whole Northwest China, eastern Southwest China and the provinces of Hubei, Hunan, Zhejiang and Hainan. Under the level of 20-year return period of annual rate of population affected by drought to total population, the serious drought area was located in the middle and east of Northwest China, eastern Southwest China, and the provinces of Anhui, Shanxi, Inner Mongolia. Under the level of 20-year return period of annual rate of direct economic losses to GDP, the serious drought area is located in Northeast China, central Northwest China, eastern Southwest China as well as the provinces of Inner Mongolia, Hainan.