Using records from more than one million trees across North America, we show that the effects of warming and drought on forest insects and pathogens depend strongly on historical climate, host species and pest identity. Under the SSP2-4.5 scenario, incidence is projected to increase across 58% of forest land for insects and 52% for pathogens, with the largest increases at high latitudes.