SLO is still pretty pricey compared with most parts of the US, but I agree that the climate and town are fantastic.
I live on an island NW of Seattle much of the year, and I find the weather reasonably mild. I’ve gone hiking on Orcas in January in shorts. It rarely snows. The cost of living is relatively high, but nothing like California. The local food is of a high quality and people are long-lived. Unlike California, there’s no income tax or capital gains tax, and the sales tax is comparable. Also, as a long-time former Southern Californian, I actually love rainy, cool weather in the winter and the opportunity to wear outerwear and wool.