The stock market’s best and worst days often land side by side. Here’s the risk of walking away.