Everyone's watching the 10-year yield, but this is the real rate that matters

While the 10-year Treasury note is garnering all the headlines, investors might be better off watching much shorter duration bonds to figure out what's really happening in the markets.

The government's benchmark debt instrument saw its yield pass 3 percent Tuesday, a four-year high that ostensibly helped to trigger a violent stock market reversal that saw the Dow industrials close lower by about 425 points.

The calculus behind fear of the 3 percent yield seems obvious: With the S&P 500 dividend yield at 1.9 percent, a risk-free investment like U.S. Treasurys yielding 3 percent makes more sense in a volatile environment.

But that reasoning is weak. The play assumes holding the bond to duration and clipping coupons, and the stock market has never shown inflation-adjusted returns that low over a 10-year period. Absent a major crash and a deep recession it likely won't over the next decade as well.

The next two years, though? That could be a different story.

While everyone on Wall Street is pounding the table over the rising 10-year yield, the 2-year note rose above 2.5 percent Wednesday, a level it last closed at August 2008, just a month before the financial crisis imploded with the collapse of Lehman Brothers.

A risk-free investment with a 2.5 percent yield over two years? That seems a little more reasonable. Investors who bought the 2-year in mid-2006 would have gotten it at 5 percent, ahead of a stock market that was about to drop 60 percent.

"As much as every investor knows market timing is very difficult, that's the sort of case study that resonates just now," Nick Colas, co-founder of DataTrek Research, said in his daily note Wednesday.

Investors have been testing the waters over the past month, yanking $868 million out of U.S. equity ETFs while pouring $5.2 billion into funds that invest in fixed income with duration of less than three years, Colas said, citing XTF data. The iShares Short Treasury Bond fund, which focuses on fixed income with duration between one and 12 months, alone has pulled in $3.4 billion over the past month, according to FactSet.

There are other explanations for the move, of course, with a big one being a brace for increasing inflationary pressures. Bond investors generally will shorten duration when they see price pressures coming, on fears that inflation will eat away at capital over longer periods. Yields and prices move in opposite directions.

But the move also comes at a time of accelerated volatility in the stock market as investors worry over how much longer the nine-year-old bull market has to run.

"If I offered you a no-fee risk-free contract to deliver a 2.5% annual return on the S&P over the next two years, would you take it?" Colas said. "On the one hand, it's well less than the historical market return. But … there's no risk of drawdowns and at least it's better than 2% inflation. You might be tempted."

The events all come following an extended period where investors were pushed out of low-yielding cash and its equivalents — like government bonds — and into the risk of equities and corporate debt.

With the pendulum swinging as higher rates set in, the dynamic is changing.

"The cost of sitting on the sidelines in cash has gone down," said Matt Toms, chief investment officer for fixed income at Voya Investment Management. "In the past years of quantitative easing, there was a penalty to not taking risk."

"Bond yields were low, cash yields were low and you were forced into the risk products," he added. "What you're seeing now is you can have a reasonable chance of 4 to 5 percent on income without the same volatility as equities. I do believe that influences equity valuations."

He cautions, though, that investors may be getting ahead of themselves on inflation fears.

Though on a consistent course higher this year, the 10-year could run out of steam around 3.25 percent, Toms said. After that, investors likely should lock in their duration plays and look to the shorter end of the curve. That in turn could put a lid on yields for the 2-year.

"We think it has more legs to it," Toms said of the desire for shorter-duration bonds. "Bias towards shorter-duration strategies in the near term makes sense. You've got not that much farther to run."

WATCH: How to invest with bond yields rising.