I did a sizable study on historical pricing from data posted by Robert Shiller on his website at Yale. The market from 1991 on does not even belong in the mix of data, save the fact that it actually occurred. In this vein, there is no prior data to support owning stocks at the prices we have seen since and the data since 2000 proves this out. http://www.econ.yale.edu/~shiller/data.htm
That said, the data I have read over the years uses 2 base years, 1926, which was prior to the 1926-1929 bubble and 1950, which was the beginning of the post war bull market and had dividend yields at near record highs. To move the market merely to a normal yield would have meant a near immediate double.
The 1929 and 1966 peak in the SPX provided for 3% dividend yields on the broad SPX. The 29 market preceeded deflation and the 66 market preceeded about 20 years of high inflation, yet the peaks of each weren't permanently to date exceeded for near 25 years for the 29 market and about 15 for the 66 market, where the CPI more than doubled.
The dividend yield in March 2000 was 1.16% and dividends were 16.76. Dividends March 2010 were 21.91 and the yield was 1.90. I bring this up for 2 reasons. One is to show the most recent yield and the other to show what growth might have occurred from the top. Reducing this factor for a 10 year compound growth rate gives a growth rate of 5.06% added to the 1.16% dividend rate for a total return of 6.23. In order for that return to have actually occurred, the SPX would have to be priced currently at 2364. As we can see, this is a pipe dream price.
For the great unwashed, the above data tells me that if the dividend yield on the SPX was 6.23% March 2000, the price today would be the same as it was in 2000. I might note that the 10 year treasury yield was in this ballpark, so the performance on treasuries exceeded stocks. There is a formula that used to be taught in finance schools when they taught finance that for stocks P=D/(k-g). I believe the past 10 years is a better than average growth rate for 2 reasons. One, that dividends were given preferrential tax treatment since 2002 and the other, that we were in a massive credit bubble, which in itself inflates cash flow to corporations. Decades were dividends grow more than 1% more than inflation are the exception rather than the rule. The official compound CPI for the 10 year period was 2.43%, so we had a real rate of return on stocks of 6.23% minus 2.43% or 3.8%. In normal times, this is about what one would expect on a AAA bond, a far superior security to all but a handful of stocks.
That given, if the performance of the last 10 years was to be expected, we would have a return of 1.90% plus 5.06%, or 6.96%. This is hardly the 9% preached and being that inflation makes up 2.43% of the return, we have an expected real rate of return of 4.53%. Plus, if we normalize stocks for what they really are, instruments with an open end life, one has to realize the return can't be compared to an instrument like the 10 year treasury, but instead to maybe a 50 year bond.
The bulls will tell you that this doesn't count buybacks, but the SPX is adjusted for buybacks and the price return on holding stocks is still negative. What they don't bring up is the stocks that go to zero and we have had some big ones since 2000. AIG, FNM, FRE, WCOM, ENE, LEH, BS and GM to name a few. Also, shareholders have lost an apparent 75% of Citi. I must also note that buybacks have been used more to compensate management than to actually shrink the number of shares in the company.
I will go farther. Historical growth rates are much closer to 1% on real terms than 2.63% we saw over the past 10 years. Credit bubbles don't go on forever and taxes on dividends can only be reduced a little. If you take a 100 year sample on the Shiller data, it reveals a compound inflation rate of 3.14% and a dividend growth rate of 3.97%. This gives us a real growth rate in the range of .83% and I believe we are at the end of a rapid real growth rate for dividends that has skewed this number upward.
The buy and hold crowd will sell you diversification of the SPX. Then we have the stock picker crowd, which on a small scale can occasionally beat the SPX, but long term the SPX gives a fairly strong sample of what the whole scale of stocks return. Thus going back to the formula P=D/(k-g) we can add the numbers. Lets take the 9% history claimed for stocks. This would give us P=21.91/(.09-.04) (I rounded 3.97% to 4% to give a simple example, which will actually slightly inflate the price). This gives us a 20 times dividend multiple for the past 100 year average, so the proper long term price on the SPX should be 21.91 times 20 or 438.20.
There can be a million and one arguments, the best probably relating to the massive amount of stock that has been repurchased over the past 10 years or the PE's, but stocks are based on long term yield, contrary to anything to the contrary. Long term gains in stocks are largely inflation and really mean little. PE's are fleeting and I believe based more on inflation than actual performance, not to forget the nonsense accounting that has been adopted over the past 20 years or so. In 2003, the same analysis came up with a price estimate of 370, so the analysis is fairly stable, even if it missed the price by a long shot. Remember, we are in a debt bubble and the next 10 years will likely be worse on a real basis than the past 10, 20, 30 or maybe 100 years. Earnings on the SPX basically went to zero in the first quarter of 2009 and were only rescued by allowed accounting fraud, massive deficit spending and Ben Bernanke money manipulation. Even if we threw in another 10 points for dividends due to stock buybacks, it would only increase the reasonable value on the index by 200 points to 638.20.
I do believe Shiller has a reasonable valuation model in the PE/10 calculation for normal times. At present we have seen dividends shrink from a peak of 28.85 in September 2008 to the current 21.91, thus the valuation of stocks is currently on a downtrend, not an uptrend. This number as of March 2010 hasn't quit falling. Earnings peaked at 84.92 in June 2007 and the worsening data since will soon reverse the earnings factor that goes into this function.
There are 2 things that cannot be overlooked. One is the fact we are in a credit bubble and the trend has apparently reversed. This will serve over time to dry up money as time goes on and destroy not only the fodder for driving stock prices, but the material that makes earnings and dividends possible in the first place. Despite the reported reversal in earnings, dividends have shown no hint of ceasing their decline. The repercusions will be huge.
In short, if stocks don't resume a long term trend upward, then we have massive problems in the area of retirement income in the form of 401K's and defined benefit pension pools. My best guess is we are not likely to see a sustained price on stocks above the current 1075 for the next 10 years, which means the actuarial estimates for returns in pensions are going to prove excessive. Maybe we get another bull, but we will eventually have to cross the bridge again as stocks will return to this price level and quite likely lower. The Nikkei, a modern day marker of what will be seen in a deflated debt bubble resides now at 25% of its 1990 peak. We are nearing the 21st anniversary of that peak. Our peak was just short of 1600 and we are only 10 years into what is sure to be a long bear. 75% is just short of 400 on the SPX, which is reflected in my numbers above. But, Japan has hit even lower prices in the interim and it has already tried the reflationary tactics that are being tried now in the US. They didn't work there and I doubt they will work here. The same problem is now here that trapped the Japanese, peak home prices which have now drained the equity out of housing necessary to sustain consumer spending and to some extent, fund retirement. When it sinks into middle aged Americans that they can no longer depend on their homes to fund their spending and retirement and stocks to provide excessive teturns, we will see a permanent decline in spending for a good generation. What created the stock bubble in the first place will be gone.
Some will say that Asia will bail us out. What isn't realized by most is the huge credit bubble already present n Asia, not to mention the low wages paid, which will never replace western demand for goods and commodities. I highly doubt Asia can move their household incomes to a level that would replace western demand, not to mention the huge investment bubble demand already present. Contrary to popular belief, the Fed is not printing money for the public at large, but replacing bank liquidity that can no longer be manufactured by financial innovation in the financial industry. We are more at threat of that system breaking down than it gaining its legs and manufacturing a new steam of credit for expansion and consumer demand. As we deflate, stock earnings will disappear like water on a noontime summer sidewalk in Phoenix.