The Wild Nature of Correlations
I was recently inspired by this paper by Carlton J. Chin, CFA. He and his colleagues wanted to test the relationship between different assets using correlation. But more specifically how correlation differs between different time intervals. That is correlation will change wildly when using daily, weekly, monthly and annual returns. As he highlights, this is an important question to answer because so much of asset allocation has to do with not only expected returns and volatility, but correlation.
There is so much data out there these days you can literally test any hypothesis, so in the spirit of being a studious financial analyst I wanted to (somewhat) replicate the findings in the paper using Python.
The paper uses daily data and looks at correlations between the S&P 500, bonds, Japanese equities, and commodities over a 10-year period. I believe sometime between 2002 and 2012. Given that I want this to be freely available to everyone, I am only going to use free data from Yahoo Finance in my Python notebook. I’ve shared the notebook above. I have data for the S&P500, Bloomberg Aggregate Bond Index (AGG), Russell 2000, Japan (EWJ), and commodities (GSCI) between 2004 and 2023. As done in the paper, I’ve calculated correlation using a 1 day interval, 2 day interval up to 365 day interval for each asset pairing with the S&P 500.
Lets first look at the S&P500 vs small cap stocks, in this case the Russell 2000.
The correlation swung between a minimum of 0.76 on day 178 and a maximum of 0.96 on day 236. The standard deviation is 0.027. But if you think that’s wild take a look at the correlation between the S&P500 and bonds.
The variance of correlation is all over the place. The minimum is -0.48 on day 333 and a maximum of 0.49 on day 159. The standard deviation is 0.17. Notice the larger deviation as compared to small cap stocks. The consistency of the correlation between large cap stocks and bonds is less as compared to the correlation between large cap stocks and small cap stocks.
Now lets look at Japan.
The minimum is 0.55 on day 288, the maximum is 0.92 on day 328, and the standard deviation is 0.059. This variable, but much less variability as compared to stocks and bonds.
Finally, commodities.
The minimum is 0.22 on day 327, the maximum is 0.70 on day 258 and the standard deviation is 0.091. Again a large variance.
My findings look to be consistent with the authors. You will notice that calculating correlation on shorter time intervals i.e., 1 month or roughly 20 trading days, have much smaller correlations as opposed to longer intervals. Longer intervals i.e., 1 year or roughly 250 trading days, have less data points and greater variability of the correlation statistic.
My key takeaway is this, if you are going to project correlations into the future using an asset class model like MVO or something similar, it’s probably best to stick with conservative estimates of correlations using shorter intervals that are consistent over many years. Even better if you can find correlations that are consistent across different financial regimes. That is not to say those correlations will hold in the future, but the empirical probability gives greater evidence to hold that view going forward.
In the next post I want to test a few additional hypotheses proposed at the ending of the paper. For one I want to test the correlation between stocks and treasuries and bonds during market drawdowns, specifically before, during and after a recession.
Thanks for reading! Happy Investing!