Many years ago, more than a decade ago in fact, I wrote a column which was a kind of a mashup of the maths and probability as they affect equity research and lab tests for diseases. I don't remember now what the context of the disease testing was. I was reminded of it a couple of days ago, while reading an article by an epidemiologist about the maths of widespread testing for Covid-19.
The idea was just to explain some basics of why equity research works sometimes and does not work at other times by thinking of it like a test for a disease. Think of equity research like a diagnostic test. The way a lab test is designed to detect disease, equity research is designed to detect which companies' stocks will generate good returns. I know diseases are bad and returns are good but this is about the testing process so bear with me please.
Just like a diagnostic test for a disease, equity research has false positives and false negatives. In a medical test, a false negative means that the person being tested actually has the disease but the test process concludes that there is no disease. A false positive means that the person does not have the disease but the test results declare him or her to have the disease. In my analogy of equity research being a diagnostic test, a false positive means a company that is a poor investment, but the test turns it up as a good one. A false negative is of course the opposite.
What had set me to think about this all those years ago was a chat with a youngster who worked with a big name national-level stock broker. His job was to deal with clients on the phone, give them advice about which stocks to buy and sell and execute the orders. The advice was based on whatever the research department would feed him. As the boy put it, after January 2008, the advice became useless. "Our analysis stopped working," was what he told me. When the markets were rising strongly, the research department's analysis used to work and after that it stopped working. At about the same time, the Chief Investment Officer of a large mutual fund company also told me that the 'accuracy of their analytical models had declined sharply' which was a fancy way of saying the same thing. In the term we have defined above, the false positive rate shot up hugely.
Why did that happen? Let's put it in terms that I read in the article about Covid testing. The impact that false positives have on the actual, delivered accuracy of any testing or selection process depends on how widespread the condition is in the population being tested. The article explained that if the real underlying rate of infection in the population being tested was, say 10 percent, and the test had a false positive rate of 3 percent (which is said to be the case), then 2.7 per cent of the test results would be false positives. However if you were testing very widely, and the infection rate in the tested population was 1 percent, then almost all the test results would be false positives.
So to get back to equity research, let's generously estimate that the equity research department's level of false positives was 30 per cent. When the markets were rising, perhaps 10 out of 100 companies' stock prices were going to do badly in the short run. So the false positives would be three companies in all. When the market started falling, then perhaps 80 companies out of 100 would do badly. A research process of the same accuracy would now produce 27 false positives.
The moral of the story is that it's simply not possible to judge any research or investment management process unless you evaluate it through an entire market cycle. There are errors and mistakes in any diagnostic process. Unless one is aware of this, and it's potential impact, the conclusions that come out of the process could be pretty misleading.
