Important Disclaimer: Ken Kam, Marketocracy Data Research's Editor in Chief, also is portfolio manager for mutual and hedge funds advised by a Marketocracy affiliate. Before relying on his opinions, always assume that he, Marketocracy, its affiliates and clients have material financial interests in these stocks and hold or trade them contrary to those opinions. Continue reading for more detailed and important disclosures, disclaimers, limitations and material conflicts of interest.

Continue reading "Disclaimer" »

January 8, 2009

Luck vs Skill

In concept, an investor has a universe of stocks from which to select a portfolio. If an investor’s portfolio delivers a higher return than a portfolio consisting of the entire universe of stocks, then the excess return — also known as “alpha" — can be attributed to the investor’s skill. Investors who don’t deliver alpha can be excluded from further consideration.

Of the investors who generated alpha, it is certainly possible that any of them could just be lucky. However, the larger the alpha, and the longer the track record, the less likely that is the case.  And, because we have so much more data, we have developed other metrics that increase our confidence that we are finding skill.

Improving Confidence

At the heart of it all, skilled investors have to do two things well — pick the right stocks, and trade them well.

To tell us whether an investor is skilled at picking stocks, we use a metric we call the winning percentage. It is simply the number of profitable stocks divided by the total number of stocks that an investor has ever put into their model portfolio. An investor with a winning percentage of 85% is someone who puts a lot of effort into stock selection and is doing a very good job.

To determine whether an investor is skilled at trading we use a metric we call the average gain to average loss ratio. We calculate it by taking the average gain from all the winners and dividing by the average loss from all the losers. A ratio of 2 means that the investor makes twice as much money when they are right about a stock as they lose when they are wrong. This investor is doing a good job of selling the losers before the losses become large, while also letting the winners run.

Putting it Together

Consider two investors who each delivered the same alpha over the last five years. Many would say they are equally skilled but also equally likely to be lucky.

But, what if I told you that one investor’s alpha came from a single trade (buying Google at its IPO) and the other made hundreds of trades of which 85% of the stocks were profitable, and the average gain to average loss ratio was 2? Would this information change your confidence in one versus the other?

It does for me. Although we use alpha as a measure of skill, it is not a good estimate of future returns because it is the result of past opportunities that may no longer be available. We should not expect that future opportunities will be the same as in the past.

However, a winning ratio of 85% and an average gain to average loss ratio of 2 tells us that the investor has good judgment about stock selection and trading decisions. I have more confidence that the investor will apply the same judgment in the future as in the past, than that he will achieve comparable alpha.

It is metrics like these (and others) that go well beyond what is typically available about mutual fund managers that improve our confidence that the investors we are selecting are skilled.

July 30, 2008

Elan's Clinical Trial Results

The objective of a clinical trial is to see if patients treated with a new drug (the “treatment group”) do better than those who do not (the “control group”).  In most clinical trials, the control group is given a placebo so that any differences in the outcomes between the two groups can be attributed to the new drug. For this trial, however, Elan did something I think more companies should do. They gave the control group Aricept, which is the best available drug currently on the market instead of an inert placebo (which would be really cruel). This is the right thing to do for the patients in the control group, but it means the difference between the treatment group and the control group will be smaller than it otherwise might have been.

In order for a trial to be “successful”, the difference between the treatment group and the control group must be large enough to refute the possibility that it arose solely due to randomness. Scientists and regulators have generally agreed that if the difference is big enough so that there is less than a 5% chance that it is was a fluke of luck, then the trial has succeeded.

In Elan’s clinical trial, patient outcomes were measured in two ways; ADAS-cog and NTB. Using ADAS-cog, the difference between the treatment group and the control group is +2.3 (p=0.078). The “p value” of 0.078 means that probability that the difference could have arisen by chance is 7.8%. On the NTB scale, the difference was +0.13 (p=0.068).

Since there is more than a 5% chance on both measurement scales that the difference could have arisen by chance, it is fair to say that the trial did not succeed by generally agreed upon standards. After the announcement of these results, Elan’s stock dropped more than 40% which would be understandable if the drug was indeed a failure.

However, I don’t think it is accurate to conclude that the drug was a failure.

Using the ADAS-cog it would be accurate to say that we can be 92.2% confident that the differences were not due to chance. Using NTB, we can be 93.2% confident. In other words, using either measure we can be more than 90% confident that the drug had a beneficial impact over the best available drug on the market. Is this a failure? I don’t think so.

The results also show that patients who carry a gene called Apoe4 respond differently than those who do not. There were 79 patients in the trial who do not carry the gene. Of these, 47 were in the treatment group and 32 in the control group. On ADAS-cog, the difference between the two groups was +5.0 (p=0.026). On NTB, the difference was +0.35 (p=0.006). In other words, for patients who don’t carry Apoe4, the drug worked.

Everyone would now be celebrating that the drug worked for non-carriers of Apoe4 except for the fact that Elan did not specify at the start of the trial that they were going to separate the results between carriers and non-carriers. Instead, critics have accused the company of being dishonest with the statistics.

In truth, clinical trial data can often be twisted and contorted until the drug shows efficacy for some subgroup of patients. For example, it could turn out that the data shows the drug was effective for patients who are more than 6 feet tall. But, if it did, people would be justified in ridiculing the results because there is no reason to believe that efficacy should be related to a patient’s height.

The difference in results for carriers and non-carriers of Apoe4, however, is not so easy to dismiss. Apoe4 is suspected to be involved with Alzheimer’s disease (we don’t know exactly how) so it is not at all surprising to learn that a drug that might interfere with the progression of Alzheimers might affect carriers of the gene differently than non-carriers. The ridicule that the company has received for reporting this result is not deserved.

In the end, I think investors can be a little more than 90% confident that Bapineuzumab has a beneficial impact on Alzheimer’s patients above and beyond the best available medication. In addition, I think its fair to say that we have learned from this trial that there is a genetic marker that can be used to identify patients for whom Bapineuzumab may have an even better chance of working.

The Phase 3 trial was clearly designed with this lesson in mind. If the Phase 3 trial results confirm efficacy for just non-carriers of Apoe4, there are so many Alzheimer’s patients that this drug would still be a blockbuster.

I think Elan’s current price can be justified on the basis of their multiple sclerosis drug (Tysabri) and their drug delivery business. This means, that if you buy Elan now, you are essentially paying nothing for Bapineuzumab. That’s a bargain.

[marketocracy]
Marketocracy Rules | Privacy Statement | Services Agreement | Questions

Copyright © 2001-2014. Marketocracy. All rights reserved
Marketocracy, the Marketocracy logo and m100 are service marks of Marketocracy, Inc.
Patents Pending

All quotes delayed 20 minutes