Performance report rule changes?

New years and fresh starts are synonymous. There is something about the passing of that one particular, loaded second, from one year to the next, that draws our minds from the moment itself and out into the future. Past mistakes are forgotten as we set out to improve on what has gone before.

That’s a grand way of introducing to you our intention to make an important change to that lengthy annual tome, our Performance Report. It may not be our New Year’s resolution, but it is a proposition, and one on which we want your opinion.

This is a deeply vexed issue and one we’ve been contemplating for some time. But before explaining the rationale for the changes, let’s examine why the Performance Report exists at all.

The finance industry talks a good game on transparency and openness but rarely delivers. Intelligent Investor’s Performance Report is, we believe, a pretty good stab at genuine transparency. It lists every recommendation ever made, good and bad, exceptional and appalling, or merely average.

Absolute disclosure

But it isn’t just the final numbers that tell you how we’ve performed across all the recommendations we’ve made; it’s about more than transparency—it’s absolute disclosure. On that score, the current report measures up pretty well.

Whether it does the job most members actually ask of it is another question. And that question is this: If I had subscribed to Intelligent Investor and acted on a representative cross section of their recommendations, how would I have performed?

The current method for calculating performance is quite simple. Dividends received during the life of a recommendation are added to the price at which the stock is sold. This figure is then divided by the purchase price to find the total return, from which the compound annual return is calculated. This is done for each change in positive recommendation. The overall performance figure is the arithmetic average of all those individual returns. So far so good.

Trouble is, the reported performance using this method can be quite different to the actual performance you might get from actually following our recommendations.

Time value of dividends

Firstly, the existing methodology ignores the time value of dividends. Consider our Buy recommendation on Macquarie Group (then Macquarie Bank) on 21 May 2001. A $1 dividend received in 2002 is clearly worth more than a $1 dividend received today, but the current method doesn't distinguish between the two, assuming each are received when the investment is sold. That means reported returns will be lower than actual returns.

More importantly, an arithmetic average isn’t as good a way of measuring compounding investments as it appears. The reason for that is best demonstrated with an example.

Table 1 shows three recommendations that appeared in 2010’s Performance Report. As you can see, the individual annualised returns average out to -2%. That appears to be a very poor result.

But take a closer look. This simple average doesn’t take into account the duration of investment. Over almost nine years, Computershare generated returns of 23% per annum. But this great long term performance is totally wiped out by Strathfield’s poor short term performance (a loss of 58% in just one year).

For an investor, the fantastic performance of Computershare more than makes up for Strathfield’s losses. But not when we measure it using our current performance report methodology. Although our investment has returned $8543 from an outlay of $3,000—nearly tripling an original $3,000 investment—the measured return as it would appear in our performance report is -2% a year.

Something is clearly wrong. If you'd invested $1,000 in a business that declined 20% per year for five years, you'd reclaim just $328 of your investment. A 20% decline after one year leaves you with $800. Under the current methodology, both outcomes would register the same percentage decline even though they’re very different in dollar terms.

As Table 2 makes clear, it also works the other way. Roc Oil was a recommendation we got very wrong. But because we recommended it all the way down over a period of nearly seven years, the current performance report methodology actually understates the effect of this terrible recommendation.

A better way?

We propose calculating performance using an internal rate of return (IRR), described in detail in How to calculate portfolio returns. Put simply, an IRR accounts for the amount of money you have invested and the compounding of gains or losses as time passes.

There are a few major advantages to using an IRR calculation. First, it correctly weights recommendations according to their duration; no longer will a stock which falls 30% over one year be equal to a stock which grows at a compound rate of 30% for eight years.

Secondly, as dividends appear as positive cash flows in the IRR calculation, their time value will now be accounted for.

Finally, and most importantly, the IRR makes intuitive and logical sense: Summing up the total cash flows into and out of our portfolio in Table 1 and running an IRR calculation yields a 21% return, which seems reasonable when we’ve invested $3,000 and received $5543 over nine years.

In Table 2 it has the opposite and equally realistic impact: our original $3000 investment returned only $1432 by throwing good money after bad.

Mathematically at least, the proposed methodology is clearly a better approximation of reality. While accepting that all methodologies are flawed one way or another, the new approach is more accurate than the old. It’s the effect of this change that is more problematic.

That brings us to three final questions. First, if adding up all the dividends to establish the total cash flows for every recommendation on which an IRR calculation is performed makes such intrinsic sense, why did we not use this methodology from the very start?

That’s a bloody good question. Believe me, we’re kicking ourselves. The answer I believe is one (small) part simplicity and one (greater) part lack of forethought.

Prior to the use of Capital IQ data—the suppliers of our financial history data—the proposed new method would have required about two to three months’ time of one very smart person. With the availability of this data, it will in future take about two weeks.

Ill-conceived

More importantly, the issue of an ill-conceived methodology arises only as time passes. In the first 12 months of the life of our very first performance report, the difference between the old and proposed new methodology would have been marginal. But with each passing year, the performance report takes a greater step away from reality. It’s an issue we should have anticipated right from the start but didn’t.

The second key question relates to the effects of a revised methodology on past recommendations performance.

There’s no mistaking the fact that this is an uncomfortable issue. Any change in the ‘rules’, especially if it has a favourable impact on an important outcome, is (and should be) viewed with suspicion. Subject to a professional auditor inspecting our figures, this particular rule change has the effect of increasing our recommendations performance by around 5% per annum over the life of the report.

Had we made more loss-making recommendations over time than we had successful ones the effect would have been reversed and the change would have a negative impact. But the fact remains that a change like this deserves genuine interrogation, not just by auditors but also by you, our members. Please pose your thoughts and questions at Ask the Experts or email our research director at researchdirector@intelligentinvestor.com.au.

The final question is really a statement: While we believe an IRR methodology offers a better approximation of reality, it isn’t perfect.

No one in the their right mind would act on the 500-odd recommendations made over the past 10 years. The performance of your portfolio will be determined by the recommendations you do act on and the percentage of your portfolio you allocate to them.

Your say

Unfortunately, and much as we’ve tried, there’s no way to accurately account for these variables in a comprehensive performance report. If you have any ideas, please let us know.

The proposed change is, we believe, a big improvement but remains a long way from perfection, which is one of the reasons why we also run two model portfolios that, incidentally, use the IRR methodology and have done since their inception. But we think the proposed change offers a more realistic picture of our stock-picking prowess over time, and relative to the market.

The question is, do you? Please lodge your comments and questions (in public) on Ask the Experts or (in private) by sending an email to researchdirector@intelligentinvestor.com.au.