Performance Returns - Quality Control

Performance systems calculate performance returns in the same way, however it is typically other factors that will cause errors in your investment return outputs. The performance calculation engine is only using the inputs provided, thereby generating returns based off of that information. Therefore, a Firm can’t always blame the performance calculation engine, for most of the time, it is typically some other factor that has caused an error in your performance calculation.

• Pricing - A Firm’s securities are typically automatically priced by a vendor, through some type of nightly data feed. Sometimes, a data feed could populate an incorrect price into your system. Additionally, a common performance error occurrence is typically created from those pesky manually-priced securities.

• Unreconciled Data - The most common performance errors are caused by unreconciled accounting (trades, general ledger, etc.) data. Properly reconciled bookkeeping data is key to generating correct performance returns.

• System Settings - Since performance is calculated basically using the same formula, your performance system will undoubtedly have Performance Settings that instruct the engine on how to calculate your performance returns. Sometimes these Settings are not configured properly which will allow for inaccurate performance results.

• Missing Fee Entries - Net of Fees performance can be calculated using a variety of methodologies. One such method is using Actual fee amounts that will generate the Net of Fee return on your specific account, using your specific fee (dollar or %) rate. When using the Actual Fee $ amount, those amounts need to be entered prior to the generation of any Net of Fees returns.

• Spreadsheets & Cell Formulas - Spreadsheets should NOT be used in calculating performance for a large number of accounts. Period. However, it is typically cell formulas referencing incorrect cells/tabs/files that cause most of the manually calculated performance return errors.

When reviewing investment performance output for a given time period (typically monthly or quarterly), the common misconception is that ‘well, if all of the individual account returns in the composite are the same, then they must be right!’. However, that is not the case in that if you have hundreds of accounts in a composite, and they are all around 4% for the month, they COULD all be correct, or in fact, could all be WRONG. Therefore, testing of your cyclical output is neccessary. Some of the basic testing methodologies are shown below.

• Test Accounts - If you have recently gone through a system conversion OR you find that your automated data feeds are not entirely consistent/accurate, utilizing a Test Account process can be sound. In that, for any given strategy, you pick apart (do a detailed reconciliation against the custodian) one specific account on a regular basis. This type of detailed testing can help to ensure that your feeds, pricing, etc. all are functioning as expected.

• Representative Accounts - Typically, if you have accounts based on a strategy (ie a composite), you will choose one account in that investment universe to ‘represent’ that strategy’s returns. Typically, the Rep. Account is the one your Firm would use as the Test Account, for that is common practice. Once the Rep. Account is chosen, you will validate that account’s performance return against some other type of calculation methodology/report. If your system has some type of holdings-based performance report that doesn’t use the Modified Dietz calculation methodolgy, then you can at least do a ‘sanity check’ against that holdings-based calculation measure.

• Net Returns Against Gross Returns - Your Firm’s Net returns will always be lower than your Gross returns. Therefore, a simple check to ensure your Net and Gross are calculated correctly is to ensure that the Gross return is higher than the Net return, and, that the Gross and Net returns are not identical. Note: In months were no fees are generated, Net and Gross will be the same, therefore, this check is only sound during the months when your Firm generates billing invoices.

• Benchmark Comparison - Your strategy should be comparable to some index that is publicly available. While the strategy may not always be inline with the index, comparing your performance return against a 3rd party comparable index is a sound check to perform.

• Third Party Comparison Checks - It is always a good measure, if available, to check your returns against some other 3rd party that is possibly calculating returns as well. Typically, your custodian could also be generating performance results on your account(s). Accessing your custodian’s performance output and comparing to your Firm’s calculated returns provides an independent, 3rd party comparative measure.

Reliance on the calculations presented by a vendor or 3rd party consultant does not absolve your Firm of accurately representing your Firm’s investment results.

GIPS Requirement 1.A.26 - The firm is responsible for it's claim of compliance with the GIPS standards and must ensure that the records and information provided by any third party on which the firm relies meet the requirements of the GIPS standards.

What this rule means it that if you are using a 3rd party system or outside vendor to calculate your performance returns, the sole reliance on that vendor/3rd party is not 100% acceptable. In that, not only does the vendor/3rd party need to provide adequate documentation surrounding its data validity measures, but must also provided documented proof of such validation. Additionally, it is a good measure for your Firm to also do independent testing, outside of the vendor/3rd party, to ensure accurate investment results.

If a Firm has adequate composite inclusion/construction policies or invests closely to a given model, performance outliers should be few and far between. Composite construction guidelines can vary wildly from Firm to Firm, with some Firms adopting ‘looser’ inclusion policies more than others. Therefore, outlier testing methodologies, and what constitutes and outlier, can vary between Firms, depending on your Firm’s rigidity surrounding each composite description. Below are some common outlier testing ideas to consider.

• Highs/Lows - Look at the most extreme outliers first (ie extreme lows and extreme highs). Your Firm should have some type of automated measure or report that will sort returns accordingly.

• Standard Deviation - Use a Standard Deviation measure in coordination with a comparison of the return differences to identify the outlier portfolios that require review. Use an appropriate + or - # of Standard Deviations that your Firm feels most comfortable.

• Frequency - Perform outlier testing on a monthly basis. Trying to review performance outliers on a quarterly basis is more frustrating and time consuming than it needs to be. Quartelry returns are made up of monthly returns, therefore, your Firm has to look at the month’s anyways, so your Firm might as well make outlier testing a standard, monthly practice.

• Common Occurrences - Develop a running list of those accounts that are always outliers. This will help your Firm sift through the returns more easily by removing those pesky, re-occurring outliers from bogging you down. Note: These re-occurring outliers shouldn’t be ignored, however, by removing them from the flotsam this may help to make your outlier testing run more efficiently.

  • Side Note: If you typically have re-occurring outliers, then maybe those accounts should be in a different composite?

• Typical Outlier Causes - Performance outliers can be caused by an infinite number of reasons. However, the basic causes of performance outliers are: Data integrity issues, Cash flows, Client implemented restrictions, Client-requested legacy positions in the account, Portfolio in the wrong composite, and even an incorrect strategy implementation on an account.