Risk and Reward — safe-f and CAR25

A trading savvy friend asked a pertinent question recently. She had some funds available and was considering three alternatives. She had tested them using good modeling and simulation technique and all three seemed safe. Safe in the sense that all three traded liquid issues and out-of-sample results were similar to in-sample backtests. The profit potential for the three were 20% per year, 10% per year, and 2% per year.

She recognized that the one that offerred 2% was probably her “risk free” alternative. It would be the place to park funds when no other system needed them for an active position. It was not a future, and there were no options, so there was no leverage available to raise the annual return.

As we talked about the 10% and the 20% systems, she said that she would be fine trading any system that had a risk within her tolerance. She described her tolerance as wanting to keep drawdowns limited — no more than 10% from highest equity. We agreed that there is always some chance of a “flash crash.”

After some discussion, she stated what we would use as her “personal risk tolerance.” She would be trading a $100,000 account, forecasting 2 years into the future, wanting to hold the probability of drawdowns greater than 10% to a chance of no more than 5%.

She is describing her desire to manage the trading of the system to avoid getting caught in the “tail risk” of the distribution of the trades.

The safe-f and CAR25 metrics I have developed are designed to do precisely that. To illustrate the techniques, we will use the same trades used in the Trade Quality posting earlier.  (This is not one of the three systems my friend was considering.)   The equity curve that was produced during validation is shown in the next chart.

There are 355 trades over the period from 1/1/2008 through 12/7/2017 — 9.93 years — 35.75 trades per year. Her forecast horizon would have an average of 71 trades. The best we can hope for is that the distribution of trades in the future will be similar to the distribution of the “best estimate” set of trades.  At this point, the best estimate set of trades is the set of trades that were made during the validation phase of development — the trades in the out-of-sample period.

The mean gain of those 355 trades is 0.30%, which can be expressed as a gain factor of 1.0030. After 71 trades, the final wealth in the trading account will be, on average, 1.0030 ^ 71, or 1.237.  After 2 years, the balance on the account that began at $100,000 would be $123,700, on average.  Final equity of a set of trades is independent of the order of the trades, but drawdown does depend on the order. This figure shows ten equally likely two year equity curves.  The heavy black line is the average of the ten.  It shows the final equity of $120,600, which is reasonably close to $123,700.  Note that all ten had account balances below $100,000 at some point in the two year period.  Two had balances below $90,000 at some point, suggesting that this system is riskier than the equity curve above would indicate. 

To estimate the drawdown that might be expected from trading 2 years, we use the Monte Carlo technique.  We create many, say 1000, equity curves, each consisting of 71 trades, and compute the final equity and maximum drawdown associated with each equity curve.  We use those 1000 individual equity curves to construct the distribution of drawdowns, allowing us to estimate the probability of drawdowns of various depths.  As the next chart illustrates, when the system is traded at full fraction, f = 1.00, the inverse cumulative distribution of maximum drawdown shows that risk is higher than the trader’s risk tolerance.

She wants to hold the risk of drawdown greater than 10% to a chance of 5%.  That is represented by the lower-right blue circle.  It is at the intersection of the 10% drawdown horizontal line with the 5% chance vertical line.  The curve of the inverse CDF of maximum drawdown is above that.  It shows that the risk of a 10% drawdown is about 50%; and the tail risk — the maximum drawdown at the 95th percentile — is greater than 24%.  

This is the risk inherent in trading this system.  All funds allocated to this system have this risk profile.  To hold the risk to her tolerance, the $100,000 must be split into two accounts.  One is used to trade the system, and the other is kept in a risk free account to act as ballast to temper the drawdown of the portion being traded.  Through trial and error, we can determine the fraction that will be used to trade in order to move the CDF curve down so that it passes through the circle at her risk tolerance.  That value is 48%.

The curve of the inverse CDF passes through the intersection of a 5% risk of a 10% drawdown. 

safe-f is 48%.

In order to hold drawdown to her tolerance, she uses 48% of account equity for each position, holding the remaining 52% in a risk free account.

We rerun the simulation with f set to 48%, this time computing many equally likely equity curves and recording final equity.  The next chart shows the inverse CDF of final equity for an account that was initially funded at $100,000 and traded at f=0.48 for two years.

The horizontal axis gives the percentiles.  At the 25th percentile, the final equity is $103,800.  The compound annual rate of return corresponding to a 3.8% gain in two years is 1.8%.  

CAR25 is a conservative metric that can be used to compare alternative uses of funds.  It is the compound annual rate of return for the risk normalized profit at the 25th percentile of the distribution.  The CAR25 metric for this system is 1.8%.

The trader can compute CAR25 for all alternative uses of the funds, including other trading systems.  When the risk is normalized, the system that has the highest CAR25 is the one that should be traded.

Even though the out-of-sample equity curve looked good, the risk-normalized profit potential is low.  The system, using the model developed and validated, cannot be made less risky.  The trader can increase the profit potential only by relaxing his or her personal risk tolerance.  When the analysis is rerun using a risk tolerance of a 5% chance of a 20% drawdown, safe-f is 0.90, final equity is $110,800, and CAR25 is 4.9%.  

Trading Management

safe-f and CAR25 continue to be useful after development and while the system is being traded.

As trades are made, either actual trades made with real money or shadow trades that would have been made, they are added to the best estimate set of trades.  safe-f and CAR25 are recalculated and provide the trader information about the health of the system.  When trades are accurate and profitable, safe-f increases and a larger portion of the funds can be safely used to take live positions.  CAR25 will increase.  Losing trades cause safe-f to drop, requiring that a smaller portion of funds be used for trading and a larger portion be kept in reserve.  CAR25 will drop.

safe-f and CAR25 will drop for any of the following reasons:

  1. The model was fit to random noise in the data rather than valid signals that reliably precede profitable trades.
  2. The system is healthy, but the trades are coming from the unfavorable tail of the distribution.
  3. The distribution is shifting.

In general, the cause is probably a mix of all three.  Whatever the cause, the trader should take a drop in safe-f as a warning that the system is becoming riskier and larger drawdowns are ahead. (Drawdowns are only opportunities to increase position size when the system is inherently profitable and is stationary — neither of which can be relied on in trading. )

Next — Impulse versus State Signals

Back to System Development

2 thoughts on “Risk and Reward — safe-f and CAR25

  1. Happy Holidays Dr. Bandy,

    I hope you are well. I have a question regarding your Safe-F and utilizing a System Stop Loss. As I’m not a big fan of applying stop losses to individual positions, I thought of possibly applying the statistics analyzed from the Safe-F/CAR25 procedure, to determine an appropriate System stop. For example, Max Drawdown of 15% at the 95th percentile could translate to a Max Loss for the System of 15%. Would you consider this a valid approach? I ask you this knowing of course your thoughts on stop losses from reading Foundations of Trading.

    Regarding the calculation for Safe-F/CAR25, is there any particular reason as to why 1000 simulations per iteration is suffice? Or could one increase the statistical reliability of the result by simply increasing the amount of simulations?

    Once again, Happy Holidays. Thank you for the insight.

    Best, E. R.

    • That is an excellent question, E. R. —

      The technique I illustrate assumes measurement is at the close of trading rather than intra-day. I have not done the calculations for applying safe-f intra-day. Applying it intra-day would have the current day’s low as the price that caused the exit. Unless you recast the procedure, that would be the only low price in the calculations — all the others are the price at the close. In managing the trading that way, you might precalculate the price at which the exit would be triggered, then enter a stop or stop limit — neither of which is my favorite order type. My experience using market close for all of the prices is that relaxing the calculation from once per day to once per every few days does not make a lot of difference in the results. (This is the opposite direction from what you are suggesting.) After you have done some experimentation with more frequent management, let me know what you find.

      There will always be some variation in results of simulations. When calculating the cumulative distribution of either maximum drawdown or terminal equity, there are three causes of variation. One is the number of data points in the sample from which the individual trades are drawn. If this is too low a number, there will be many duplicates, causing many similar equity sequences. Second is the number of simulations that each contribute to the distribution. In general, more is better, but you will reach a point where increasing that number does not improve resolution. Third is the randomness of the Monte Carlo technique. You can get an idea of how much difference that might make by fixing the first two, then running about ten simulations.

      There is so much noise and such a weak signal in the financial data that it will be frustrating to try to make these calculations and rules precise.

      Be certain you understand one thing. The risk associated with a trading system will always apply to the funds exposed. Assume you hope to hold drawdown to a 5% chance of a 20% drawdown. If the CDF of the un-normalized results show there is a 5% chance of a 30% drawdown, the system will experience the 30% drawdown. You cannot change that. Reducing the position size by keeping some funds aside will hold the drawdown of the entire — exposed plus safe — account to a 20% drawdown.

      You are watching for any of three things, any of which will cause a drawdown. One is faulty validation. Using trades that are not truly out-of-sample to populate the best estimate set. Two is distribution drift. Whatever is underlying the fluctuations has changed and the model no longer fits the new data distribution. Third is bad luck. There is a 5% chance that results will be in the tail of the distribution, even though the validation was done correctly and the distribution of the future continues to resemble the distribution of the past. While we cannot tell which the cause is, either or both of the first two are much more likely than the third. The lowest probability of the three is that of being in a drawdown from which we confidently expect there will be a recovery, and which we might use as an opportunity to “double down.” So we must react cautiously and reduce position size in anticipation of deeper drawdowns to come.

      Best regards, Howard

Leave a Comment