Jack Schwager has written a great series of Market Wizards books interview a series of fund managers and traders with different styles who are all successful. In his most recent book Hedge Fund Market Wizards, he interviews quant manager Jaffray Woodriff. Woodriff is head of Quantitative Investment Management (QIM), a successful CTA with a different approach than the rest. As the book explains:
The majority of futures traders, called CTAs, use trend-following methodologies.1 These programs seek to identify trends and then take a position in the direction of the trend until a trade liquidation or reversal signal is received. A smaller number of systematic CTAs will use countertrend (also called mean reversion) methodologies. As the name implies, these types of systems will seek to take positions opposite to an ongoing trend when system algorithms signal that the trend is overextended. There is a third category of systematic approaches whose signals do not seek to profit from either continuations or reversals of trend. These types of systems are designed to identify patterns that suggest a greater probability for either higher or lower prices over the near term. Woodriff is among the small minority of CTAs who employ such pattern-recognition approaches, and he does so using his own unique methodology. He is one of the most successful practitioners of systematic trading of any kind.
Source: Hedge Fund Market Wizards
Several posts have been written investigating Woodriff’s methods, specifically what’s been labeled the Internal Bar Strength indicator. You can check out some of the other discussion at the following posts:
Doing the Jaffray Woodriff Thing (Kinda), Part 1
I wanted to highlight some of my favorite quotes and takeaways from the interview. Most have to do with the issue of robustness in trading systems design. All the following quotes are from Hedge Fund Market Wizards. Here we go:
I discovered that it was much better to use multiple models than a single best model
Here Woodriff highlights the diversification benefits of having multiple trading systems versus a single trading system. Of course we know if the systems are relatively uncorrelated the result of combining the equity curves will be a better risk adjusted single system.
I found that using the same models across multiple markets provided a far more robust approach. So the big change that occurred during this period was moving from separate models for each market to common models applied across all markets.
This is a common technique to increase robustness. If a system is optimized for a single market, it can be easily over-optimized. If the rules have to work on several different markets and they still yield good results, they will be more likely to repeat in live trading.
Are all your secondary variables derived just from daily open, high, low, and close price data?Absolutely. That is all I am using.
I was a little suprised that all he uses are OHLC bars. I know several successful traders who only use these datapoints and thus I know how much information is conveyed in them. Perhaps we should limit our dataset and simply refine our existing research methods rather than search for more information in our quest for the ultimate system.
Sometimes we give a little more weight to more recent data, but it is amazing how valuable older data still is. The stationarity of the patterns we have uncovered is amazing to me, as I would have expected predictive patterns in markets to change more over the longer term.
Woodriff uses data going back several decades in his analysis. I know people that trader intraday only use the last 3-10 years so it all depends on the timeframe you are trading. For daily bars i’d say more is better, but you also have to consider the quality of the data you have.
combined different secondary variables into trend-neutral models. They were neither trying to project a continuation of the trend or a reversal of the trend. They were only trying to predict the probable market direction over the next 24 hours.
Here Woodriff talks about his direction recognition approach which stands in contrast to the typical trend following or mean-reversion CTA strategies.
I don’t do that. I read all of that just to get to the point that I do what I am not supposed to do, which is a really interesting observation because I am supposed to fail. According to almost everyone, you have to approach systematic trading (and predictive modeling in general) from the framework of “Here is a valid hypothesis that makes sense within the context of the markets.” Instead, I blindly search through the data. It’s nice that people want hypotheses that make sense. But I thought that was very limiting. I want to be able to search the rest of the stuff. I want to automate that process. If you set the problem up really well with cross validation, then overfitting is a problem that can be overcome. I hypothesized that there are patterns that work, and I would rather have the computer test trillions of patterns than just a few hundred that I had thought of.
This is really interesting. For years I’ve read that you need to start with a valid market or eocnomic hypothesis for a trading system in order for it to be successful. Woodriff’s approach is the opposite, in that he just data-mines ‘blindly’ but uses robust techniques to avoid over-fitting of data. It does seem very limiting to only test hypothesis you can come up with yourself vs looking at all other possible combinations. This will require more analysis….
A lot of people think they are okay because they use in-sample data for training and out-of-sample data for testing.6 Then they sort the models based on how they performed on the in-sample data and choose the best ones to test on the out-of-sample data. The human tendency is to take the models that continue to do well in the out-of-sample data and choose those models for trading. That type of process simply turns the out-of-sample data into part of the training data because it cherry-picks the models that did best in the out-of-sample period. It is one of the most common errors people make and one of the reasons why data mining as it is typically applied yields terrible results.
Again, Woodriff goes against conventional wisdom and pokes holes in the most common method of system robustness testing, namely Out-Of-Sample tests. He mentions several times in the interview the use of the Cross Validation technique as a better alternative (something that is also highlighted by fellow market wizard and CTA Bill Eckhardt).
You can look for patterns where, on average, all the models out-of-sample continue to do well. You know you are doing well if the average for the out-of-sample models is a significant percentage of the in-sample score. Generally speaking, you are really getting somewhere if the out-of-sample results are more than 50 percent of the in-sample.
Here he basically describes the cross validation process and puts some concrete thresholds on what types of results he looks for in the test; perhaps I will try this out.
This was a great interview with a successful quant. I recommend purchasing the Hedge Fund Market Wizards book and reading the full interview as well as the interviews of traders with other approaches. It always helps to get insights from the best….