Brute Force, Bad Math, and the Trader’s Edge

In this post, I want to share some insights into my thought process as a trader. I will follow up with a post looking at Relative Volume. I originally defined, quantified, and coded that tool in 2009-2010, but it grew out of a foundation much deeper.

My thought process might be interesting to readers with a wide range of backgrounds. If you don’t have a strong background in some related disciplines, you’ll see that I was in the same boat and how I transformed a weakness into a strength. If you do happen to have a strong background, there are dangers as well—it’s very easy to skip over “basic” elements and to ignore foundations. A few easy adjustments might really enhance your vision and offer new insight.

As many of you know, I did not have a strong quantitative background when I started trading. I did have highly trained pattern recognition skills, developed as a professional-level classical pianist and a composer. When I began trading, I quickly realized I needed to be able to wrangle some numbers. I taught myself some math, but I did so through a process that immediately took me to applications—so I was learning deeply, and with real-world influences always present.

For years, I felt apologetic that I did not know the closed-form solutions to many problems and, instead, had to kludge together solutions that often felt like compromises. I would later find out that this is, indeed, a time-honored approach. Monte Carlo simulation was actually invented when Stanislaw Ulam and John von Neumann were working on a complex question related to the atomic bomb. They were at an impasse, and had the idea to simply simulate what would happen at the atomic level. Drawing on work that Markov had done decades earlier, they created early computer simulations and found a solution that eluded other means of computation.

Both my math education and the complexity of the problems I was working on were orders of magnitude less, but I stumbled into the same solution. I learned to simulate outcomes based on sometimes millions of trials and then learned another thing or two about the perils of generalizing to real-world outcomes. I might’ve reinvented the wheel (i.e., elements of inferential and non-parametric statistics), but the insights from that work were a foundation that supported everything that came after.

I’m not saying this is the best way to work for everyone. It is painfully slow and you’ll spend a lot of time asking questions for which there are already known solutions. But there’s a particular kind of insight that I think can only come from this work. I do worry that, as the world becomes increasingly more AI-driven, this kind of insight will be missing from the human skill set. This type of insight is largely responsible for the trading edges I possess and for the maintenance of those edges over the past few decades.

It is good, however, to have a little education to go with your brute-force work. I found myself in a situation where I was working with and for a group of traders who employed a small office of coders. These traders were working with various activity measures, but were mostly focused on range and looking at range over various time periods. (Nothing wrong with that, to be clear.) However, the issues arose when they took tools that worked on the daily timeframe to the weekly.

Their math was very clear and also very clearly wrong. They simply multiplied measures that worked on the daily by five. Though this code was not directly under my purview, I saw it and raised some simple questions: “are weekly ranges actually (observably) five times a daily range?” No one in the shop had ever looked into that. I explained that the theoretical level should be the square root of five, that it would be close but not match precisely, and produced a quick spreadsheet showing that empirical values matched that very closely. The team took the data and chewed on it for a few days. My final shot across the bow was asking if monthly ranges were truly 20 (approximately the number of trading days in a month) times the daily range, and were yearlies roughly 250 times the daily?

The head programmer, who, in the hierarchy, was the answer to all quantitative questions, spent a few days on the question. He looked at the data I had collected (hundreds of datapoints across all markets), and came back to the boss with something like, “So it looks like what Adam says is right, but I don’t understand it because we don’t have anything squared or any square roots in our calculations.”  My attempts to explain variance and standard deviation were ignored, and the team kept using 5.

Lessons nested within lessons in this blog post: this was a very well-capitalized group of traders and they had been trading for well over a decade. They had bled money consistently, to the tune of millions of dollars. After I was no longer working with them, they continued along the same trajectory and eventually closed shop, never having found the success they were looking for.

Don’t assume that being well-capitalized means being successful in the market. Don’t assume that large traders are smart. These traders had a certain kind of street smarts, but I was the only person in the room with even half a quantitative background. (And, to be clear, I had less than half a quantitative background—I fumbled through some PhD-level math and econometrics courses when I did my MBA. I was grateful to have the opportunity to do that, but I’m perfectly aware that there are teenagers who eat math that’s completely beyond my understanding for breakfast.)

It's easier to do this work today than when I started. I started working with Excel spreadsheets and writing in BASIC. I quickly learned about issues with PRNGs after I realized that months of work were compromised. It could take hours to run a test on a 486 or Pentium processor. (Modern chips are literally a thousand times faster, and there are other aspects that further multiply the speed advantage.) It was extremely laborious to create clean graphics. It wasn’t easy to get clean data. I could go on and on with the issues that are now easily solved with modern tools.

(I’ll drop an Easter egg here: if you’re going to do work like this, it’s worth acquiring a hardware RNG today.)

So these were the tools I had:

• An understanding of how to simulate outcomes rather than going for closed-form solutions
• An intuition to slice data sets based on time or activity
• A respect for the complexities that arise when we transition from models to actual trading
• Deep knowledge of both Technical Analysis and econometrics. I knew how people had looked at data and what approaches led to real insight.
• I was very well read in academic finance research, and understood the types of edges researchers were both finding and debunking.

These tools served me well. In my next post, I’ll tell you how I put them to work creating the Relative Volume measure that is used today.