Earlier this week I wrote about what an MIT DFS paper taught my NHL lineup optimizer. Read that paper and you quickly run into its follow-up: Martin Haugh (Imperial College) and Raghav Singal (Columbia), How to Play Fantasy Sports Strategically (and Win), later published in Management Science. It builds directly on the MIT work and asks the one question that paper skipped: what is everybody else doing? Their answer is that to play fantasy sports strategically you can’t just maximize your own points. You have to beat a field of lineups you can’t see yet.
I read it for a selfish reason. I have been paper-trading my NHL lineups for a week (I build them before lock, then score them against the real DraftKings fields once the games are over), and after five slates no setup has made money. None of them (that is not a typo). So this post is half review, half lab notebook: what the paper does, what I borrowed, and the different kind of lineup generator I am building along the same lines.
The problem they are actually solving
A DFS lineup is nine players under a salary cap, and your score only matters relative to everyone else’s. The paper treats the two contest types separately. In a double-up, roughly the top half of the field doubles its money, so the bar to clear is a score near the middle of the pack. In a top-heavy tournament a handful of places take most of the prize money, so the bar is the score of first, second, third place and so on.
Both bars depend on two unknowns: how the real players perform, and which lineups everyone else picked. The MIT paper modeled the first and ignored the second. Haugh and Singal model both, and say that as far as they know theirs is “the first academic work to develop such an approach in the context of fantasy sports.”
Modeling the other players
This is the part I liked most. For each position, the share of the field that picks each player is treated as random, drawn from a Dirichlet distribution, and the parameters of that distribution come from a regression on things you know before the contest: a public ownership estimate (FantasyPros, for NFL), salary, and projected points. They call it Dirichlet regression and fit it on the ownership numbers published after past contests.
Then they glue the positions together into whole lineups with a simple copula: with some probability an opponent is a “stacker” who pairs a quarterback with his main receiver, otherwise the positions are picked independently. Lineups that leave too much salary unspent are thrown out, because real people almost never do that. Generate a few thousand of these fake opponents and you get a distribution of the winning scores instead of a single guess. A footnote admits they first assumed ownership was known exactly, and that it “led to over-certainty and poor performance” (I have been there).
The finance trick in the middle
Beating a random bar is an old problem in finance: maximize the chance a portfolio beats a stochastic benchmark. If you approximate the gap as normally distributed, the best portfolio sits on a mean-variance frontier, and the paper uses that to turn “maximize the probability of winning” into a series of binary quadratic programs, one per risk weight, solved with Gurobi. The intuition is worth more than the math. In a tournament your expected score is below the bar, so you want variance, and you want it in a specific way: players everybody owns raise the bar when they score, so a lineup full of them can’t pass the field even on a good night.
For several entries they split again. In double-ups they submit copies of one lineup, because you only need to clear the middle. In top-heavy contests they diversify: the objective is (approximately) submodular, so a greedy algorithm that adds the entry with the biggest gain each time is guaranteed to get within about 63% of the best possible set. Each new lineup can share at most six of its nine players with the ones already chosen. That is the same portfolio problem I wrote about in One Good Lineup Is Easy. Twenty Is the Hard Part., with a better yardstick.
Did it work?
They played the 2017 NFL regular season on FanDuel: 17 weeks, each week 50 entries in a $1 top-heavy contest with about 200,000 entries, 25 in a $2 quintuple-up and 10 in a $2 double-up. In the top-heavy series the strategic model made $280.74, over three times the benchmark model that ignores opponents, and the whole season was funded with $75.76 of capital, which they report as a return of over 350% in 17 weeks. The double-up and quintuple-up series lost a little, and in the quintuple-up the benchmark ended ahead. They donated the winnings, like the MIT group did.
What made me trust it is how candid they are about the misses. Their projections were off-the-shelf and taken a day before kickoff, so they missed late news: a starting running back went questionable, his cheaper backup was obviously underpriced, and around 60% of their double-up opponents took the backup while they didn’t. I lost money the same way on Oct 5, when a center I had in a lot of lineups was scratched after I built them. They also note that expected profit came out above realized profit, which they put down to “the bias that results from optimizing within a model.” Keep that phrase in mind for later.
Two more results stood out. First, most of the value of modeling opponents came from simply knowing the ownership: under their own model, the season’s expected profit in the top-heavy series was about $1,400 for the benchmark, $5,400 with a fixed ownership estimate, and $6,000 with the full random model (measured with their model as the truth, so take the exact gaps loosely). Second, they priced insider trading and collusion: five players pooling their entries into one diversified set raised expected profit by 44% in their top-heavy example. And they close with the line that pushed me to build something: NFL is one short, high-variance season, and they expect the approach to suit hockey, baseball and basketball better.
What I am building differently for NHL
It is a work in progress, and I am not porting their model. Hockey isn’t football, and my data is different, so I am keeping the ideas and changing the machinery. I had the simulator half-built before I read the paper closely; reading it changed how I model the other players and told me which bias to expect.
Player outcomes. They draw player points from a multivariate normal with a mean and a covariance matrix. In hockey, fantasy points come from goals, and a goal is scored by a line or a power-play unit, so I am simulating the game instead. Each team’s goals come from its Vegas implied total, each goal is handed to a forward line and defense pair (or a power-play unit), the scorer and assisters come from inside that group, and shots, blocks and the goalie’s saves follow. So far it is calibrated on 1,356 NHL games from last season and this one, and it runs 10,000 simulated nights in about a second. Linemates come out correlated at about 0.2, against 0.16 for a rough history-based estimate. That correlation is the whole point: it is what makes a 190-point lineup possible.
The other players. I don’t have a FantasyPros-style ownership source for NHL that I trust, but DraftKings shows the percent drafted for every player once a contest is live, and I have saved full fields for five slates (between 8,000 and 53,000 lineups a night). An ownership model fitted on four slates and tested on the fifth correlates 0.72 with real ownership for the players who matter (those above 2% owned), and a sampler builds stacked lineups the way the real field does. Same spirit as their Dirichlet regression and stacking copula. One honest wrinkle: my synthetic field’s top-1% scores come out about five points too low, because real players react to late news my model never sees, so I add five points.
The objective. Instead of the mean-variance approximation, I am computing each lineup’s expected payout directly: in every simulated night, where would it rank in the simulated field, and what does that contest’s real payout table pay for that rank?
The candidates. Their greedy step solves a quadratic program per entry. I am solving about 2,500 small problems instead, each one the best lineup for one simulated night (a “this night breaks right” lineup), using Google’s CP-SAT solver. Then I pick the set greedily, each new lineup chosen for the simulated nights where none of the earlier picks reached the top 1%. That is their submodular greedy idea with a different yardstick.
Why the field is easier to predict than the scores
This was the part that made the paper click for me. Ownership turned out to be much easier to predict than points. My ownership model hits 0.72 correlation on the players who matter. Points are another story: on the Oct 6 slate the best skater projection I tested correlated about 0.46 with what players actually scored, DraftKings’ own projection 0.27, and plain salary did as well as any of them. For goalies every source I tried came out negative (yes, worse than a coin flip).
It makes sense once you say it out loud. Ownership is thousands of people reacting to the same public information: salaries, projections, Vegas lines. Points are mostly the randomness of a hockey game, which nobody predicts well. So the edge, if there is one, isn’t in out-predicting the scores. It’s in knowing roughly what the field will do and positioning against it, so that when a lightly owned player has the random big night, you are one of the few holding him. That is Haugh and Singal’s whole argument in one sentence, and my own numbers ended up agreeing with it.
First results (and a familiar bias)
I ran the first version on the same five slates, Oct 2 through 6, filling every contest to its entry cap and scoring against the real fields. The coverage version lost 60% overall (67% with each night’s best prize removed). My plain settings lost 79% (81%). A power-play stack restricted to the teams Vegas liked showed a profit of 54%, but that was one $10,000 hit on Oct 3; without it, it lost 64%. The simulation’s lineups landed in the top 1% nine times, where the same number of random lineups from the field would have done it about 20 times. So it beat my old settings, and it did not beat the field.
And there was the bias Haugh and Singal warned about. The model predicted about 2.4 times the prize money its own picks actually won (yes, I checked twice). The lineups it liked most were exactly the ones where it was most wrong. I tried shrinking every prediction toward the field average, and it changed nothing, because the over-confidence sits in the handful of top-ranked lineups, not spread across all of them.
Five nights can’t separate any of these methods. One jackpot decides the totals, and the top-0.1% counts are zero, one or two for every method.
What happens next
From now on every method runs on every slate, with no tuning on the night being tested, and I judge them on how often they reach the top 1% compared with the 1% a random lineup gets. A tournament set needs roughly 1.2 times that rate to break even after the site’s cut, and right now I am at about half. I am also pulling the player pool closer to lock, since late scratches have already cost me.
None of this is in the public tool yet. If it holds up over a few weeks of slates it will be, and I will say so here either way. Meanwhile the free NHL lineup optimizer works as described in A Free NHL Lineup Optimizer for DraftKings, and the paper is free on SSRN (the journal version is at INFORMS) for anyone who wants the proofs I skimmed.
Things that I use, like, and am affiliated with:
Mint Mobile offers great cell phone service for $15 flat, get $15 off using the link. Get discounted phones with service activation and no contract.
I never spend money before I check Mr Rebates or Rakuten to get cashbacks, rebates, discounts, coupons or cheaper gift cards.