I built free DFS lineup optimizers for NHL and NBA, and the honest question anyone should ask me is: do they work? So I started DFS paper trading my own tools every night, and from now on I’m publishing every result, good and bad, on a public scoreboard.
Spoiler: so far every set of lineups is losing money. That’s exactly why I’m paper trading and not playing for real (I’m cheap, and I’d rather lose fake dollars while I learn).
What “paper trading” means here
Every night I pick a real DraftKings classic contest, the kind with a few thousand entries and a 150-lineup cap. Before lock, the optimizer builds its lineups. While the games are live I download the lineups of everybody who actually entered (DraftKings only lets you grab them while the games are on). The next day I score my lineups and the whole field with the final box scores, rank mine against the real field, and pay myself whatever the contest’s real payout table says I would have won.
So the contest is real, the field is real, the payouts are real. The only fake part is the entry fee.
What I don’t do (this is the important part)
I don’t pick players. Not one. I don’t look at a lineup and think “nah, I don’t like that guy tonight.” Every lineup comes straight out of the solver.
I also don’t bring any sports knowledge to the decisions. I’m not following the teams day to day like a true fan would, I don’t read beat reporters, and I don’t have hunches about who’s hot. The projections come from public sources (DailyFantasyFuel, and Frozen Tools from DobberHockey for NHL), and the settings for each set are fixed before the night starts. If a set is called “plain” tonight, it was called “plain” with the same settings last week.
And I don’t cherry-pick results. The scoreboard shows every set I run, every night, including the ones that get embarrassed.
The one thing I do fix by hand
There’s one place where the test knows something a real player wouldn’t know for sure at lock. I set the actual starting goalies and remove players who got scratched late.
Here’s why. If I were playing with real money I’d be watching the news closely right up to lock, and you should too. But I’m not, and that’s not what I’m testing. I want to know how good the optimizer is at building lineups, not how good I am at refreshing Twitter at 6:55 pm. If I left a scratched player in, the lineup would score zero for that slot and the result would measure my news-reading, not the tool. (Late scratches are their own problem, and I wrote about handling late scratches in NHL lineups separately.)
That does make the paper results a little kinder than real life would be. I’d rather tell you that up front than have you find it in a footnote.
The sets I test
I don’t test one magic configuration. Each night I run the same group of sets side by side against the same contest: 12 for NHL and 6 for NBA. Some are plain “maximize projected points” builds, some force teammates to stack together, some spread goalies or star players across the lineups, some use a simulation to pick lineups by their odds of a top finish. They all face the same field on the same night, which makes the comparison fair (and also means they win and lose together on the same nights, so they’re not independent tests).
To avoid picking a winner after the fact, I named one headline set per sport when the scoreboard went live, and it stays the headline no matter how it does from here on. The rest are shown right below it.
How I judge them
Not by average score. In big tournaments the average lineup loses money; what pays is the rare lineup that finishes near the very top. So the scoreboard shows return on investment (winnings divided by fees, minus one), how many lineups landed in the top 1% and top 0.1% of the field compared to what pure chance would give, top-10 finishes, and the best finish of the night.
The “compared to chance” part matters. If 150 lineups were random, you’d expect about 1.5 of them in the top 1% of the field. If a set keeps landing more than that, it’s doing something right, even on nights it loses money.
Where things stand
Since October 27, when I froze the testing rules, I have 12 NHL nights and 11 NBA nights on the books. Every set is in the red. The best NHL set is down about 32%, the worst about 71%. In NBA the best is down about 30%, and most of its money came from a single lineup that finished 2nd out of 5,248 on one night.
That last sentence is the whole problem with DFS testing in one line: one big hit can make a set look brilliant, and one slate tells you almost nothing. A few NHL sets are landing in the top 1% more often than chance would predict, which is encouraging, but it’s not money yet. It will take a lot more nights before I trust any of it.
I’ll keep adding nights as long as I don’t get tired of doing the backtesting (no promises, but the scripts make it a lot less painful than it used to be).
Try the same tools
The optimizers I’m testing are the same ones on the site, free and with no signup: NHL lineup optimizer, NBA lineup optimizer, NFL and MLB. If you want to see how they’re doing before you trust them, that’s what the DFS paper trading scoreboard is for.
Play responsibly: DFS is a game of skill with a lot of luck in it, and even a good optimizer loses most nights. If you want help, the National Council on Problem Gambling is a good place to start.
Things that I use, like, and am affiliated with:
Mint Mobile offers great cell phone service for $15 flat, get $15 off using the link. Get discounted phones with service activation and no contract.
I never spend money before I check Mr Rebates or Rakuten to get cashbacks, rebates, discounts, coupons or cheaper gift cards.