A test suite told me my scheduler was broken. It wasn’t. What actually happened is stranger and more useful: on one of the shipped sample rosters my solver said optimal and handed back a schedule with two night posts nobody was assigned to. Not a crash, not an infeasibility, not a timeout. A clean optimal, with holes in it. Here is what I measured, what I had wrong in my own code comments, and the one dependency I had been describing to people as a speed optimization when it is really a correctness one.
The failure that started it
My check script asserts that every shipped sample roster reaches 100% coverage. On my laptop, the hospital-ward sample — 28 days, 16 people, three demand streams — came back at 97.6%. Two Aide night posts, 20 and 27 September, unfilled. On Render and in CI the same commit passed. Same code, same data, two different answers.
My first guess was wrong and I want to say so plainly, because a wrong guess left lying around is worse than no guess: I assumed an unpinned pulp had pulled in a newer CBC that resolved differently. It hadn’t. The difference was that my laptop had pulp installed and highspy not. The engine picks HiGHS when highspy imports and falls back to CBC when it doesn’t — and it did that silently.
Four runs, same model, same data
HiGHS proved optimality in about eighteen seconds with full coverage. CBC finished in about sixteen — faster — with 97.6% coverage and an objective of 111,831.8 against HiGHS’s 111,631.2. CBC is not merely slower on this model. It finishes with a worse roster and calls the result optimal.
“Optimal” means “within the gap you asked for”
The two objectives differ by 0.18%. The sample is configured with a 2% relative gap. So CBC is behaving exactly as instructed and there is no bug in it to find. The word optimal in a solver’s status field is a statement about a tolerance, not about your business rule.
And my business rule — every post covered — is not a hard constraint in this model. Unfilled posts carry a shortage penalty, which is deliberate: a hard coverage constraint turns an understaffed week into a bare INFEASIBLE, and a manager needs to see which two shifts are short, not a refusal. The cost of that design is exactly what bit me. A solver allowed 2% of slack can spend part of it on two nights of unfilled Aide shifts and stop, honestly.
What surprised me is that generosity did not rescue CBC. I gave it a zero gap and ten times the wall clock — GapPercent = 0, a 150-second limit — and it returned the same 97.6%, the same two posts, the same objective, now correctly labelled feasible because it hit the limit rather than proving anything. It is not that CBC needed longer. In that run it never found the roster HiGHS proves in eighteen seconds.
A correction to something I wrote in my own code
There was a comment sitting in my shift engine claiming that setting the absolute gap below the cost of the cheapest shortage “makes that impossible by construction: the solver can never trade a fillable post for a faster finish.” That is false, and it had been reassuring me for weeks.
Both CBC and HiGHS treat the relative gap and the absolute gap as stopping tolerances OR’d together — whichever is satisfied first ends the search. A tight absolute gap can never pull a loose relative one tighter. The test is the second row of the table: I forced the absolute gap to 100,000, a value so loose it should have let HiGHS stop almost immediately if the gap were doing the work, and it still returned 100% coverage and the identical objective. The solver was doing the work. The gap was not. (PuLP does forward both settings to CBC correctly — the forwarding was never the problem.)
highspy is a quality dependency, not a speed one
I wrote about CBC and HiGHS on the slot-mode model a couple of months ago, and there the story was speed and feasibility: CBC returned nothing in thirty-plus seconds, HiGHS proved optimality in thirty-one. That framing is right for that model, and it quietly taught me the wrong general lesson — that HiGHS is the faster option. On the 24/7 shift model it is not meaningfully faster at all. It is better.
So highspy is in my requirements file because the schedules are worse without it, not because they are slower. Render has it, CI has it, my laptop didn’t, and the difference showed up as a mysterious coverage failure three frames deep in a test suite rather than as “you forgot to install something.” If you ship a PuLP model with an optional HiGHS path, treat a missing highspy as a correctness problem.
What I changed
Four things, in order of how much they mattered. The gap settings are now computed once and passed to both solvers; CBC had previously been handed the relative gap only. The automatic fallback to CBC now warns — it used to warn only when someone had explicitly asked for HiGHS, so on the default path a quality drop was completely silent, and that silence was the actual bug. The overclaiming comment was replaced with the measurement. And the test suite now prints which solver is in use and, when highspy is missing, says the coverage checks are expected to fail and how to fix it. That failure cost me an hour once; it should cost thirty seconds next time.
How far this generalizes
Not very far, and I would rather say so than dress this up as a benchmark. One model, one dataset, one machine, one afternoon. I am not claiming HiGHS beats CBC in general — that is a claim you need a benchmark library and a great many runs to make, and the people who maintain those make it far more carefully than I can from a nurse-rostering sample.
The narrower claim is the one I think travels. A solver status of optimal is a statement about a tolerance, so check the quantity you actually care about — coverage, in my case — before you believe a result is complete. And an optional dependency that silently downgrades your answers is worse than one that fails loudly, because a loud failure gets fixed in a minute and a quiet one ships.
The model in question is the same one behind the thirteen industry shift-scheduling pages, and it is free to run on your own roster with no signup.
Things that I use, like, and am affiliated with:
Mint Mobile offers great cell phone service for $15 flat, get $15 off using the link. Get discounted phones with service activation and no contract.
I never spend money before I check Mr Rebates or Rakuten to get cashbacks, rebates, discounts, coupons or cheaper gift cards.