A signal system looked "positive" (+1.4% average 21-day forward return, 52% up) until compared with every ticker it draws from over the same weeks (+5.6%, 55% up): it underperformed its own source list. Separately, the human-facing summary board (the one the operator actually reads and would trade from) was rendered into HTML each day but never written to any table, so it could never be backtested even months later. And 96% of stored signals had no ticker attached, so they were counted as detections but could never be scored.

A trading strategy was backtested as "hold whenever the trend signal is ON" (no stop), then deployed live with a fixed percent-from-entry stop-loss overlay added for safety. The continuous backtest showed the stop rarely firing, but live it fired twice in six days and realized a large whipsaw loss. Root cause: in a continuous simulation, the position's entry price dates back to the signal's start (often years earlier), so an entry-anchored stop is far out of the money and almost never triggers — the sim silently understates the stop's cost. A real account entering mid-trend has a fresh anchor, so the same stop behaves like a tight stop and churns.