Commit f20bf92
committed
fix(selfhost): widen orb-relay-drain checkin margin to match redeploy cadence
LOOPOVER-12 fired 1600+ missed-checkin outages over 2+ weeks. Ruled out the
reentrancy-guard overlap-skip (0 occurrences in 14 days of live
instrumentation) and in-process timeouts (~13 events, too few to explain the
volume). The container's own StartedAt resets on every redeploy unlike its
stable sidecars, and boot (image pull + Postgres-readiness wait + init)
routinely exceeds the old 2-minute margin on a fleet redeploying multiple
times a day. Raise checkinMargin from 2 to 6 minutes so a normal redeploy
doesn't trip a false outage, while a genuinely stuck instance still alerts
within ~21 minutes.1 parent 47b34e9 commit f20bf92
2 files changed
Lines changed: 39 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
109 | 109 | | |
110 | 110 | | |
111 | 111 | | |
112 | | - | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
113 | 127 | | |
114 | 128 | | |
115 | 129 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
816 | 816 | | |
817 | 817 | | |
818 | 818 | | |
| 819 | + | |
| 820 | + | |
| 821 | + | |
| 822 | + | |
| 823 | + | |
| 824 | + | |
| 825 | + | |
| 826 | + | |
| 827 | + | |
| 828 | + | |
| 829 | + | |
| 830 | + | |
| 831 | + | |
| 832 | + | |
| 833 | + | |
| 834 | + | |
| 835 | + | |
| 836 | + | |
| 837 | + | |
| 838 | + | |
| 839 | + | |
| 840 | + | |
| 841 | + | |
| 842 | + | |
819 | 843 | | |
820 | 844 | | |
821 | 845 | | |
| |||
0 commit comments