Repository navigation
fix(sql): async-safe comms queue, keep DB across maps, index-friendly SteamID lookups - #83
Conversation
… SteamID lookups
- sbpp_comms: replay queued blocks against the main database, not the
SQLite queue; keep the connection across maps; only drop it on real
connection-lost errors; SQLite queue queries get their own callback
- SteamID lookups use authid IN ('STEAM_0:x','STEAM_1:x') instead of
REGEXP / LIKE so the authid indexes are usable (main, comms, checker,
sleuth); the banlog insert reuses the bid VerifyBan already has
- sbpp_main / sbpp_checker / sbpp_sleuth retry failed connects on a
timer; null guards on sm_banip / sm_unban / sm_addban / reports
- Legacy CloseHandle / WritePackString / ReadPackCell and raw client
indexes in async callbacks replaced
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
WareZ57
left a comment
There was a problem hiding this comment.
Went through the whole diff plus the surrounding code in all four plugins. Most of it checks out. There's one small issue in the new sbpp_main retry and a nit in sbpp_checker.
What I checked and agree with
- Comms queue: confirmed on
mainthatQuery_ProcessQueuebuilds and sends theINSERT INTO %s_commson the SQLite handle it got asdb, so queued blocks could never reach MySQL. Moving tog_hDatabase.Format/Queryfixes that, and the queued strings now get MySQL escaping instead of SQLite's. The in-flight counter is balanced on every path (SELECT error,DB_Connect()failing, truncated rows, insert callbacks). The queueDELETEand the next queueSELECTgo through the same SQLite handle in order, so a row can't be read again before its delete runs. TIMER_FLAG_NO_MAPCHANGEremoval onTimer_StopWait/Timer_ReconnectDB: onmaina map change during the wait killed the timer and left the state onWait, soDB_Connect()returned false until the plugin was reloaded. This is a good fix to have.- SteamID lookups:
IN ('STEAM_0:x', 'STEAM_1:x')returns the same rows as the oldREGEXP '^STEAM_[0-9]:x$'/LIKEfor universes 0 and 1. The columns use the panel charset's case-insensitive collation, so case behaves the same too. The admin sub-selects still keep the exactauthid = '%s'match inside the IN list. For SourceSleuth,BanTypeonly has 0 and 1 in the web panel, sotype IN (0, 1)doesn't drop any rows. VerifyBanbanlog: using theiBidfrom the row that was just matched is better than the old sub-select. That one had no expiry check and could log an expired ban's bid.ClientRecheck/SQL_OnIPMendnow get userids. The null-handle guards onsm_banip/sm_unban/sm_addban/SBPP_ReportPlayercover everyDB.use that could run before the connection exists.- Built locally with spcomp 1.12.0.7259 (same
-i includeas CI): all 6 plugins compile with no errors or warnings, on this branch and merged with #82.git merge-treeagainstmainis clean, and with #82 onlyCHANGELOG.mdconflicts.
Retry timers can stack up in sbpp_main
Every failed connect now schedules its own Timer_RetryConnect (GotDatabase, line 1167), and OnRebuildAdminCache() still starts a connect of its own whenever DB is null and !g_bConnecting (line 432). While MySQL is down, each map change can start a second chain on top of the running one:
- Chain A's timer is pending, so
g_bConnectingis false. - The map changes,
OnRebuildAdminCacheconnects, the connect fails, andGotDatabaseschedules timer B. - A and B now both keep retrying every
RetryTime.
A chain only stops when its timer fires while another connect is still in flight. With a fast failure ("Connection refused" while MySQL restarts or is stopped) that almost never happens, so a long outage ends up with roughly one chain per map change. Each chain retries the connect, logs to sourcebans.log and re-parses the overrides backup every 15 to 60 s. It's harmless once the DB comes back, but it's easy to avoid with a single timer handle:
Handle g_hRetryConnect = null;
// GotDatabase(), failure branch
if (g_hRetryConnect == null)
g_hRetryConnect = CreateTimer(RetryTime, Timer_RetryConnect);
public Action Timer_RetryConnect(Handle timer)
{
g_hRetryConnect = null;
if (DB == INVALID_HANDLE && !g_bConnecting)
{
g_bConnecting = true;
Database.Connect(GotDatabase, "sourcebans");
}
return Plugin_Stop;
}Nit: sbpp_checker reconnect check skips players who are still loading
OnDatabaseConnected now calls LateLoading(), but LateLoading() skips anyone who isn't IsClientInGame() (line 745). Take a player who got OnClientAuthorized while the DB was down (it returns early on g_DB == null) and is still loading when the connection comes back. That player is never checked, and the counts stay at -1 until the next map. OnClientAuthorized() doesn't need the player in game, so IsClientConnected() && IsClientAuthorized() && !IsFakeClient() should be enough there.
Out of scope, possible follow-up
Native_SBReportPlayer still puts the raw iReporter / iTarget client indexes in the DataPack, and SQL_OnReportPlayer passes them to SBPP_OnReportPlayer after the async insert. If the reporter leaves and the slot is reused, the forward gets the wrong client. In sbpp_report that's only the "Report Sent" chat line, but third-party listeners could log the wrong player. Userids would fix it, same as you did for ClientRecheck.
…connect - sbpp_main: GotDatabase() schedules Timer_RetryConnect only when no retry is pending (g_hRetryConnect). A connect started by OnRebuildAdminCache while a retry was already pending used to start a second retry chain, so a long outage ended up with roughly one chain per map change. - sbpp_checker: LateLoading() no longer requires IsClientInGame(). A player authorized while the database was down and still loading when it reconnected was never checked, and the counts stayed at -1.
|
Thanks for the review @WareZ57. Both points are fixed in aedf11e. Retry timers stacking in sbpp_main sbpp_checker skipping players who are still loading Report native (out of scope) Checks
|
Keep both CHANGELOG sections: 2.2.8-srcdslab (#85) above this PR's 2.2.7-srcdslab.
Summary
sbpp_comms.sp):Query_ProcessQueueis the callback of the SQLite queueSELECT, so itsdbwas the SQLite handle and theINSERT INTO %s_commswas built and sent on SQLite (no such table there). Every mute/gag queued during a MySQL outage stayed inqueue2forever and never reached the panel. The insert now usesg_hDatabase.Format/Query(after aDB_Connect()guard), the same fix sbpp_main already has for its ban queue. A small in-flight counter stops a second queue pass from sending the same rows twice. I kept one INSERT per row instead of one Transaction, because each row is deleted from the queue only when its own insert succeeds, and one bad row must not block the others.sbpp_comms.sp):OnMapEnddeletedg_hDatabase("necessary for SQL_SetCharset", which is not true). On every map this caused a reconnect, a blockingSetCharset, and aVerifyBlockfor every player. Players who joined before the reconnect finished were marked as checked without a query, so muted players could talk for a moment. The handle is now kept across maps, andSetCharsetruns once per new connection inGotDatabase(the driver keeps the charset across its automatic reconnects). The queue flush that used to happen as a side effect of the per-map reconnect now runs fromOnMapStartwhen the connection is up.sbpp_comms.sp):DB_Conn_Lostdeletedg_hDatabaseon every failed query, including the SQLite queue queries (CREATE TABLE queue2, queue insert/delete) that went throughQuery_ErrorCheck. It now takes the error text and drops the handle only for real connection errors ("Lost connection", "gone away", "Can't connect", "Broken pipe"). SQLite queue queries use their ownQuery_QueueErrorCheck, which only logs. The reconnect timers no longer useTIMER_FLAG_NO_MAPCHANGE, so a map change can't leave the state stuck onWait. A failed connect now waitsRetryTimebefore trying again, instead of reconnecting on every block check. IfVerifyBlockfails with a plain SQL error, the player is rechecked afterRetryTimeinstead of every second.authid REGEXP '^STEAM_[0-9]:%s$'andLIKE 'STEAM_%:%s'can't use an index. They are nowauthid IN ('STEAM_0:%s', 'STEAM_1:%s'), with the existingauthid = '%s' ORfolded into the same IN list. Changed queries: the ban check (sbpp_mainCheckClientBan),VerifyBlock/ unblock select / queue and punishment admin sub-selects (sbpp_comms), the connect counts andsm_listbans/sm_listcomms(sbpp_checker), the alt check (sbpp_sleuth), and all admin sub-selects insbpp_mainban/unban/queue inserts. Thesb_banslookups now usetype_authid, and thesb_commslookups useauthid.sbpp_main.spVerifyBan): theINSERT INTO %s_banlogsub-selected the bid again with the REGEXP/IP condition (and without the expiry check). It now uses theiBidthat VerifyBan already fetched.sbpp_sleuth.sp):WHERE ip = '%s'with no leadingtypecould not usetype_ip. Both ban types store an IP, sotype IN (0, 1)keeps the same rows and lets MySQL use the index.sbpp_main: a failed connect was only retried at the next admin cache rebuild. It now retries on aRetryTimetimer, guarded byg_bConnecting. I left perf: set the database charset once per connection in sbpp_main #82'sGotDatabaseSetCharset/InsertServerInfo lines alone, and git merges this file cleanly with perf: set the database charset once per connection in sbpp_main #82.sbpp_main:sm_banip,sm_unban,sm_addbanandSBPP_ReportPlayercalledDB.Formaton a null handle. The commands now reply that the database is not available. The report native logs the failure and tells the reporter.sbpp_checker:SetFailStateon the first failed connect is replaced by log + 30 s retry. On connect it now checks players who are not checked yet, which also fixes the late-load check, since that used to run before the DB was connected.sbpp_sleuth: a failed connect was never retried. It now retries after 30 s.CloseHandle→delete(sbpp_mainmenus and reason pack,sbpp_commsexpire timers),WritePackString/ReadPackCell→ DataPack methods.sbpp_main'sClientRechecktimer andSQL_OnIPMendnow get a userid instead of a client index.2.2.7-srcdslab(next after perf: set the database charset once per connection in sbpp_main #82's 2.2.6). The plugin version comes from the release tag.Not included
SQLite_UseDatabase(sbpp_main / sbpp_comms queue) is still synchronous, but it runs only once at plugin start on a local file.Database.Connectwith a missingdatabases.cfgentry falls back to the"default"config (MySQL), not to SQLite, so an async switch would send the queue to MySQL on installs without a queue entry. I found no clean fallback, so I left it as is.KEY(authid)onsb_admins(web schemastruc.sql): this PR only changes the SourceMod side. The admin sub-selects are index-ready now. Manual SQL for existing installs:IN ('STEAM_0:…','STEAM_1:…')form no longer matches the never-used universesSTEAM_2…STEAM_9. The old REGEXP/LIKE matched them.sbpp_main.spmerges cleanly. Both PRs add a new entry at the top ofCHANGELOG.md, so whichever merges second will need that one-line conflict resolved.Test plan
main)sm_mutea player (goes toqueue2), start MySQL, change map: the mute shows up insb_commsand the row is gone fromsourcecomms-queue.sq3sb_srvgroups): the error is logged, but there is no "Lost connection to DB" and no reconnectsm_banip/sm_unban/sm_addbanreply "database is not available" while it is downsb_banlogrow is written with the rightbid;EXPLAINon the connect query showstype_authid/type_ipsm_listbans/sm_listcommsand the connect-time admin notice still show the right counts; SourceSleuth still detects an alt on a shared IP🤖 Generated with Claude Code