Chat's updates for security, db and workflow

PRODUCTION-FIXES.md

@@ -0,0 +1,253 @@
+# Production safeguard changes — October 19, 2026
+
+This implements the production review's security defects, financial history and
+corrections, protected audit writes, mailbox verification, and deployment tooling.
+It does not deploy to a public host or silently change the development database.
+
+## Apply the new schema before running this code against an existing database
+
+Back up the target database, verify that the backup restores, and use a maintenance
+window. With separate migration credentials and the target environment configured:
+
+```text
+php bin/migrate-auth.php
+php bin/migrate-roster.php
+php bin/migrate-production.php
+```
+
+All three migrations are now applied to development. The production-safeguards
+migration was applied there on October 20, 2026, after a fresh verified backup;
+see the rollout record below. It has not been applied to a production host here. It adds
+mailbox verification, financial correction/reversal tables, and payment metadata;
+it enlarges journal entries, ensures transactional audit storage, and refreshes
+the user view. It can be rerun. Existing financial records retain their amounts;
+their previously unknown actors remain null and appear as historical records.
+Schema changes can lock tables; they are CLI operations, never web requests.
+
+Existing addresses are not automatically marked verified. Ordinary users must
+verify their email before viewing club data or receiving/reactivating membership.
+They can use the new verification flow or complete a password setup/reset link.
+Existing administrators retain access to administer this transition. User records,
+memberships, and financial history are preserved.
+
+## Account security and browser dependency
+
+- Password-reset issuance now locks the account and compares the originally
+  resolved email, status, and account version. Account edits and status changes
+  delete outstanding reset tokens under the same account lock. This handles both
+  interleavings: an edit before token creation, and an edit during email delivery.
+- Username/email/role/status changes invalidate reset links and verification
+  links. Changing an email also clears its verified status. Disable/reactivate
+  cannot revive a reset link. Existing session revocation remains intact.
+- Replaced jQuery 3.2.0 with official jQuery 3.7.1 on current and legacy templates.
+  Removed the old asset. Its downloaded SHA-256 matches the official integrity
+  value recorded in browser-dependencies.json. Existing jQuery 3.x plugin behavior
+  is retained; a major-version migration is not necessary for this fix.
+- Browser tests cover account creation, deletion safeguards, the alert and popover
+  editors, rosters, financial corrections, downloads, and verification requests.
+- Fixed the garbled username validation message and normalized newly added view
+  text to UTF-8.
+
+## Financial history, corrections, and audit
+
+- New LedgerService coordinates dues, payments, corrections, and reversals.
+  Charges and payments record their actor; payments also store method/reference.
+- Itemized payment history shows date, amount, member, method/reference, actor,
+  and reversal details. Members see only their own financial history.
+- Correcting dues appends an adjustment with the previous/effective amounts,
+  actor, reason, and request key. It preserves the original charge. A correction
+  cannot reduce dues below received payments; reverse an erroneous payment first.
+- Reversing a payment appends one reversal with an actor and reason. It restores
+  the amount owed and retains the original payment. This records a bookkeeping
+  correction; it does not send a refund or move money.
+- Repeated payment, correction, and reversal requests are idempotent. Changed
+  payloads with a reused request key are rejected. Concurrent writes use account,
+  club, and dues locks as appropriate; overpayment is still rejected.
+- Financial CSV includes effective dues, paid/balance cents, individual payments,
+  corrections, reversal reasons, and actors. User text is protected against
+  spreadsheet formula injection. Monetary calculations use integer cents.
+- Financial, club, membership, season, and administrative account changes write
+  audit entries in the same transaction. An audit failure rolls back the change.
+  Entries include entity IDs and relevant before/after values, amounts, or reasons.
+  Passwords and tokens are never audit payloads.
+- Removed the journal-clearing route, button, and handler. The legacy PHP helper
+  also refuses clearing. Accounts with recorded journal history are retained;
+  disable them instead of deleting their identity.
+- Runtime grant examples allow journal inserts but deny journal update/deletion
+  and financial update/deletion. Migrations and backup/restore use separate roles.
+  Authentication audit errors continue to go to server logs without preventing
+  login; configure alerts on those errors.
+
+## Mailbox verification
+
+- Signup requests an email verification link and explains how to resend it.
+  Public requests return neutral messages for unknown, verified, rate-limited,
+  or undeliverable accounts.
+- Links contain 256-bit random values, stored only as hashes, expire after 24
+  hours, and are bound to the account email/version. Failed delivery removes the
+  unsent token. Expired records are handled by the cleanup CLI.
+- Opening a link does not consume it. Confirmation requires POST, CSRF protection,
+  a valid token, and the account password. This prevents a mailbox owner from
+  accidentally activating an account whose password was chosen by someone else.
+- Successful password setup/reset also proves ownership of its current mailbox.
+  Changing a known password without a delivered token does not verify an email.
+- Active membership assignment requires an active, verified account. Regular
+  members also need verified email to read club data. Admin summaries show the
+  verification state and retain the password setup-link action.
+
+## Production configuration and release tooling
+
+- Remote production database connections require HCR_DB_TLS_CA. Certificate
+  validation is enabled and the negotiated TLS cipher is checked. Localhost/
+  loopback database connections remain available; explicitly protect any local
+  tunnel. Production TLS must still be tested against the actual server.
+- Set HCR_ENVIRONMENT=Production explicitly. That prevents automatic loading of
+  settings.local.xml; environment variables or an explicitly selected external
+  HCR_SETTINGS_FILE supply production secrets. Never package development secrets.
+- HTTPS is recognized directly or from X-Forwarded-Proto only when REMOTE_ADDR is
+  in HCR_TRUSTED_PROXIES, a comma-separated list of exact IP addresses. The proxy
+  must overwrite forwarded headers and direct access to its backend must be
+  restricted. Untrusted and ambiguous forwarding headers are rejected.
+- Application responses hide PHP errors and send CSP, frame/referrer/content-type
+  protections and Permissions-Policy. Production HTTPS sends HSTS. Inline scripts
+  and JavaScript links were replaced so script-src does not need unsafe-inline.
+  Inline styles remain permitted for Bootstrap/editor compatibility.
+- Apache protections now require mod_rewrite rather than silently skipping the
+  private-directory rules. Apache needs AllowOverride support. IIS additionally
+  blocks sensitive file extensions. Nginx and PHP configuration examples are in
+  deploy/. These examples must be adapted and tested on the chosen server.
+- `python bin/build-release.py /protected/releases/hcr.zip` creates a clean zip and
+  SHA-256 manifest from the workspace, including new files. It excludes local
+  settings, backups, tests, Git metadata, and IDE files. Use locked Composer
+  dependencies and review the archive before deployment; it is not published.
+- `python bin/check-production.py https://app.myhuntingclubroster.com` performs
+  read-only HTTPS, header, cookie, and private-path checks after deployment.
+  It sends no email and changes no accounts. The actual production host was not
+  contacted in this change.
+- `php bin/check-health.php` exits nonzero on database/schema failures. Run it
+  from the server monitor with production settings and alert on failures.
+
+## Operational setup on the hosting server
+
+The project now supplies the entry points; the server administrator must install
+their schedule, permissions, storage, and alerts on the actual production host.
+No hosting credentials or host details were supplied for that work.
+
+1. Provision migration, runtime, and backup accounts separately. Adapt
+   deploy/runtime-grants.sql.example. Keep secrets outside the document root and
+   restrict file access to the service/job accounts. Test the runtime role.
+2. Schedule `php bin/cleanup-auth.php` daily. It removes expired reset/verification
+   links and stale rate-limit buckets without deleting audit history.
+3. Schedule `php bin/backup-database.php /protected/backups` using backup-specific
+   credentials. Set HCR_DUMP_BINARY to the installed mariadb-dump executable if it
+   is not on PATH. The directory must exist outside the application tree. The job
+   writes a complete transactional SQL dump and checksum metadata and removes its
+   temporary credentials file. All tables must be InnoDB. It exits on failure.
+4. Restrict the backup directory before running the job, especially with Windows
+   ACLs. Use encrypted storage and protected off-host copies, define retention and
+   recovery objectives, and regularly restore-test backups. The script does not
+   implement encryption, retention, or off-host storage by itself.
+5. Alert on failed backups/cleanup/health jobs, PHP errors, audit failures, SMTP
+   failures, disk pressure, and database failures. Configure log rotation and
+   protected audit retention/archival; there is no web audit-deletion facility.
+6. Run Composer platform checks and the isolated regression runner before release.
+   The GitHub workflow supplies a disposable MariaDB and browser; it runs when
+   pushed to a repository with Actions enabled. This change does not push code or
+   claim that GitHub CI has run.
+7. Deploy the complete release during the migration window. Run host checks, then
+   send a real invitation/reset to a controlled mailbox and verify inbox delivery
+   and link completion. Local tests deliberately use simulated delivery.
+8. Keep the prior release and the verified pre-migration backup. Recovery after
+   new financial writes needs reconciliation; do not blindly restore an old dump
+   and discard subsequently received entries.
+
+## Verification and remaining scope
+
+The final local run passed **308 checks plus 5 concurrency scenarios**:
+
+| Suite | Passed |
+| --- | ---: |
+| Authentication service/templates | 49 |
+| Maintenance validation/pagination | 52 |
+| Authentication HTTP | 27 |
+| Project HTTP, ledger, exports, and access boundaries | 99 |
+| Reset invalidation, verification, corrections, audit rollback, transport | 42 |
+| Restricted runtime database permissions | 5 |
+| Browser workflows, editors, corrections, downloads, and CSP | 21 |
+| Migration, release, backup, health, and private-path tooling | 13 |
+| Concurrent defaults, payments, reversal retries, correction retries, final admin | 5 scenarios |
+
+Source checks passed for 41 PHP files, 55 JavaScript files, 39 compiled XSL
+stylesheets, and the jQuery integrity hash. IIS configuration is well-formed XML.
+Composer's final locked dependency audit reported no advisories or abandoned
+packages. The GitHub workflow and actual Apache/IIS/Nginx production hosts have
+not been executed or contacted here.
+
+The verified development backup was also restored locally. After bringing the
+copy to the existing roster schema, the new migration was run twice. Every
+original column value across all 17 existing tables was unchanged. The isolated
+restored copy was removed. That October 19 implementation run did not migrate
+development; the subsequently authorized rollout is recorded below.
+
+Tests use generated disposable databases; they do not send real mail, move money,
+or modify existing development accounts. Temporary databases and the restricted
+test account were removed, and the local test server was stopped. Production TLS
+certificate negotiation and real inbox delivery require the actual hosting setup.
+
+Global site administrators continue to manage all clubs. Active club members
+continue to see their club's active contact directory. These are existing product
+policies, not newly introduced club-admin roles or contact-consent settings.
+Delegated club administration, MFA enrollment/recovery, online payments/refunds,
+and large-scale pagination/background exports remain separate enhancements. The
+production-specific TLS, web-server configuration, monitoring schedules, off-host
+backup policy, and real mail delivery remain deployment acceptance work.
+
+## Development rollout — October 20, 2026
+
+At the user's request, `bin/migrate-production.php` was applied to
+`192.168.1.190/huntingclubroster` at 6:32 a.m. Eastern. The previously approved
+unencrypted development connection was used. The target was checked explicitly;
+no production host was changed.
+
+Before migration, a complete 43,100-byte backup was saved at
+`.tools/backups/20261020-063143/huntingclubroster.sql`. Its SHA-256 is
+`526113c2575927d33680cb696a398f59352554a5892c23c1a8ff414fe6f1e3ca`.
+The backup was restored locally and its checksum and all 17 table counts matched.
+Every existing column value in the original 17 development tables was checked
+before and after migration and was preserved. The database now has 20 base tables.
+
+**48 acceptance checks passed against development**, using temporary accounts and
+an owned temporary club through a local application server. They verified:
+
+- Verification link generation, trusted URL, POST confirmation, correct-password
+  enforcement, single use, stored verification state, and membership gating.
+- Password reset, old-password rejection, new-password login, consumed-token
+  rejection, session revocation, and invalidation after an email change.
+- Clearing verified state on email change and restoring club access only after a
+  fresh bound verification link is confirmed.
+- Dues corrections preserving original charges, safe retries, correct balances,
+  and rejection of corrections below received payments or excess payments.
+- Reversals preserving original payments, safe retries, duplicate-reversal
+  rejection, restored balances, and settlement with a replacement payment.
+- Ordinary-member restrictions, detailed financial CSV, and audit events.
+
+Email issuance used the real application services and development database, with
+only the delivery boundary replaced by local capture. No real emails were sent;
+SMTP/inbox delivery was not retested. The public URL was checked in generated
+links without requiring development to be served at that public hostname. All
+payments were manual test ledger entries; no money moved.
+
+The first helper attempts encountered a Windows temporary-session-path parsing
+issue. The helper was corrected and those temporary records were removed. During
+the first complete run, an independent administrator login changed existing login
+timestamps, a journal entry, and rate counters. The differences were proved to
+match that login and were preserved; no original data was restored over user
+activity. A final complete run finished with a clean baseline comparison across
+all 20 tables. Existing accounts were not used as test accounts or reset by the
+verification scripts.
+
+All temporary accounts, memberships, seasons, charges, payments, adjustments,
+reversals, tokens, test audit/rate-limit entries, and temporary sessions were
+removed. Auto-increment sequences may have advanced. A final CLI health check
+passed. The local restore copy was removed and its database server stopped.
+Private migration results and the check-by-check report remain beside the backup.