Draft: Accounting v2

Accounting v2

Refactoring of the accounting subsystem to improve operational experience with ARC accounting in ARC7.

First of all, the core change is a redesign of the database schema to better address the production use-cases of publishing summaries to APEL and dealing with benchmarks changes. It also take into account cleanup operations and change the integer types in C++ code to match SQLite data structures.

Upgrade from ARC6

After a lot of testing on @maikenp data to evaluate in-place schema change on-upgrade. The operation is too intrusive and will cause looong downtime of the service, while database conversion is ongoing. Taking the big sizes of prod databases, this way was refused.

Suggested non-intrusive approach:

• ARC7 will start wrinning the data to the new accounting database file accounting_v2.db immediately, without blocking startup and job submission
• CE admin runs arcctl accounting database migrate specifying from which point in time to migrate records from old database to new database. For production sites publishing to APEL it is recommended to mirage just from beginning of current month. If site is not publishing, this can be skipped.
• Previous data can safely stay in old database, the new arcctl will keep support for v1 schema and can query/republish/etc ARC6 data.

Summary of changes

Both already done and planned changes are included:

• DB schema v2: tables for Benchmarks and FQANs, cascade deletes.
• Write data using new schema to accounting_v2.db in A-REX. Use signed integers.
• Work with different database locations (to specify v1 database or to query the backup copy)
• Implement support for both v1 and v2 queries in arcclt (both stats and publishing queries)
• Dry-run mode for re-publishing to improve testing and troubleshooting
• Implement accounting database backups capability in arcctl
• Implement records migration logic in arcctl
• Implement records cleanup logic in arcctl
• Implement database rotation in arcctl

/cc @Konya