ECS-investigations/T4A-DB-logs
SHA256
2
0
Fork 0
Code Issues 5 Pull requests Projects Releases Packages Wiki Activity Actions
PostgreSQL log analysis for T4A (Tracks4Africa) Django app — investigating mass listings_listing.date_modified overwrite on 2026-03-30
2 commits 1 branch 0 tags 68 KiB
  • Python 96.5%
  • Shell 3.5%
Find a file
Exact
hclaude 1f33e78949 Full-history rerun: name the job (last_igo_export), quantify scope, add PDF 
- INVESTIGATION_LOG.md Entry 2: full 3538-file rerun (Mar25-Jun06).
  Job identified = last_igo_export (t4a_last_run_date app) from host ::28,
  ~32 parallel workers, per-row Django .save() clobbering auto_now
  date_modified. 282k distinct rows on 03-30; ~99.5k remain @03-30
  (matches client 102555). Secondary +2h timezone bug found.
- md_to_pdf.py: markdown->PDF (fpdf2) for emailable reports.
- INVESTIGATION_LOG.pdf: rendered report.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 06:31:23 +00:00
analysis/scripts Full-history rerun: name the job (last_igo_export), quantify scope, add PDF 2026-06-06 06:31:23 +00:00
.gitignore Scaffold T4A-DB-logs + day-1 smoking gun 2026-06-06 06:07:23 +00:00
CLAUDE.md Scaffold T4A-DB-logs + day-1 smoking gun 2026-06-06 06:07:23 +00:00
INVESTIGATION_LOG.md Full-history rerun: name the job (last_igo_export), quantify scope, add PDF 2026-06-06 06:31:23 +00:00
INVESTIGATION_LOG.pdf Full-history rerun: name the job (last_igo_export), quantify scope, add PDF 2026-06-06 06:31:23 +00:00
README.md Scaffold T4A-DB-logs + day-1 smoking gun 2026-06-06 06:07:23 +00:00

README.md

T4A-DB-logs

PostgreSQL log analysis for T4A (Tracks4Africa) — a Django web app.

Why this repo exists

Client reported ~102,555 listings_listing rows with date_modified = 2026-03-30. That field should track real website edits, so a mass overwrite breaks it. This repo hunts the responsible user / app / query in the PostgreSQL statement logs.

Day-1 result: a bulk Django .save() job on host ::28 (user t4a_django) re-saved the whole listings catalogue on 2026-03-30, stamping date_modified on every row. See INVESTIGATION_LOG.md.

Layout

  • analysis/scripts/hunt_listings_dm.sh — reproduces the overwrite analysis.
  • analysis/scripts/*.py — generic PG-log parse/aggregate pipeline (fast_parse, pandas_analysis, load_to_duckdb, …) shared with the HEV_PG_analysis / AD_BMP-Analysis investigations.
  • analysis/summaries/ — REPORT_*.md deliverables.
  • Raw logs live in /scratch/Claude/T4A-db-logs/ and are never committed.

Quick start

analysis/scripts/hunt_listings_dm.sh            # default day 2026-03-30
DAY=2026-04-01 analysis/scripts/hunt_listings_dm.sh

See CLAUDE.md for log format and conventions.