README.md

07-Automation/README.md

1. ---
2. type: automation-index
3. scope: 07-Automation
4. tags: [claude, outreach, automation]
5. created: 2026-07-22
6. last_modified: 2026-09-21
7. ---
9. # 07-Automation: Weekly Partner Client Pipeline
11. The **live** Cowork automation that turns live Plinth order data into a partner client
12. classification each week, maintains the partner stubs and the XML census, and (on your
13. approval) continues to enrichment and Bitrix. Everything here is plain files you can
14. edit in VS Code and re-upload.
16. ## The files
17. - **[[partner-thresholds]]**: config + the activation checklist. Tier bands are live
18. program values; `min_orders_to_qualify: TBD` is the arming switch. Single source of
19. truth; the driver parses the ` ```yaml ` block out of it.
20. - **[[Weekly-Partner-Pipeline-SOP]]**: the full A0→H procedure and the guardrails,
21. including **"Why incremental"**, the Plinth truncation bug that dictates the whole
22. architecture. Read that section before touching Stage A.
23. - **`weekly_partner_pipeline.py`**: the deterministic engine, v2 incremental:
24. `seed-baseline` → `ingest` (dedupe by `order_ref`) → `classify` (from local history)
25. → `verify` (no-shrink) → guarded stub/XML/workbook writes, then the read-only Bitrix
26. stages (`reconcile-deals`, `route`, `outreach-pull-list`, `outreach-audit`). Does not
27. call Plinth or Bitrix; Claude feeds it the raw page dumps and deal pulls. The file is
28. the command-line entry point; the code is in **`partner_pipeline/`**, one module per
29. stage (`stages.py` holds the funnel, stage-name and send-token tables).
30. - **`requirements.txt`**: pyyaml and openpyxl for the driver, freezegun for the replay.
31. - **`data/order_history.json`**: the local order history (created at seed time). This,
32. not the API, is the source of truth for lifetime value.
33. - **[[scheduled-task]]**: the live weekly schedule + the exact prompt (ENABLED
34. 2026-07-23).
35. - **[[pipeline-routing-rules]]**: the entry rules `route` implements, and the four-funnel
36. audit scope.
37. - **`sample/`**: the generator and replay that produced every client file in this repo
38. (see "The sample" below).
40. ## Current state (2026-07-24)
41. 1. **LIVE.** Seeded from Ines's roster workbook (baseline as_of 2026-07-20), armed
42. (`min_orders_to_qualify: 1`), and scheduled: Mondays 06:00 America/Chicago,
43. autonomy `through_vault`. First run completed 2026-07-23: verify OK, stubs + XML census
44. refreshed for the whole partner base. (The shipped sample's first run, 2026-09-14:
45. 128 partner clients, Palladium 85 · Platinum 28 · Iridium 8 · Rhodium 7.)
46. 2. `refresh-xml` maintains the XML map as a **complete partner census** (adds
47. empty-pipeline `pl-` chunks for anyone missing; idempotent).
48. 3. Enrichment (Stage G) and Bitrix push (Stage H) remain **manual and human-gated**;
49. nobody is routed into an outreach/watch pipeline until those entry rules are decided.
51. ## Activating it
52. Follow **[[Activation-Runbook]]**: the definitive step-by-step, with commands
53. validated against the roster workbook (in this repo, the sample roster in
54. `sample/inputs/`). Short version: two-pass seed (All Accounts then the Complete list),
55. catch-up pull, set `min_orders_to_qualify`, classify + verify + reconcile, one manual E to H
56. pass, enable the schedule.
58. ## The sample
59. Meridian Display Co. and every client in this vault are fictional. The client data is not
60. hand-written: two scripts in `sample/` invent a client book and then run the real driver
61. over it.
63. - **`sample/generate_sample.py`** invents the book: 381 Plinth accounts with a
64. roster workbook (`sample/inputs/Partner-Clients-Roster_2026-09-07.xlsx`, tabs
65. "2026 Partner Working List", "Complete Partner Clients List" and "All Accounts", totals
66. through 2026-09-07), two weeks of Plinth `orders.fetch` page dumps, and two weekly
67. Bitrix deal enumerations with their stage histories and contacts. It is seeded, so every
68. run produces the same world. The world is built to exercise the parts of the driver that
69. matter: a Plinth day (2026-09-17) that truncates silently at 50 rows with overlapping
70. offsets; cancelled (27) and refunded (28) orders; clients crossing a tier band mid-week
71. and a brand-new customer who qualifies on a first order (`has_baseline=False`); an
72. approved merge; the directly-partnered Harlow Ridge Homes mailboxes (Rhodium by
73. override, never cold-emailed) and a staff mailbox on our own domain; an identity twin
74. where only one twin carries the Not Interested closure; a shared company mailbox; a
75. role-named account and a messy shipping-label name; retired Bitrix stages, same-minute
76. workflow re-entries, and a cart that moved into production funnels after converting.
77. - **`sample/run_sample.py`** replays two Monday runs (2026-09-14 and 2026-09-21) of
78. `weekly_partner_pipeline.py` against those inputs, doing what the scheduled session does
79. with the MCP calls stood in for by files: seed the baseline in two passes and copy the
80. roster as the first Drive-swap base; then per run, pull and ingest each day of the
81. window until a page adds nothing, `classify`, `verify`, `write-stubs`, `refresh-xml`,
82. `refresh-workbook`, `reconcile-deals`, `route`, `outreach-pull-list` (attaching stage
83. histories and contacts only for the deals it asks for), and `outreach-audit`. The clock
84. is pinned to each run date with freezegun, because the driver dates its outputs by
85. today's date; that is the only thing it changes. Each run's console output is saved as
86. `05-Outputs/partner-runs/run-log_<date>.txt`.
88. **Regenerating.** The replay refuses to run if `07-Automation/data/order_history.json`
89. exists, so run it on a scratch copy with the generated state removed. The root
90. [[README]] ("Replay the sample") has the exact commands and the `diff` that checks the
91. result: only the ten hand-researched stubs differ, because the replay does not regenerate
92. them. The replay copies each day's Plinth pages into `05-Outputs/partner-runs/raw/`, which
93. is not committed (the committed copy is `sample/inputs/plinth/`).
95. **What came from the replay.** Every stub in `03-Clients/client-stubs/`, the census
96. `03-Clients/partner-client-master-list.xml`, everything in `05-Outputs/partner-runs/` (plus
97. the two `Bitrix_Pipeline_Outreach_Audit_*.xlsx` workbooks in `05-Outputs/`), and everything
98. in `07-Automation/data/` (the merge registry and suppression list are copied in from
99. `sample/inputs/`). State after the 2026-09-21 run: 381 Plinth accounts in history; 130
100. partner clients (Palladium 86, Platinum 29, Iridium 8, Rhodium 7), 130 stubs and 130 census
101. chunks, 43 of them with pipeline placement; 54 deals across the four audited funnels, 28
102. with a verified touch; route: Winback 50, cat 24 first-touch 31, Dormant queue 4, excluded 45.
104. **What was added by hand on top.** Ten stubs carry invented research under `## Research` (written the way `client-enrich` writes it, with sources on `.example` hosts), a few carry Human Notes and Outreach Log rows that match their Bitrix history, `05-Outputs/First Email/` and `05-Outputs/Second Email/` hold eleven sample drafts spread across the six stages, and `04-Working/dual-active-pipelines_2026-09-21.md` is a review note built from the 09-21 census. A fresh replay does not recreate these.
106. ## Design decisions worth knowing
107. - **Incremental by necessity.** Plinth `orders.fetch` truncates at roughly 70 orders per
108. response with no signal and unreliable offsets (verified on one long-standing account,
109. 2026-07-22; see [[DECISIONS]]). Lifetime totals therefore come from the local history
110. store, never from a single API call. The `verify` no-shrink guardrail is the tripwire.
111. - **Two existing skills do the last two stages**: `client-enrich` (enrichment) and
112. `partner-pipeline-push` (Bitrix). This template chains them; it does not duplicate them.
113. - **Modification date is new.** Neither stubs nor the XML map carried one. The driver adds
114. `last_modified` (frontmatter) and `<last_modified>` (XML), deliberately named apart from
115. the historical `updated` key the enrichment writeback avoids.
116. - **Human Notes and Research are never written** by the classifier. Research is only ever
117. touched by `client-enrich`. This is the [[CLAUDE-clients]] scraping-dump contract.
118. - **Bitrix is always human-gated.** Even at full autonomy the SOP holds the push for a person.
119. - **Shared vault:** every write stage appends one [[LEDGER]] entry.
121. ## Dependencies
122. `pip install -r 07-Automation/requirements.txt` (openpyxl optional for the driver, which
123. falls back to CSV; freezegun is only for replaying the sample). Plinth MCP must be connected for the pull;
124. Apollo/Apify for enrichment; Bitrix for the push.