--- id: EMAIL-WP-0002 type: workplan title: "MVP Mailbox Evidence Scanner" domain: custodian repo: email-connect status: finished owner: codex topic_slug: custodian created: "2026-06-02" updated: "2026-06-02" state_hub_workstream_id: "c81788aa-0d0a-4493-bf41-ab6cc2068f2f" --- # EMAIL-WP-0002 - MVP Mailbox Evidence Scanner ## Review Fixes Applied This workplan was reviewed and registered against the local State Hub workplan convention. Fixes applied: - Added ADR-001 frontmatter so State Hub can index the workstream. - Converted the MVP work packages into `EMAIL-WP-0002-TNN` task blocks. - Clarified that out-of-office replies are evidence signals, not proof of reachability or awareness. - Aligned suggested repository paths with the current repo layout. Implementation should preserve the product rule from `INTENT.md`: email events are evidence, not result satisfaction. ## 1. MVP Name **Mailbox Evidence Scanner MVP** ## 2. Purpose This MVP establishes the first practical implementation slice of `email-connect`. Given access to a mailbox that receives bounce mails, delivery-status notifications, out-of-office replies, human replies, complaints, unsubscribe messages, and other return messages from a previous batch of emails, the system shall scan and rescan the mailbox, classify inbound messages, extract email-channel evidence, and generate timestamped CSV evidence reports. The MVP proves the core `email-connect` value without requiring outbound sending, provider webhook integration, template management, or a full UI. ## 3. Core MVP Hypothesis `email-connect` can provide immediate standalone value by turning an inbound return mailbox into structured, timestamped email-channel evidence. The MVP validates: ```text mailbox access message scanning rescan safety bounce parsing reply classification evidence normalization deduplication endpoint-quality hints CSV reporting ``` The result should be useful to humans, scripts, and future `coordination-engine` integrations. ## 4. In Scope The MVP shall support: * Connecting to one IMAP mailbox. * Scanning messages in a selected folder. * Incremental scans using a stored cursor. * Full rescans. * Raw message metadata extraction. * Basic MIME parsing. * Bounce / DSN classification. * Out-of-office classification. * Human reply classification. * Unknown/unparseable message classification. * Extraction of affected recipient address where possible. * Extraction of SMTP status code and enhanced status code where possible. * Evidence event candidate generation. * Deduplication of already-seen mailbox messages. * Deduplication of already-emitted evidence. * Timestamped CSV report generation. * Basic local storage for scan state and parsed evidence. * CLI entry point. * Minimal configuration file. ## 5. Out of Scope for MVP The MVP does not need to support: * Outbound email sending. * Provider-specific webhooks. * Multiple email providers. * Full UI. * OAuth mailbox login. * Advanced deliverability analytics. * Advanced natural-language reply interpretation. * Full suppression management UI. * Full endpoint quality dashboard. * coordination-engine live integration. * Database server deployment. * Multi-tenant operation. * Complex mailbox write-back actions. * Deleting, moving, or marking mailbox messages. * Legal delivery assessment. ## 6. MVP User Story As an operator or developer, I want to point `email-connect` at a mailbox containing return emails from a previous outbound batch, scan and rescan it safely, and receive a timestamped CSV report showing what email-channel evidence was found for each affected address or message. ## 7. Target Workflow ```text 1. Configure mailbox access. 2. Run scanner. 3. Scanner fetches messages from selected folder. 4. Scanner parses headers, body, MIME parts, and DSN attachments. 5. Scanner classifies each message. 6. Scanner extracts evidence fields. 7. Scanner deduplicates message and evidence records. 8. Scanner stores scan state. 9. Scanner writes timestamped CSV report. 10. User reviews report or imports it elsewhere. ``` ## 8. CLI Target Initial CLI command: ```text email-connect scan-mailbox --config config/mailbox.yml --out reports/ ``` Recommended CLI variants: ```text email-connect scan-mailbox --config config/mailbox.yml --out reports/ email-connect scan-mailbox --config config/mailbox.yml --full-rescan --out reports/ email-connect scan-mailbox --config config/mailbox.yml --since 2026-01-01 --out reports/ email-connect scan-mailbox --config config/mailbox.yml --report-only-new --out reports/ email-connect scan-mailbox --config config/mailbox.yml --dry-run ``` ## 9. Configuration Example `config/mailbox.yml`: ```yaml mailbox: id: return-mailbox-default protocol: imap host: imap.example.com port: 993 tls: true username_env: EMAIL_CONNECT_IMAP_USER password_env: EMAIL_CONNECT_IMAP_PASSWORD folder: INBOX scan: mode: incremental max_messages_per_run: 5000 since: null include_seen: true mark_seen: false store_raw_headers: true store_raw_body: false store_raw_message_ref: true storage: path: .email-connect/state.sqlite reports: output_dir: reports include_all_evidence: true include_unknown_messages: true timestamp_timezone: UTC ``` ## 10. Minimal Data Model ### 10.1 MailboxScan ```yaml MailboxScan: scan_id: string mailbox_id: string started_at: timestamp finished_at: timestamp? scan_mode: incremental | full_rescan since: timestamp? folder: string status: running | completed | failed messages_seen: integer messages_new: integer messages_parsed: integer evidence_events_created: integer report_path: string? ``` ### 10.2 InboundMailboxMessage ```yaml InboundMailboxMessage: mailbox_message_id: string mailbox_id: string imap_uid: string? message_id_header: string? received_at: timestamp? from_address: string? to_addresses: - string subject: string? raw_headers_ref: string? raw_message_ref: string? first_seen_at: timestamp last_seen_at: timestamp deduplication_key: string ``` ### 10.3 ParsedMailboxMessage ```yaml ParsedMailboxMessage: parsed_message_id: string mailbox_message_id: string parser_version: string message_class: hard_bounce | soft_bounce | delayed_delivery_notice | final_delivery_failure | out_of_office | human_reply | complaint_or_abuse | unsubscribe_or_opt_out | challenge_response | unknown_return_message | unrelated_message | parse_failed affected_email_address: string? original_message_id: string? original_recipient: string? smtp_status_code: string? enhanced_status_code: string? reason_code: string? confidence: low | medium | high parsed_at: timestamp notes: - string ``` ### 10.4 EmailEvidenceCandidate ```yaml EmailEvidenceCandidate: evidence_candidate_id: string mailbox_message_id: string parsed_message_id: string event_type: string assessment_category: success | fail | undef assessment_subclass: string affected_email_address: string? original_message_id: string? confidence: low | medium | high evidence_strength: none | weak | medium | strong | negative | ambiguous occurred_at: timestamp? observed_at: timestamp deduplication_key: string raw_message_ref: string? notes: - string ``` ## 11. Message Classification Rules ### 11.1 Hard Bounce Signals: ```text Delivery failure notice Permanent failure Unknown user Mailbox not found Domain not found 5xx SMTP status Enhanced status code 5.x.x ``` Normalized event: ```text notification.endpoint.rejected_permanent ``` Assessment: ```text category: fail subclass: fail.hard_bounce ``` ### 11.2 Soft Bounce Signals: ```text Temporary failure Mailbox full Greylisting Temporary server failure 4xx SMTP status Enhanced status code 4.x.x ``` Normalized event: ```text notification.endpoint.rejected_temporary ``` Assessment: ```text category: undef subclass: undef.deferred ``` ### 11.3 Delayed Delivery Notice Signals: ```text Delivery delayed Will keep trying Message not yet delivered ``` Normalized event: ```text notification.endpoint.deferred ``` Assessment: ```text category: undef subclass: undef.deferred ``` ### 11.4 Final Delivery Failure Signals: ```text Could not deliver after retry period Final failure Giving up ``` Normalized event: ```text notification.endpoint.rejected_permanent ``` Assessment: ```text category: fail subclass: fail.expired_without_delivery ``` ### 11.5 Out-of-Office Reply Signals: ```text Auto-reply Out of office Vacation Abwesenheitsnotiz Ich bin nicht im Büro ``` Normalized event: ```text interaction.out_of_office_received ``` Assessment: ```text category: undef subclass: undef.out_of_office ``` ### 11.6 Human Reply Signals: ```text Non-automated reply No bounce markers No auto-reply markers Human-written body likely ``` Normalized event: ```text interaction.reply_received ``` Assessment: ```text category: success subclass: success.reply_received ``` Note: this is email-channel success, not necessarily coordination success. ### 11.7 Complaint / Abuse Signals: ```text Abuse report Spam complaint Feedback loop Complaint notification ``` Normalized event: ```text notification.channel.complaint_received ``` Assessment: ```text category: fail subclass: fail.complaint_received ``` ### 11.8 Unsubscribe / Opt-Out Signals: ```text Unsubscribe request Opt-out Remove me STOP ``` Normalized event: ```text notification.channel.unsubscribe_received ``` Assessment: ```text category: fail subclass: fail.unsubscribed ``` ### 11.9 Unknown Return Message Signals: ```text Message appears related to return mailbox but no reliable classification is possible ``` Normalized event: ```text notification.endpoint.unknown ``` Assessment: ```text category: undef subclass: undef.conflicting_evidence or undef.no_signal ``` ## 12. CSV Report Format ### 12.1 Required Columns The timestamped CSV report shall include: ```text report_generated_at scan_id mailbox_id mailbox_message_id mailbox_received_at source_from source_to source_subject message_id_header detected_message_class normalized_event_type assessment_category assessment_subclass affected_email_address original_message_id original_recipient smtp_status_code enhanced_status_code reason_code confidence evidence_strength occurred_at observed_at first_seen_at last_seen_at deduplication_key raw_message_ref notes ``` ### 12.2 Filename Convention ```text email-channel-evidence-report-YYYYMMDD-HHMMSS.csv ``` Example: ```text email-channel-evidence-report-20260602-173000.csv ``` ### 12.3 Optional Secondary Reports The MVP may also generate: ```text email-channel-summary-report-YYYYMMDD-HHMMSS.csv email-endpoint-quality-report-YYYYMMDD-HHMMSS.csv email-parse-failures-YYYYMMDD-HHMMSS.csv ``` ## 13. Deduplication Strategy ### 13.1 Message Deduplication Message deduplication should use: ```text mailbox_id imap_uid message_id_header received_at from_address subject hash body hash if available ``` ### 13.2 Evidence Deduplication Evidence deduplication should use: ```text mailbox_message_id parser_version normalized_event_type affected_email_address original_message_id smtp_status_code enhanced_status_code reason_code ``` ### 13.3 Rescan Behavior A rescan should not duplicate existing evidence. Rescans should support: ```text same parser version → preserve previous result unless raw message changed new parser version → create new parse result version report-only-new → export only newly discovered evidence full report → export all current evidence ``` ## 14. Storage Plan For MVP, use local SQLite. Suggested tables: ```text mailbox_scans mailbox_messages parsed_messages evidence_candidates endpoint_quality scan_cursors raw_event_refs ``` This keeps the MVP simple while preserving a path to a server database later. ## 15. Parser Implementation Plan ### 15.1 Parser Pipeline ```text load raw message → parse headers → parse MIME parts → identify DSN/report parts → extract original recipient → extract final recipient → extract original message id → extract SMTP/enhanced status codes → classify message → create parsed message record → create evidence candidate ``` ### 15.2 Parser Layers Implement parsers in layers: ```text HeaderParser MimeParser DsnParser BounceHeuristicParser AutoReplyParser ComplaintParser UnsubscribeParser HumanReplyHeuristicParser EvidenceMapper ``` ### 15.3 Parser Versioning Every parse result shall include: ```text parser_version ``` This allows rescanning old messages after parser improvements. ## 16. Evidence Mapping Initial mappings: | Parsed class | Normalized event | Assessment | | ------------------------- | ------------------------------------------- | ------------------------------- | | `hard_bounce` | `notification.endpoint.rejected_permanent` | `fail.hard_bounce` | | `soft_bounce` | `notification.endpoint.rejected_temporary` | `undef.deferred` | | `delayed_delivery_notice` | `notification.endpoint.deferred` | `undef.deferred` | | `final_delivery_failure` | `notification.endpoint.rejected_permanent` | `fail.expired_without_delivery` | | `out_of_office` | `interaction.out_of_office_received` | `undef.out_of_office` | | `human_reply` | `interaction.reply_received` | `success.reply_received` | | `complaint_or_abuse` | `notification.channel.complaint_received` | `fail.complaint_received` | | `unsubscribe_or_opt_out` | `notification.channel.unsubscribe_received` | `fail.unsubscribed` | | `unknown_return_message` | `notification.endpoint.unknown` | `undef.conflicting_evidence` | | `parse_failed` | no event or diagnostic event | `undef` | ## 17. Endpoint Quality Updates The scanner should update basic endpoint quality. Examples: | Evidence | Endpoint quality update | | ------------- | ---------------------------------------------------------- | | Hard bounce | `reachability = unreachable`, `last_failure_at = now` | | Soft bounce | `reachability = degraded`, `last_failure_at = now` | | Complaint | `suppression_state = suppressed` | | Unsubscribe | `suppression_state = opted_out` | | Human reply | `last_success_at = now` | | Out of office | `last_auto_reply_at = now`, `reachability = uncertain` | Endpoint quality is diagnostic and must not be treated as coordination success. ## 18. Work Packages ## T01 - Repository Bootstrap ```task id: EMAIL-WP-0002-T01 status: done priority: high state_hub_task_id: "3a17215d-62a9-48ef-877f-a6fbc7e95a22" ``` Tasks: ```text Create repo structure Add INTENT.md Add or update spec/ProductRequirementsDocument.md Add this MVP workplan Set up basic build/test tooling Add initial CLI entry point Add config loading ``` Acceptance: ```text Project can run a placeholder CLI command. Config file is loaded and validated. ``` ## T02 - Mailbox Connector ```task id: EMAIL-WP-0002-T02 status: done priority: high state_hub_task_id: "25a4da12-1bcd-4c6d-a0eb-a2f525b9c4b9" ``` Tasks: ```text Implement IMAP connection Support TLS Read credentials from environment variables List/select configured folder Fetch message metadata Fetch full message source Support max_messages_per_run Support dry-run mode ``` Acceptance: ```text CLI can connect to mailbox and list/fetch messages without modifying mailbox. ``` ## T03 - Scan State and Storage ```task id: EMAIL-WP-0002-T03 status: done priority: high state_hub_task_id: "16b95a6b-1375-4c91-8b78-0b75d51e0aeb" ``` Tasks: ```text Add SQLite state store Create schema Store scan records Store mailbox messages Store scan cursor Implement message deduplication Implement incremental scan Implement full rescan ``` Acceptance: ```text Running scanner twice does not duplicate mailbox messages. Full rescan can revisit all messages while preserving deduplication. ``` ## T04 - MIME and Header Parsing ```task id: EMAIL-WP-0002-T04 status: done priority: high state_hub_task_id: "5a50cd85-b0ab-4017-aba0-b2087068abb4" ``` Tasks: ```text Parse RFC message headers Parse Message-ID Parse From/To/Subject/Date Parse MIME body parts Extract text/plain Extract text/html fallback Extract message/delivery-status parts where present Store raw headers reference Optionally store raw body reference ``` Acceptance: ```text Scanner extracts basic metadata and text from representative bounce and reply messages. ``` ## T05 - Bounce and DSN Parser ```task id: EMAIL-WP-0002-T05 status: done priority: high state_hub_task_id: "8ea826d1-0add-4573-9bb4-2b73adefba55" ``` Tasks: ```text Detect delivery-status notifications Extract original recipient Extract final recipient Extract action Extract status Extract diagnostic code Extract remote MTA if present Classify hard vs soft bounce Map SMTP/enhanced status codes ``` Acceptance: ```text Representative hard and soft bounce samples are classified correctly. ``` ## T06 - Auto-Reply and Human Reply Classifier ```task id: EMAIL-WP-0002-T06 status: done priority: high state_hub_task_id: "4d94a332-173b-4787-8fb2-27aa63db6a8d" ``` Tasks: ```text Detect out-of-office patterns Detect auto-reply headers Detect common German and English OOO phrases Detect human reply fallback Classify challenge-response as separate class if possible ``` Acceptance: ```text Representative OOO and human reply samples are classified with confidence. ``` ## T07 - Complaint and Unsubscribe Classifier ```task id: EMAIL-WP-0002-T07 status: done priority: high state_hub_task_id: "8637d383-25f7-45b5-9680-427ed2ca87bf" ``` Tasks: ```text Detect abuse/complaint messages Detect unsubscribe/opt-out requests Map to channel complaint/unsubscribe events Create suppression candidates ``` Acceptance: ```text Representative complaint and unsubscribe examples are classified. ``` ## T08 - Evidence Candidate Generation ```task id: EMAIL-WP-0002-T08 status: done priority: high state_hub_task_id: "6d62dea0-f416-4c0b-80a0-7c16422b8e5f" ``` Tasks: ```text Map parsed classes to normalized event types Generate EmailEvidenceCandidate records Assign assessment category/subclass Assign evidence strength Assign confidence Generate evidence deduplication key ``` Acceptance: ```text Parsed messages produce evidence candidates according to the mapping table. ``` ## T09 - Endpoint Quality Updates ```task id: EMAIL-WP-0002-T09 status: done priority: medium state_hub_task_id: "0d110877-953f-4aa2-961b-eec81e0159d4" ``` Tasks: ```text Create endpoint_quality table Update endpoint quality from evidence Track last failure/success Track suppression state Track reason code history ``` Acceptance: ```text Hard bounce updates endpoint quality to unreachable. Complaint/unsubscribe updates suppression state. ``` ## T10 - CSV Report Generator ```task id: EMAIL-WP-0002-T10 status: done priority: medium state_hub_task_id: "5ab35176-d6c2-4c73-b7b3-bde4c097e3ee" ``` Tasks: ```text Generate timestamped CSV report Include required columns Support report-only-new Support full evidence report Support parse failure report Write deterministic headers ``` Acceptance: ```text Running scanner creates a CSV report with evidence rows. Report can be opened in spreadsheet tools. ``` ## T11 - Golden Test Corpus ```task id: EMAIL-WP-0002-T11 status: done priority: high state_hub_task_id: "514fa099-781b-4590-aae4-c28970413b3f" ``` Tasks: ```text Create test fixture directory Add synthetic hard bounce Add synthetic soft bounce Add delayed delivery notice Add final failure Add out-of-office Add human reply Add unsubscribe Add unknown return message Add parse failure sample ``` Acceptance: ```text Automated tests verify expected classification and normalized event output. ``` ## T12 - Minimal Documentation ```task id: EMAIL-WP-0002-T12 status: done priority: medium state_hub_task_id: "a5f7067e-87be-4438-ba35-b12d06a8181e" ``` Tasks: ```text Add README quickstart Document config file Document CLI commands Document CSV format Document evidence mapping Document limitations ``` Acceptance: ```text A developer can run the scanner against a test mailbox or fixture directory. ``` ## 19. MVP Milestones ### Milestone 1: Scan and Store Goal: ```text Connect to mailbox, fetch messages, store metadata, deduplicate. ``` Includes: ```text T01 T02 T03 ``` ### Milestone 2: Parse and Classify Goal: ```text Parse messages and classify bounces, OOO, replies, complaints, unsubscribe. ``` Includes: ```text T04 T05 T06 T07 ``` ### Milestone 3: Evidence and Reports Goal: ```text Generate normalized evidence candidates and timestamped CSV reports. ``` Includes: ```text T08 T09 T10 ``` ### Milestone 4: Confidence and Repeatability Goal: ```text Add golden tests, parser versioning, and documentation. ``` Includes: ```text T11 T12 ``` ## 20. MVP Acceptance Criteria The MVP is complete when: 1. A user can configure access to one IMAP mailbox. 2. The scanner can run without modifying mailbox contents. 3. The scanner can perform incremental scans. 4. The scanner can perform full rescans. 5. Already-seen messages are deduplicated. 6. Hard bounces are classified. 7. Soft bounces are classified. 8. Delayed delivery notices are classified. 9. Out-of-office replies are classified. 10. Human replies are classified. 11. Complaints or unsubscribe messages are classified where detectable. 12. Unknown messages are preserved as unknown rather than ignored silently. 13. Evidence candidates are generated. 14. Endpoint quality is updated. 15. A timestamped CSV report is produced. 16. Golden tests cover representative sample messages. 17. The report format aligns with the `email-connect` evidence model. 18. The implementation does not overclaim email evidence. ## 21. Design Rules ### 21.1 Do Not Overclaim The scanner must not infer more than the mailbox evidence supports. Examples: ```text No bounce found ≠ delivery success Out-of-office ≠ recipient completed action Human reply ≠ legally valid acceptance Unknown message ≠ failure ``` ### 21.2 Preserve Unknowns Unknown and parse-failed messages should be visible in reports. ### 21.3 Prefer Evidence Over Status The scanner should produce evidence rows, not only final statuses. ### 21.4 Make Rescans Safe Rescans should be safe, deduplicated, and parser-version-aware. ### 21.5 Keep Raw References Store enough raw reference data to allow later inspection. ## 22. Suggested Initial Repository Structure ```text email-connect/ INTENT.md spec/ ProductRequirementsDocument.md workplans/ EMAIL-WP-0002-mvp-mailbox-evidence-scanner.md docs/ EmailAdapterSpecification.md src/ email_connect/ cli/ config/ mailbox/ parsing/ evidence/ reporting/ storage/ tests/ fixtures/ hard_bounce/ soft_bounce/ delayed_delivery/ out_of_office/ human_reply/ unsubscribe/ unknown/ test_mailbox_scanner.py test_bounce_parser.py test_evidence_mapping.py config/ mailbox.example.yml reports/ .gitkeep ``` ## 23. Future Extensions After MVP Possible next steps: ```text Provider webhook ingestion Outbound send API Template manager Minimal UI Multi-mailbox scanning OAuth mailbox access Mailbox write-back actions Advanced DSN parsing Advanced German/English auto-reply classification Natural-language reply intent extraction Suppression export coordination-engine evidence event push Endpoint quality dashboard Provider-specific bounce mappings ``` ## 24. Summary The Mailbox Evidence Scanner MVP is a strong first implementation slice for `email-connect`. It delivers immediate practical value by converting a return mailbox into structured email-channel evidence reports. It also validates the core `email-connect` evidence model before implementing outbound sending, provider webhooks, or full coordination-engine integration. The guiding rule is: > Scan the mailbox, preserve the evidence, classify conservatively, report clearly, and never treat missing evidence as delivery success.