email-connect/spec/ProductRequirementsDocument.md

ProductRequirementsDocument.md

1. Product Name

email-connect

2. Product Intent

email-connect is a headless email communication and evidence service.

It provides practical tooling for sending, tracking, querying, diagnosing, and managing email communication while also implementing the adapter contract needed to integrate naturally with coordination-engine.

The product treats email as a useful but uncertain communication channel. It can provide evidence that an email was accepted, rejected, bounced, opened, clicked, replied to, complained about, unsubscribed from, or suppressed, but it must not overclaim what email can prove.

The core principle is:

Email events are evidence, not result satisfaction. email-connect reports email-channel facts and uncertainty. coordination-engine evaluates intended results.

3. Executive Summary

Modern applications frequently need to send email, track what happened, diagnose delivery problems, and connect email events to larger business processes. Existing email providers expose useful events, but their event models are provider-specific and often easy to overinterpret. For example, a provider status called delivered may only mean that the recipient mail server accepted the message. It does not prove inbox placement, human awareness, payload access, or business result completion.

email-connect solves this by providing a provider-neutral email communication layer with an evidence-oriented model. It supports email sending, event ingestion, message timelines, bounce classification, suppression management, endpoint-quality diagnostics, and conservative evidence assessment.

The product should be useful independently through a generic API and UI, while also serving as the first reference adapter for coordination-engine.

In standalone mode, email-connect helps users and systems manage email communication.

In adapter mode, email-connect implements AdapterInterfaceSpecification.md and emits normalized evidence events for coordination-engine.

4. Problem Statement

Applications often integrate directly with email providers. This creates several problems:

1. Provider lock-in Each provider has its own API, event names, webhook structure, status model, error codes, and suppression behavior.

2. Weak semantic clarity Provider statuses such as processed, sent, delivered, deferred, dropped, or failed can mean different things across systems.

3. Overclaiming risk Applications may incorrectly treat email server acceptance as successful notification or recipient awareness.

4. Poor evidence traceability Raw provider events, normalized states, and business conclusions are often mixed together.

5. Limited operational visibility Support and operations teams need to inspect what happened to a message, but provider dashboards are often not enough or not integrated with business context.

6. Weak handling of uncertainty Email does not guarantee inbox delivery or human awareness. Systems need to represent unresolved and suspicious states explicitly.

7. Missing adapter discipline To fit into coordination-engine, email needs a strict adapter boundary with capability declaration, evidence ceilings, status mapping, late-event handling, and advisory assessment.

email-connect addresses these problems by creating a reusable, provider-neutral, evidence-aware email communication service.

5. Product Vision

email-connect should become a practical, headless email capability service that can be used by applications, humans, and agents to send, manage, inspect, and reason about email communication.

It should also become the reference implementation for how *-connect adapters integrate with coordination-engine.

The long-term vision is:

A developer can send email through a provider-neutral API,
inspect a complete message timeline,
understand what the evidence does and does not prove,
and feed normalized evidence into coordination-engine.

6. Core Product Principle

Email is a weak-to-medium evidence channel for awareness.

email-connect must distinguish:

provider acceptance
recipient mail-server acceptance
temporary deferral
hard bounce
soft bounce
async bounce
provider drop
suppression
complaint
unsubscribe
open tracking
proxy/privacy open
scanner click
unverified click
human-like reply
out-of-office response

It must avoid treating technical email delivery as human awareness.

Important distinctions:

provider accepted ≠ recipient server accepted
recipient server accepted ≠ inbox placement
inbox placement ≠ human awareness
open event ≠ human reading
click event ≠ authenticated recipient action
email reply ≠ legally valid acceptance

7. Target Users

7.1 Primary Users

Application Developers

Developers who need a provider-neutral way to send and track transactional or notification emails.

Operations Teams

Users who need to inspect email delivery timelines, diagnose bounces, investigate complaints, and understand uncertain delivery states.

Product Teams

Teams building communication-heavy workflows such as secure document delivery, account notifications, onboarding, payment reminders, contract workflows, or incident alerts.

Automation Agents

Agents that need to send, query, inspect, or classify email communication through a stable API.

coordination-engine

The consuming orchestration runtime that uses email-connect as a notification and evidence adapter.

7.2 Secondary Users

Support Teams

Users who need to answer questions such as “Did this email go out?” or “Why did the recipient not receive it?”

Compliance and Audit Stakeholders

Stakeholders who need traceability from raw provider events to normalized evidence.

Integration Partners

Teams integrating different email providers or internal systems.

8. Key Use Cases

8.1 Send Transactional Email

A client sends an email through a generic API without binding directly to a specific provider.

8.2 Send Coordination Notification

coordination-engine sends a notification email through email-connect and receives normalized evidence events.

8.3 Inspect Message Timeline

An operator opens a message detail view and sees the full timeline:

created
rendered
adapter accepted
provider accepted
MX accepted
opened by proxy
scanner-like click
no authenticated portal access

8.4 Diagnose Bounce

A support user inspects a failed message and sees whether the bounce was hard, soft, temporary, mailbox-not-found, domain-not-found, policy-related, or authentication-related.

8.5 Manage Suppression

A user or system suppresses an address due to hard bounce, complaint, unsubscribe, manual policy, or provider suppression.

8.6 Classify Engagement Signals

The system distinguishes proxy opens, scanner clicks, unverified clicks, human-like opens, replies, and out-of-office responses.

8.7 Query Email Evidence

An automation agent queries all messages for a participant, endpoint, correlation ID, or coordination case.

8.8 Maintain Endpoint Quality

The system tracks endpoint quality signals such as syntax validity, bounce history, complaint history, suppression state, catch-all suspicion, and last engagement.

8.9 Integrate Provider Webhooks

A provider webhook posts events into email-connect, which verifies, stores, normalizes, and emits evidence.

8.10 Provide Operational UI

A user views messages, timelines, endpoint quality, suppressions, provider health, and evidence classifications.

9. Scope

9.1 In Scope

email-connect shall provide:

• Provider-neutral email send API.
• Message and attempt records.
• Provider abstraction.
• Provider webhook/event ingestion.
• Raw event reference preservation.
• Native status mapping.
• Normalized evidence event generation.
• Bounce classification.
• Complaint and unsubscribe handling.
• Suppression management.
• Endpoint quality model.
• Message timeline API.
• Message evidence assessment API.
• Provider health diagnostics.
• Adapter descriptor and health endpoints.
• coordination-engine adapter contract implementation.
• Minimal operational UI.
• Golden tests for overclaim prevention.

9.2 Out of Scope for MVP

The MVP does not need to provide:

• Full marketing automation.
• Newsletter list management.
• Visual campaign builder.
• A/B testing.
• Advanced deliverability warmup.
• CRM functionality.
• Full workflow engine.
• Legal delivery guarantee.
• Built-in document portal.
• Payment or signature handling.
• Natural-language reply classification.
• Multi-provider routing optimization.

These may be added later or integrated through other systems.

10. Conceptual Model

10.1 EmailMessage

A logical email created by a client or coordination case.

EmailMessage:
  email_message_id: string
  coordination_case_id: string?
  participant_id: string?
  purpose: string?
  subject: string?
  template_ref: string?
  content_ref: string?
  tracking_context: TrackingContext
  created_at: timestamp

10.2 EmailAttempt

One send attempt for one message through one provider/configuration.

EmailAttempt:
  email_attempt_id: string
  email_message_id: string
  provider_name: string
  provider_account_ref: string?
  from_endpoint: EmailEndpoint
  to_endpoint: EmailEndpoint
  cc_endpoints:
    - EmailEndpoint
  bcc_endpoints:
    - EmailEndpoint
  provider_message_id: string?
  adapter_operation_id: string?
  state: EmailAttemptState
  created_at: timestamp
  updated_at: timestamp

10.3 EmailEndpoint

A sender or recipient email endpoint.

EmailEndpoint:
  endpoint_id: string?
  email_address: string
  display_name: string?
  endpoint_role: from | to | cc | bcc | reply_to | return_path
  verification_state: unknown | syntax_validated | verified | suspected_invalid | invalid
  suppression_state: active | suppressed | complaint_suppressed | bounce_suppressed | unsubscribed | unknown
  metadata: object?

10.4 EmailEvent

A native or normalized event in the email lifecycle.

Examples:

message.created
message.rendered
message.render_failed
message.accepted_by_adapter
message.accepted_by_provider
message.rejected_by_provider
message.queued
message.provider_dropped
message.provider_suppressed
message.deferred
message.endpoint_accepted
message.soft_bounced
message.hard_bounced
message.async_bounced
message.opened
message.opened_proxy_like
message.clicked
message.clicked_scanner_like
message.replied
message.out_of_office_received
message.complaint_received
message.unsubscribe_received
message.suppression_added

10.5 EmailEvidenceAssessment

An email-native assessment of what the email channel currently indicates.

EmailEvidenceAssessment:
  email_message_id: string
  participant_id: string?
  category: success | fail | undef
  subclass: string
  confidence: low | medium | high
  strongest_signal: string?
  evidence_summary:
    - string
  recommended_coordination_interpretation: string?

The assessment is advisory. coordination-engine remains authoritative for participant and case state.

11. Email Evidence Model

11.1 Success

Email-level success means that the email channel produced useful positive email-channel evidence. It does not necessarily mean coordination success.

Possible subclasses:

success.endpoint_accepted
success.reply_received
success.human_like_open
success.human_like_click

11.2 Fail

Email-level fail means strong evidence that the email channel failed or should not be used for this endpoint/attempt.

Possible subclasses:

fail.invalid_address
fail.missing_address
fail.render_failed
fail.template_invalid
fail.provider_rejected
fail.provider_dropped
fail.provider_suppressed
fail.hard_bounce
fail.domain_not_found
fail.mailbox_not_found
fail.authentication_rejected
fail.policy_rejected
fail.content_rejected
fail.message_too_large
fail.complaint_received
fail.unsubscribed
fail.suppressed
fail.expired_without_delivery

11.3 Undef

Email-level undef is the normal state when the adapter cannot infer reliable human awareness.

Possible subclasses:

undef.pending
undef.provider_accepted_only
undef.queued
undef.deferred
undef.delayed
undef.endpoint_accepted_only
undef.no_signal
undef.weak_positive
undef.proxy_open
undef.scanner_click
undef.unverified_click
undef.identity_uncertain
undef.mailbox_possibly_reached
undef.possibly_spam_or_quarantine
undef.possibly_silent_drop
undef.forwarding_unknown
undef.catch_all_domain
undef.conflicting_evidence
undef.channel_degraded
undef.recipient_reported_missing
undef.out_of_office

The undef category must not be empty because email does not provide guaranteed delivery or awareness evidence.

12. Evidence Ceiling

By default, email-connect shall declare this evidence ceiling:

EvidenceCeiling:
  max_positive_event: interaction.unverified_actor_interaction
  max_positive_strength: medium
  can_prove_human_awareness: false
  can_prove_payload_access: false
  can_prove_payload_delivery: false
  can_prove_identity: false
  can_prove_authority: false
  can_prove_non_repudiation: false
  limitations:
    - Email provider delivery does not prove inbox placement.
    - Open tracking can be caused by privacy proxies.
    - Click tracking can be caused by security scanners.
    - Replies may indicate awareness but do not automatically prove identity or authority.

Stronger evidence should come from other systems, such as:

• portal login
• authenticated document download
• payment settlement
• signature completion
• verified acknowledgement

13. Functional Requirements

13.1 Email Sending

The system shall accept provider-neutral send requests.

The system shall support:

• sender endpoint
• recipient endpoint
• subject
• text body
• HTML body
• template reference
• variables
• tracking context
• idempotency key
• optional attachments or content references
• provider selection metadata

The system shall return an action result that indicates whether the send request was accepted by email-connect.

13.2 Idempotency

The system shall require idempotency keys for externally visible send actions.

Repeated requests with the same idempotency key shall not create duplicate sends.

If the same idempotency key is used with different message content, recipient, sender, template, or tracking context, the system shall return an idempotency conflict.

13.3 Provider Abstraction

The system shall support provider modules.

Provider modules shall handle:

• sending
• provider message IDs
• webhook validation
• event ingestion
• bounce parsing
• complaint events
• unsubscribe events
• suppression events
• open and click events
• provider health

The core model shall not be hardcoded to one provider.

13.4 Provider Event Ingestion

The system shall ingest provider events through webhooks, API polling, or simulated event input.

The system shall:

• preserve raw event references
• deduplicate events
• correlate events to messages and attempts
• normalize native provider events
• record evidence events
• handle late events
• handle out-of-order events

13.5 Native Status Mapping

The system shall maintain provider-specific native-to-normalized event mappings.

Example:

provider delivered → notification.endpoint.accepted

not:

provider delivered → recipient aware

If a native event is ambiguous, the system shall use the weakest safe mapping.

13.6 Bounce Classification

The system shall classify bounces.

Bounce classes should include:

hard
soft
transient
unknown

Bounce reasons should include:

mailbox_not_found
domain_not_found
mailbox_full
message_too_large
authentication_failed
policy_rejected
content_rejected
reputation_rejected
rate_limited
greylisted
temporary_server_failure
suppressed
unknown

13.7 Suppression Management

The system shall support suppression records.

Suppression types should include:

hard_bounce
complaint
unsubscribe
manual
provider_policy
unknown

Suppression scopes should include:

global
tenant
sender_identity
campaign
coordination_case

13.8 Open and Click Classification

The system may support open and click tracking.

It shall classify such events conservatively.

Open classifications:

human_like_open
proxy_open
privacy_proxy_open
image_proxy_open
security_scanner_open
unknown_open

Click classifications:

human_like_click
scanner_click
link_prefetch
multi_link_scan
unverified_click
unknown_click

Scanner/proxy classification shall be heuristic unless provider-specific evidence is strong.

13.9 Reply and Auto-Reply Handling

The system should support inbound replies and auto-replies where available.

Classifications should include:

human_reply
out_of_office
auto_reply
ticket_system_reply
bounce_like_reply
unknown_reply

13.10 Endpoint Quality

The system shall maintain email endpoint quality.

Signals may include:

syntax_valid
domain_exists
mx_exists
verified
catch_all_suspected
role_address_suspected
disposable_suspected
suppression_state
last_successful_endpoint_acceptance_at
last_hard_bounce_at
last_complaint_at
last_engagement_at

Endpoint quality is diagnostic information. It does not by itself prove coordination success.

13.11 Message Timeline

The system shall expose a message timeline.

EmailMessageTimeline:
  email_message_id: string
  email_attempts:
    - email_attempt_id: string
      provider_name: string
      provider_message_id: string?
      events:
        - EmailTimelineEvent
  current_assessment: EmailEvidenceAssessment

13.12 Evidence Event Generation

The system shall emit normalized evidence events compatible with AdapterInterfaceSpecification.md.

Required normalized events include:

notification.attempt.accepted_by_adapter
notification.attempt.rejected_by_adapter
notification.attempt.accepted_by_provider
notification.attempt.rejected_by_provider
notification.attempt.queued
notification.attempt.delayed
notification.endpoint.accepted
notification.endpoint.deferred
notification.endpoint.rejected_temporary
notification.endpoint.rejected_permanent
notification.endpoint.unreachable
notification.channel.complaint_received
notification.channel.unsubscribe_received
notification.channel.suppression_added
notification.channel.suppression_removed
interaction.notification.opened
interaction.proxy_or_privacy_interaction
interaction.scanner_or_bot_interaction
interaction.unverified_actor_interaction
interaction.reply_received
interaction.out_of_office_received

13.13 Adapter Contract Implementation

The system shall implement the adapter contract for coordination-engine.

It shall expose:

• AdapterDescriptor
• AdapterCapabilityProfile
• EvidenceCeiling
• AssuranceCapability
• AdapterHealth
• action handling
• evidence event output
• late-event policy
• native status mapping
• endpoint quality
• golden tests

13.14 Generic UI

The system shall provide or enable a generic operational UI.

Initial UI views should include:

Message list
Message detail
Message timeline
Evidence assessment
Endpoint quality
Suppression list
Provider health
Event search
Bounce diagnostics
Complaint diagnostics
Adapter compatibility

14. Non-Functional Requirements

14.1 Headless First

The product shall be API-first and usable without a UI.

14.2 Practical UI

The UI shall be an operational companion to the API, not the only way to use the product.

14.3 Provider Neutrality

The core model shall not depend on one provider.

14.4 Evidence Correctness

The system shall underclaim rather than overclaim.

14.5 Auditability

The system shall preserve enough raw event references and normalized mappings to reconstruct why a message assessment was derived.

14.6 Reliability

The system shall support retries, duplicate event handling, late events, and out-of-order events.

14.7 Security

The system shall protect provider credentials, webhook secrets, tracking tokens, message content, and endpoint data.

14.8 Privacy

The system shall support metadata-only mode, event redaction, message-content retention controls, endpoint masking, and deletion/anonymization workflows where appropriate.

14.9 Agent-Friendliness

The system shall provide clear schemas, stable APIs, event mappings, and golden tests.

15. Generic API Requirements

15.1 Standalone API

The product should expose conceptual operations such as:

POST /email/send
GET /email/messages
GET /email/messages/{id}
GET /email/messages/{id}/timeline
GET /email/messages/{id}/assessment
GET /email/endpoints/{id}/quality
POST /email/suppressions
DELETE /email/suppressions/{id}
GET /email/channel-health
GET /email/events

15.2 Adapter API

The product should expose conceptual adapter operations such as:

GET /adapter/descriptor
GET /adapter/health
POST /adapter/actions
POST /adapter/events/provider
GET /adapter/events

The exact transport may evolve, but the conceptual operations should remain stable.

16. Generic UI Requirements

The UI should support:

16.1 Message List

A searchable overview of messages with filters for:

• time range
• recipient
• sender
• provider
• status
• assessment category
• bounce state
• suppression state
• coordination case
• participant
• correlation ID

16.2 Message Detail

A detail view showing:

• message metadata
• provider attempt information
• recipient endpoint
• provider message IDs
• current evidence assessment
• evidence ceiling warning
• raw provider references

16.3 Timeline View

A chronological event timeline showing:

• adapter events
• provider events
• normalized evidence events
• late events
• event confidence
• evidence strength

16.4 Endpoint Quality View

A diagnostic view showing:

• endpoint validity
• MX/domain signals
• bounce history
• suppression state
• complaint history
• engagement history

16.5 Suppression Management View

A view for adding, removing, and inspecting suppressions.

16.6 Provider Health View

A view for provider configuration, webhook status, recent failures, and channel health.

16.7 Adapter Compatibility View

A view exposing:

• adapter descriptor
• capability profile
• evidence ceiling
• emitted events
• supported actions
• conformance level
• known limitations

17. MVP Scope

The MVP should prove both standalone value and coordination-adapter value.

17.1 MVP Functional Scope

The MVP shall include:

1. Message creation.
2. Idempotent send request handling.
3. Simulated provider or one real provider.
4. Provider event ingestion.
5. Message and attempt storage.
6. Native status mapping.
7. Evidence event generation.
8. Bounce classification.
9. Suppression records.
10. Endpoint quality basics.
11. Message timeline API.
12. Message assessment API.
13. Adapter descriptor.
14. Adapter health endpoint.
15. Basic operational UI.

17.2 MVP Required Events

The MVP shall support at least:

notification.attempt.accepted_by_adapter
notification.attempt.rejected_by_adapter
notification.attempt.accepted_by_provider
notification.attempt.rejected_by_provider
notification.endpoint.accepted
notification.endpoint.deferred
notification.endpoint.rejected_temporary
notification.endpoint.rejected_permanent
interaction.proxy_or_privacy_interaction
interaction.scanner_or_bot_interaction
interaction.unverified_actor_interaction
notification.channel.complaint_received
notification.channel.suppression_added

17.3 MVP Acceptance Criteria

The MVP is acceptable when it can:

1. Accept a send request.
2. Dispatch or simulate an email.
3. Preserve correlation and idempotency.
4. Prevent duplicate sends for duplicate idempotency keys.
5. Ingest or simulate provider events.
6. Produce normalized evidence events.
7. Classify provider acceptance as weak send evidence.
8. Classify MX acceptance as weak technical evidence.
9. Classify hard bounce as strong failure evidence.
10. Classify proxy opens and scanner clicks as ambiguous.
11. Provide a message timeline.
12. Provide an email evidence assessment.
13. Provide endpoint quality basics.
14. Manage suppressions.
15. Expose adapter descriptor and health.
16. Provide a minimal inspection UI.
17. Integrate with coordination-engine without overclaiming success.

18. Golden Test Requirements

The product should include golden tests for overclaim prevention.

Required test scenarios:

provider accepted does not mean endpoint accepted
provider delivered does not mean human awareness
MX accepted does not mean inbox delivery
no bounce does not mean delivery success
hard bounce maps to permanent endpoint rejection
soft bounce maps to temporary endpoint rejection or deferral
proxy open does not mean human open
scanner click does not mean recipient clicked
unverified click does not mean authenticated recipient action
reply received is awareness evidence but not authority proof
late async bounce is accepted and recorded
duplicate idempotency key does not send twice
same idempotency key with changed content causes conflict
suppressed endpoint blocks send or marks attempt failed

19. Risks

19.1 Overclaiming Risk

The system may accidentally treat weak email signals as strong evidence.

Mitigation:

• evidence ceiling
• golden tests
• conservative mappings
• explicit UI warnings

19.2 Provider Lock-In

The system may become too tied to one provider.

Mitigation:

• provider abstraction
• native mapping modules
• provider-neutral message model

19.3 Tracking Misinterpretation

Opens and clicks may be caused by proxies or scanners.

Mitigation:

• classify proxy/scanner-like events
• preserve uncertainty
• avoid strong success mapping

19.4 Privacy Risk

The system may store sensitive message content or endpoint data.

Mitigation:

• metadata-only mode
• retention controls
• masking
• raw event redaction

19.5 Operational Complexity

Event ingestion, deduplication, and late events may become complex.

Mitigation:

• append-oriented event storage
• deduplication keys
• raw event references
• idempotent processing

20. Open Questions

1. Which provider should be implemented first?
2. Should the MVP use a simulated provider before a real provider?
3. Should message content be stored by default or referenced externally?
4. How should inbound replies be ingested in the first version?
5. How much click/open tracking should be built into the MVP?
6. Should template management be internal or external initially?
7. Should suppression state be global, tenant-scoped, sender-scoped, or all three?
8. Should endpoint quality be shared across tenants or isolated?
9. What is the first UI technology stack?
10. Should email-connect provide an event bus output directly?
11. How should provider webhook security be configured per provider?
12. What minimum conformance level should the first coordination-engine integration require?

21. Strategic Positioning

email-connect is not merely an email-sending wrapper.

It is:

provider-neutral email communication tooling
+ operational email diagnostics
+ email evidence normalization
+ endpoint quality and suppression management
+ coordination-engine reference adapter

It should be useful for any system that needs practical email communication visibility, while also proving the adapter model for the broader coordination framework.

22. Guiding Statement

email-connect sends and tracks email, normalizes provider events, exposes operational APIs and UI, manages endpoint quality and suppressions, and integrates with coordination-engine by reporting conservative email-channel evidence under uncertainty.