For business
What it is
Everything the pipeline works from: recorded meetings, uploaded documents, CRM records and events from other systems.
Why it matters
Deciding up front which sources count, and who owns them, is what makes the output trustworthy later. Every arrival is logged, so any answer can be traced back to where it came from.
Risk without it
Data arrives through side doors, nobody can say where a statement came from, and compliance reviews stall.
For engineers
How it works
Each source gets a connector: pre-signed uploads to object storage for audio and PDFs, API pulls or webhooks for CRM events. Every arrival is written as an immutable raw object plus a small metadata record (source, owner, received-at, checksum) before anything else touches it.
Typical stack
- S3 pre-signed uploads
- API Gateway webhooks
- CRM REST APIs
- EventBridge
Pitfalls & trade-offs
- Webhooks retry, so expect duplicate deliveries.
- Keep the raw input: you will want to reprocess it when models improve.
- Large files go through multipart uploads, not request bodies.
In this example
In
→ meeting_0412.m4a fact-find.pdf crm: client A-17
Out
3 raw inputs received 09:14
Fictional sample · not a client system