Work Data platform
Near-real-time payment-data platform on AWS
Built a near-real-time payment-data ingestion pipeline from global processors, then led the platforms that ingested and analysed millions of card-not-present transactions a month.
- Role
- Technical Lead, then Engineering Manager
- Organisation
- Pagos Solutions, Inc.
- When
- 2021 – 2025
The problem
Payment data arrives from global payment processors in different shapes and on different schedules. To analyse approval rates and declines across processors, it has to be ingested continuously and normalised into one model.
My role
- As Technical Lead, I built the near-real-time payment-data ingestion pipeline and defined the cloud-native service architecture (.NET Core and Python on Fargate and Lambda).
- As Engineering Manager, I led the platforms that ingested and analysed millions of card-not-present transactions a month.
What the team delivered
- Near-real-time ingestion from global payment processors on AWS: Python, Lambda, Fargate, SQS, S3, RDS, Redshift and Athena.
- A data model feeding analytics and merchant dashboards.
- Datadog observability, SLOs, runbooks and PagerDuty rotations.
Outcome
The platforms ingested and analysed millions of card-not-present transactions a month, serving merchants including Adobe, Eventbrite, GoFundMe, McAfee and StubHub.
Ownership
The platforms belong to Pagos and were built by its engineering team, which I led. Pagos engaged me full-time, contracted through my company, which also placed one engineer on the team.