, I constructed a totally automated clever doc processing (IDP) system that ran within the cloud on Amazon Internet Companies. The system was designed to permit Freedom of Data-type requests from Irish residents desirous to know whether or not their particulars (title, deal with, and so forth.) have been recorded in a big insurance coverage claims database maintained by my employer.
At a high-level, the workflow schedule was this,
- Consumer solicitors despatched an e mail to a particular Outlook deal with requesting a search of the shopper’s particulars within the database. For the search, we would have liked the primary and final title of the shopper, along with their date of delivery and present deal with
- A search may solely proceed if there have been corroborating paperwork hooked up to the e-mail to “show” who the shopper was.
- The attachments would usually be PDFs containing pictures of passports, driving licences, and different official paperwork. As well as, we may additionally obtain MS Phrase paperwork or PDFs of financial institution statements and utility payments
- At a specified time every day, the IDP processed these emails, extracted and labeled the attachments and extracted any related Personally Identifiable Data (PII), which was despatched to a claims database for looking out.
- If declare particulars have been discovered within the database, a password-protected PDF containing these particulars was created and despatched again to the solicitor, together with a separate e mail containing the password to unlock it. If no outcomes have been discovered, a easy e mail was despatched again indicating this.
Other than a Human-In-The-Loop (HIL) course of that allowed customers to carry out a ultimate cross-check as as to whether the PII extracted from completely different paperwork matched, the method was totally automated.
By the way, this new course of proved roughly 90% extra environment friendly than the guide system it changed.
On this article, I’m going to point out you the steps I took to implement this method on the AWS cloud platform, albeit a barely simplified model of it.
Observe that aside from being a consumer of Amazon Internet Companies (AWS), I’ve no affiliation or affiliation with the corporate.
In the actual system I developed, our supply of emails got here from a company Outlook e mail account. I don’t have entry to a type of now, so as a substitute, I’ll use my private Gmail as our e mail server and assume every e mail accommodates just one attachment. I’m assuming that any attachment will at all times be a picture file (JPG or PNG) of a passport, driving licence, or financial institution assertion. Additionally, the ultimate step in our course of might be a easy outcome file on S3 that signifies the classification and PII extraction values for the attachment.
Because of the size of the code snippets and so forth., I’ll embody a hyperlink on the finish of the article to a GitHub repo, the place yow will discover all of the Lambda code, the Step perform code, IAM permissions, and all the things else required for this mission.
As that is primarily in regards to the AWS facet of issues, I’m going to imagine that all the things is ready up in your e mail shopper to allow an AWS Lambda course of to learn your emails.
For Gmail, briefly, this course of would come with:-
- creating or deciding on a Google Cloud mission
- enabling the Gmail API
- enabling OAuth and finishing a one-time Google consent circulate.
After that, you’ll retailer the returned credentials/refresh tokens securely in one thing like AWS Secrets and techniques Supervisor.
The purpose is to extract the next items of PII from every e mail attachment.
Doc Kind, First Title, Final Title, DoB and Tackle
Our check Gmail attachment pictures
I requested Codex to create 3 clearly faux pictures of a passport, a driver’s licence, and a financial institution assertion. Though faux, they comprise the identical data as the actual factor. Listed below are these pictures.
I emailed these pictures to my Gmail account and labelled the emails “DocumentProcessing”.
We now have all the things in place to start out creating our IDP.
Listed below are the AWS companies we’ll use to implement our IDP.
- IAM for permissions
- EventBridge for scheduling
- Step perform for orchestration
- Lambda for compute
- S3 for storage
- Secrets and techniques Supervisor for storing OAuth and different secret data
- Textract for OCR
- Bedrock for AI inference
1/ Storing our OAUTH credentials securely
The very first thing we have to do is retailer our Gmail OAuth credentials and refresh token in AWS Secrets and techniques Supervisor. For me, my e mail setup returned a JSON doc that seemed like this.
{
"put in": {
"client_id": "my_client_id.apps.googleusercontent.com",
"client_secret": "MY_CLIENT_SECRET",
"project_id": "my_project_id",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token"
}
}Open the AWS administration console and go to the Secret Supervisor display screen. Click on the “Retailer a brand new secret” button and select “Different sort of secret” as the key sort. Underneath the key/values pairs part, choose the Plaintext TAB and enter your JSON from above. So your display screen ought to appear like this,
Click on Subsequent, then enter an acceptable title and outline to your new secret. Click on Subsequent, then Subsequent once more, earlier than clicking the Retailer button.
2/ Arrange our S3 buckets
We’ll have one general-purpose bucket and 5 folders that may maintain our information.
- uncooked: that is the place the uncooked e mail attachments might be downloaded to
- textract: it will maintain the textual content returned by Textract
- outcomes: it will maintain the outcomes of the doc classification and PII data
- audit: logging data
- quarantine: any unclassified pictures/paperwork might be positioned right here
3/ Creating our Lambdas
We’ll want three.
i) ingest_email_image retrieves eligible e mail messages and writes every picture to our uncooked folder on Amazon S3.
ii) extract_text sends the S3 pictures to the Amazon Textract service and shops the extracted textual content.
iii) classify_and_extract_pii sends that extracted textual content to Amazon Bedrock Claude Sonnet 4.6, which makes an attempt to categorise the doc sort and extract any related PII and writes the outcome.
Right here’s a simplified diagram of the method we’ll construct.
EventBridge Scheduler
|
v
Customary Step Capabilities workflow
|
+-- Ingest Gmail picture attachments -> S3 uncooked/
|
+-- Map: one e mail/picture at a time
|
+-- Textract Lambda -> S3 textract/
|
+-- Bedrock Lambda -> S3 outcomes/
|
+-- Document successful or failure outcomeLambda 1 – ingest_email_image
The primary Lambda polls our configured Gmail label utilizing read-only OAuth credentials from AWS Secrets and techniques Supervisor. It validates that every e mail accommodates precisely one JPEG or PNG attachment, saves legitimate pictures underneath the S3 uncooked/ folder, and returns outputs for the following processing stage; invalid or duplicate attachments are reported individually.
After this Lambda runs, it is best to see the three picture information saved underneath the uncooked folder of your S3 bucket.
Additionally notice that the Lambda makes use of the next atmosphere variables.
Variable Worth
-------- ------
DOCUMENT_BUCKET YOUR_OUPUT_BUCKET_NAME
GMAIL_LABEL_NAME YOUR_GMAIl_LABEL_NAME
GMAIL_SECRET_ARN YOUR_SECRETS_MANAGER_ARN
LOG_LEVEL INFO
MAX_IMAGE_BYTES 10000000
MAX_MESSAGES_PER_RUN 25Lambda 2 – extract_text
The second Lambda takes the pictures saved underneath uncooked/ and runs Amazon Textract’s synchronous document-text detection on them. It saves the extracted textual content, particular person strains, confidence scores, and supply metadata as JSON underneath the S3 textract/ folder,
Here’s a prettified, shortened model of the standard output after I ran this in opposition to the financial institution assertion picture.
{
"schema_version":1,
"processed_at":"2026-06-25T11:04:19.850574+00:00",
"supply":{
"bucket":"MY_BUCKET",
"image_key":"uncooked/financial institution.png",
"gmail_message_id":"19efe3cd80025b99",
"gmail_attachment_id":"ANGjdJ-NRkTD-45NFgNRZfHyWSKezw5MI03xN7bvTs4PZKuDjbMCZPk--bnSp4lMVldYhISyd64jH9JgN5cJXTm2_KVDFDsrIFDWJAWs7K6CtMJvsb9OYNX7GZSnwptK_U6M3npTu_02mNbOY_Q-PTKt58rcYbIOA7aDbOxlrAqFh3NmoT69FV92oMjy6NqaT1uSXXZY40h62Ofp-JT4s2JGLKCymhi_o49eAPvBj8azguB4WyoowqWiQEh_cL5IdI7uJC7GEIu9LoUZYuxr7AHZBwACeasd-sNbKLYUb7FwDhPpla_iGD9aDyncH8Q",
"mime_type":"picture/png"
},
"textract":{
"model_version":"1.0",
"document_metadata":{
"Pages":1
},
"strains":[
{
"text":"UK Bank Statement",
"confidence":99.98
},
{
"text":"bank",
"confidence":99.99
},
{
"text":"Unastign Uiver Pol, 01426",
"confidence":93.85
},
...
...
...
{
"text":"Terms UK Banks statement fan Their Band condition, dlesstery. AT stalewaly for diescavey fevien 720, 00553 Dear",
"confidence":93.75
},
{
"text":"Contrined Parih Bank ellamty of allimd elines and ptke and corv. llartes hier. uniedarfc lard sand colunrty mp conmontivre",
"confidence":79.02
},
{
"text":"tharter m allar mpercian the Cant'g paymer Is amowe Sane UK Banklila the Serd Promonry à a corditions of larril and",
"confidence":78.91
},
{
"text":"Statemant of Teard of thre D2. Carpltionrnes Other whitan and Contiiomed Earm Conditions of Terms witte pittier tied Oswllas",
"confidence":89.62
},
{
"text":"Terms and Condition, lod Cporl and Balance.",
"confidence":82.19
}
],
"textual content":"UK Financial institution StatementnbanknUnastign Uiver Pol, 01426nUttesline 952016nVobue. 4346nContasc! 84 6207nContasc! 16777982nMame HoldernAccountnJohn SmithnJohn Smithn11 The Excessive StreetnNewTownnNT1 3WEnAccount NU83632nNomenStatement PeriodnSort CodenValidn01/05/2026 to 01/06/2026nStatement PeriodnOpening Balancen01/05/2026 to 01/06/2026nClosing BalancenTransactionsntonDatenDescriptionnAmountnBalancen07906TNnDirect Debitsn£72,000n£59,000n080161BnCard Paymentsn£73,000n£95,000nCard Paynditn090061NnSalary Creditn£95,000n£56,000n064061BnSalary Creditn£23,000n£64,000n095061DnATM Withdrawalsn£60,000n£60,000n075061NnATM With drawalyn£60,000n£64,000n374091DnATM Vithdrawalyn£60,000n£63,000n340061DnATM Withdrawalsn£60,000n£73,000n153381DnATM Withdrawalyn£60,000n£13,000n150491DnATM Withdrawalyn£10,000n£13,000n169010PnATM Withdrawalsn£11,000n£19,000n188191DnATM Withdrawalsn£10,000n£95,000n11111TDnSalary Creditn£60,000n£55,000n11211TDnATM Withdrawalsn£60,000n£95,000nPage NumbernTerms UK Banks assertion fan Their Band situation, dlesstery. AT stalewaly for diescavey fevien 720, 00553 DearnContrined Parih Financial institution ellamty of allimd elines and ptke and corv. llartes hier. uniedarfc lard sand colunrty mp conmontivrentharter m allar mpercian the Cant'g paymer Is amowe Sane UK Banklila the Serd Promonry à a corditions of larril andnStatemant of Teard of thre D2. Carpltionrnes Different whitan and Contiiomed Earm Circumstances of Phrases witte pittier tied OswllasnTerms and Situation, lod Cporl and Stability."
}
}Lambda 3 – classify_and_extract_pii
The ultimate Lambda takes every block of JSON textual content that Textract scraped from the enter pictures and makes use of AWS Bedrock to categorise the doc sort and extract any PII it may well glean. The important thing to success on this step is the mannequin used and the immediate handed to it. For my work mission, I used Claude Sonnet 4.6 and a really giant immediate as a result of it needed to deal with extra complicated inputs. However for this extra simple instance, whereas we’ll nonetheless use Sonnet 4.6, we are able to get away with a a lot less complicated immediate.
First, we have to notice the mannequin ARN related to Sonnet 4.6 within the area you’re working in. You do that from the Bedrock console, so choose that, then click on on the Inference Profiles hyperlink on the left-hand menu bar. Seek for Sonnet 4.6 and notice the suitable ARN. As that is an inference profile, additionally, you will must pay attention to the areas that AWS can route inference calls by, as these will must be added to your IAM permissions along with the inference profile permission.
Listed below are the ultimate outputs I obtained after processing the financial institution, passport and driver’s licence attachments.
Financial institution attachment
{
"schema_version":1,
"processed_at":"2026-06-25T11:04:23.221887+00:00",
"supply":{
"bucket":"MY_BUCKET",
"textract_key":"textract/financial institution.png.json"
},
"mannequin":{
"model_id":"us.anthropic.claude-sonnet-4-6"
},
"classification":"bank_statement",
"pii":{
"first_name":"John",
"last_name":"Smith",
"date_of_birth":null,
"deal with":"11 The Excessive Road, NewTown, NT1 3WE"
}
}Passport attachment
{
"schema_version":1,
"processed_at":"2026-06-25T11:04:21.829715+00:00",
"supply":{
"bucket":"MY_BUCKET",
"textract_key":"textract/passport.jpg.json"
},
"mannequin":{
"model_id":"us.anthropic.claude-sonnet-4-6"
},
"classification":"passport",
"pii":{
"first_name":"AVERY",
"last_name":"EXAMPLE",
"date_of_birth":"1991-08-30",
"deal with":"14 Instance Lane, Testford, TE1 2ST"
}
}Driver’s licence attachment
{
"schema_version":1,
"processed_at":"2026-06-25T11:04:22.507427+00:00",
"supply":{
"bucket":"MY_BUCKET",
"textract_key":"textract/driving.jpg.json"
},
"mannequin":{
"model_id":"us.anthropic.claude-sonnet-4-6"
},
"classification":"driving_licence",
"pii":{
"first_name":"AVERY",
"last_name":"EXAMPLE",
"date_of_birth":"1991-08-30",
"deal with":"14 Instance Lane, Testford, TE1 2ST"
}
}This Lambda makes use of the next atmosphere variables.
Variable Worth
-------- ------
BEDROCK_MODEL_ID us.anthropic.claude-sonnet-4-6
DOCUMENT_BUCKET MY_BUCKET
LOG_LEVEL INFO
MAX_INPUT_CHARS 30000
MAX_JOBS_PER_RUN 25The Step Operate
Though for this straightforward instance we may in all probability simply daisy-chain the three Lambdas to run one after one other in code, the very best apply is to make use of AWS’s orchestration software, referred to as Step. Step capabilities have the added benefit of being retryable after failures and elegantly dealing with errors and timeouts.
AWS makes use of a language referred to as the Amazon State Language (ASL) to outline their Step Capabilities. The ASL code is within the repo. Inside Step, you possibly can render a course of right into a helpful circulate diagram, which, in our case, seems to be like this.
It would look a bit difficult, however numerous that’s the typical scaffolding that surrounds a course of like this comparable to error dealing with. It’s principally simply orchestrating the three Lambdas by calling them one after the opposite.
The EventBridge scheduler
To tie all the things collectively, I wanted a solution to kick off the Step perform every day. A straightforward manner to try this on AWS is by way of a service referred to as EventBridge. EventBridge is able to many issues, however certainly one of its most helpful capabilities is to arrange a cron primarily based timing occasion that may name different AWS methods at a specified date and time. On my work mission, I set a scheduled activity to run at 11:55 PM every night time to catch all emails within the inbox for that day, so I’ll repeat that right here.
Click on on the EventBridge service from the primary console, then click on the Schedule menu merchandise on the left-hand bar. On the display screen show, click on the “Create schedule” button. You need to see a display screen like this
Kind in a reputation and outline to your schedule. Depart the “Schedule group” discipline because the default, then select the “Recurring schedule” choice. Choose your required Timezone and go for the cron-based schedule. From right here, fill out the fields as you’ll for a cron job.
For our instance of Monday by Friday at 11:55 PM, this is able to be
Subsequent, set a versatile time window if you need, together with non-compulsory begin/finish dates. Click on the Subsequent button and choose “AWS Step Capabilities” as your goal. Choose the Step perform title you need to run from the drop-down checklist and enter any inputs.
Click on Subsequent. On this display screen, enter NONE for the motion to take after the occasion has completed. You may also enter a lifeless letter queue (DLQ), encryption particulars, and IAM permissions.
Click on Subsequent as soon as once more to go to the Evaluate display screen, and in case you’re proud of all the small print, click on the “Create schedule” button.
Your schedule will now be “dwell”, and your Step perform will execute each weekday at 11:55 PM.
What’s wanted to show this right into a manufacturing system?
So, what I’ve proven you on this article is the naked bones of an automatic IDP system. Wanting again on the real-life IDP system I applied, listed here are the adjustments you would wish to correctly productionise this.
- Deal with extra forms of attachment paperwork, e.g., MS Phrase, plain textual content, PDF, pictures, and handwritten materials.
- Deal with a number of attachments per e mail.
- Code a front-end HIL in order that customers can cross-check the returned PII from completely different paperwork to make sure they match.
- Have the HIL entrance finish robotically set off the following automated stage, e.g. a separate Step perform that implements …
- Sending the PII to the back-end database for search.
- Automating the manufacturing of PDFs containing database search outcomes.
- Routinely password-protecting the PDFs containing the database search outcomes.
- Automating the sending of the PDFs and the separate password file again to the requester.
- Making certain all the things is logged
Observe that there’s a value related to constructing and creating the system I’ve described above so in case you do observe alongside and create actual assets on AWS, please pay attention to this and delete something you not require to keep away from shock costs.
Abstract
This text walks by the method of constructing a simplified Clever Doc Processing system on AWS. The system begins with emails with a particular label in Gmail, extracts picture attachments to Amazon S3, makes use of Amazon Textract to learn the textual content, after which sends that textual content to Amazon Bedrock with Claude Sonnet to categorise the doc and extract key PII comparable to title, date of delivery, and deal with. This work is carried out by AWS Lambda capabilities.
The Lambda calls are orchestrated with a Step Operate, scheduled by EventBridge, and secured with IAM permissions and Secrets and techniques Supervisor. The result’s a sensible AWS pipeline that demonstrates how email-based doc consumption, classification and information extraction will be automated whereas remaining production-aware in design.
We coated numerous floor and doubtless over-engineered the answer primarily based on the “simplified” enter pictures we have been making an attempt to course of. For our check circumstances, we may in all probability have bypassed the AWS Textract step altogether and simply had Bedrock classify and extract the data instantly from the pictures.
I wanted the Textract step in my work mission as a result of the enter I used to be coping with was extra complicated, together with PDFs and pictures of hand-written textual content. I believe it was value conserving in that step to point out that you’ve choices in case your inputs are lower than simple.
Lastly, I discussed ed some steps you would wish to take to show the system I described into a correct, production-ready course of.
For all of the code and ancillary information, take a look at my GitHub repo on the hyperlink under. Observe that I’ve redacted ARNs, bucket names, AWS account numbers, and the rest which will pose a privateness or safety danger.
https://github.com/taupirho/document-intake-aws-pipeline
PS I’m out there for contract work simply now. In case you or somebody you already know is in search of an skilled information engineer, both distant or Edinburgh, UK-based, with expertise in AWS, AI, Python, SQL, PySpark, DuckDB, and so forth., let me know. You’ll find me on LinkedIn.
