Real talk about using OCR, NER, and LLMs to handle India's document chaos. What works, what breaks, and why it matters.
I've been tinkering with Indian legal documents for my side project, Courtbase. Scanned PDFs, handwritten notes, stamps everywhere. The kind of stuff that makes you question why we still do things this way in 2025.
But here's the thing - this isn't just a legal problem. It's everywhere in India. Banks drowning in KYC docs. Hospitals with prescription chaos. Government offices with decades of paper records. And everyone's trying to "digitize" by... scanning papers into PDFs. Yeah.
Let me break down what makes this hard:
Courts still want physical stamps. Banks need wet signatures. Government offices require attestations from gazetted officers (yes, that's still a thing). We've built digital systems on top of paper requirements. It's a mess.
I've seen land records with English headers, Hindi body text, and Kannada annotations - all on the same page. Try explaining that to your OCR model.
This drives me crazy. Instead of creating actual digital documents, we scan physical papers into image PDFs. So now you have a 10MB file that's basically a photo. Can't search it. Can't copy from it. But hey, it's "digital" right?
I tried Tesseract first. Then tried some expensive RPA tools. Here's why they all failed:
Layouts are chaos: Court orders have stamps covering text. Handwritten notes in margins. Text at weird angles. Traditional OCR just gives up.
Scan quality is terrible: Most docs are scanned on ancient machines. You get skewed pages, faded text, sometimes half the page is cut off. Good luck with that.
Zero context understanding: OCR might read "Sri Ramesh Kumar" correctly, but it has no idea that's a person's name. Or that "5,00,000/-" means 5 lakhs (not 500,000).
Here's what changed the game for us:
These models actually understand document structure. They know that number in the top-right is probably a case number. Text after "Petitioner:" is a party name. It's not just reading - it's understanding layout.
MuRIL and IndicBERT are trained on Indian languages. Finally. They get that "Kumar" is a surname, not just random text.
This blew my mind - GPT-4 Vision and Claude can just... look at the document and answer questions. Stamp covering text? Handwritten note? They handle it. Not perfect, but way better than traditional OCR.
Here's what I've been experimenting with:
For my Courtbase hobby project, I needed to extract party names, case numbers, and dates from court orders. Here's a simplified version of what worked:
# What didn't work: Basic OCR
def extract_text_basic(image_path):
# This gave us garbage with Indian docs
text = pytesseract.image_to_string(image)
return text # "Sr! R@mesh Kum&r vs Un10n 0f lnd!a"
# What I'm trying: Multi-step approach
def extract_legal_info(doc_path):
# Step 1: Use layout understanding (testing Azure's API)
# Step 2: Extract text by regions
# Step 3: Use GPT-4V for messy parts like stamps
# Step 4: Validate with regex patterns
# Indian case numbers follow patterns like "WP(C) 1234/2024"
# Still experimenting with the best combination
# The idea is to combine:
# - Layout understanding for structure
# - Vision models for complex parts
# - Regex for known patterns
# - Human review for edge cases
# Not production ready yet, but promising
Banking: A friend at a fintech told me they're using Azure Form Recognizer for PAN cards. Works 90% of the time. The other 10%? Manual review. Still saves tons of time.
Healthcare: Saw a startup demo prescription digitization. They trained a custom model on doctor handwriting (yes, that's a dataset now). Not perfect but good enough to flag drug interactions.
Government: The less said the better. Most "digitization" is still just scanning to PDF. But I've heard whispers of some states experimenting with AI for land records.
Let's be honest about what's still broken:
Especially in regional languages. I've seen models confidently read Tamil handwriting as complete gibberish. Doctor prescriptions? Forget it.
One line has English, Hindi, and Tamil. Most models just give up. You need separate models for each script, then somehow merge the results. It's a hack.
Global models don't know that "S/o" means "Son of". Or that "5,00,000" is how we write 5 lakhs. You have to build this knowledge yourself.
You're dealing with Aadhaar cards, medical records, financial docs. One leak and you're done. Most startups are winging it on security.
Here's where startups can actually win:
Don't build "AI for all Indian documents". Pick rent agreements. Or PAN cards. Or prescriptions. One thing. Make it work 99% of the time.
Someone needs to build:
I'd contribute to this.
Pure ML won't work. Pure rules won't work. But regex for PAN numbers + ML for names + manual review for edge cases? That works.
Someone could build:
Simple. Focused. Valuable.
After months of hacking on this as a side project, here's my take:
Start with one document type: For Courtbase, I just focus on legal docs. That's it. Not trying to solve all of India's paper problems. Pick your niche.
Humans aren't going anywhere: The best setup I've seen is AI doing 80% of the work, humans fixing the last 20%. Anyone selling "fully automated" is lying.
Accuracy metrics are BS: "95% accurate" means nothing if it gets the case number wrong. Measure what matters - can a human trust this extraction?
Expect garbage inputs: If your model needs high-res scans, you've already lost. Real users have blurry photos taken on 2015 phones.
Indian context matters: A model trained on US documents is useless here. You need Indian training data, Indian patterns, Indian understanding.
India runs on documents. That's not changing anytime soon. But we can make it suck less.
The opportunity isn't in replacing paper - it's in making paper-based processes bearable. Every startup that picks one painful document workflow and makes it 10x faster is doing God's work.
I'm betting we'll see a wave of narrow, focused tools that just work. Not sexy. Not "AI revolution". Just tools that save someone 2 hours of mind-numbing data entry every day.
That's the future I'm building for.