Engineering notes
Buyers needed one list of properties from many public records. We started with scripts. These notes are how that became signals on a grain — and the decisions that followed.
- 📌
Signals on a grain
Scripts and address lookup could inspect a parcel. They could not make one list of parcels with a court filing, an assessor column, and an absentee-owner flag. That mixed list needs a grain.
-
Queue grain, not raw HTML
After I match a signal to a parcel, I may queue that grain. I do not queue raw HTML or PDFs. The transform and the lists stay a separate track.
-
Scale sources, not requests
When sources multiply, I schedule one polite job per source. I do not speed crawls, and I do not put raw HTML on a queue.
-
The shell scripts are already the pipeline
The data path is already a medallion pipeline. An orchestrator would rewrite the scheduler, not the system.