Web Data Services: One Source of Truth
A central data service I led to production for the California Legislature, where one source holds the record of members, districts, committees, and deadlines, and every application reads from it.
01 — CONTEXT
Within the California Legislature, hundreds of applications and websites need the same core reference data: who the members are, the districts they represent, the committees they sit on, the offices and caucuses they belong to, and the deadlines and recesses that govern the calendar. For years each application kept its own hand-maintained copy, with no shared source to keep them aligned. For routine updates, that was workable.
The high-stakes moments were different: an election, the start of a new legislative session, a swearing-in. The data had to change everywhere at once, officially, and the content team updated the same member, the same assignment, the same district in application after application. The copies drifted: one record could read one way on one application and another way on the next. Giving each application its own store had been a reasonable trade when deadlines demanded it, but at scale, maintaining all those copies stopped being sustainable. The data needed one source.
02 — SOLUTION DESIGN
One service holds the reference data. Every application reads from it.
The schema has to outlast every other decision in the system. Members change. Committees re-form. New legislative sessions reset everything. I built one model that holds all of it.
It rests on two axes the institution runs on. Time, scoped through the legislative session each record belongs to. Structure, the relationships that connect members to the districts they represent, the committees they sit on, and the offices they hold. With those two in place, almost every question an application asks becomes a clean walk through the data.
Applications read the data in different ways. Some want a clean shaped read. Some want it computed, searched, or paginated. A few need raw access. I built the delivery layer to match those needs directly, so no application has to bend its shape around the wrong interface.
Updates propagate on demand. When an editor saves a change, the service tells every application reading from it to fetch the new record. There is no polling interval. There is no window in which a record is current here and stale there. The one exception is a cache still serving an old record, and a cache can be cleared.
03 — ROLE & LEADERSHIP
What I set out to do was bring this service online at a level the institution could run on for the long term, legislative session after legislative session.
A developer on the team had started a smaller version of this idea for one application, and it showed the approach could work. I built on that work and turned it into the central data service for the team and the applications that read from it.
I designed the schema. I designed the API and wrote its specifications. I built the on-demand sync that pushes a change out to every application reading from the service the moment it is saved.
The longer game was making the service something the institution could run on its own. The web content team had been updating this data across the institution's sites by hand. The service gives them a single place to make those updates instead, with no engineer in the loop for routine work. The data stays accurate because the people who know it best are the ones keeping it that way.
04 — TRADEOFFS
Centralizing the data made one service responsible for what every application reads. If a record is wrong here, it is wrong everywhere. I accepted that on purpose. The alternative was what we already had: the same record wrong in different ways from one site to the next. Uptime concentrates the same way, though the cached copy each application already holds keeps it serving the last record through a brief outage rather than going down with the service.
The delivery layer gives applications more than one way to read the data. A team building a new application has to choose which pattern matches their needs and learn its conventions. I could have forced every application through the same shape, but the flexibility is what let each application adopt the service, reading the data in the way that suited it.
05 — IMPACT
More than twenty applications read from the service: internal tools, public rosters, seating charts, gallery slideshows, deadline calendars, mapping tools, and floor session apps. The same record, managed in one place, reaches all of them.
The shift from one legislative session to the next used to be the longest lift of the year. The service makes it a routine operation now. When an election turns over a roster or a member is sworn in, the content team makes the change once and every application that reads from the service shows it.
The model outlived its first dataset. Other engineers extended the same approach to new datasets, built their own clients against the service, and shaped their own workflows on top of it. Some hardened the service further for historical records. The interface became something teams could build on rather than work around.
What I set out to do is what the service does today. The data is current. The content team keeps it accurate. The applications stay in sync. The institution runs on it legislative session after legislative session, and that is the bar I built it to meet.