Civic Notice

For webmasters and clerks

The bot in your logs.

You found this page from a User-Agent string, which means something of ours fetched something of yours. Here is exactly what it is, what it takes, and how to slow it down or stop it.

Civic Notice is a free app that shows residents what their city, county, state and federal governments are doing: what is on the agenda, who represents them, and what was decided. It reads your published record and links every item back to your page. It has no ads and sells nothing. The overview has the rest.

Which requests are ours

Every request we make carries one of four strings, and nothing else of ours exists:

User-AgentWhat it is
civic-notice/1.0 (+https://civicnotice.net/bot)Scheduled reading. This is nearly all of it.
civic-notice scout (+https://civicnotice.net/bot)A one-off check of which records software a government runs.
civic-notice extract (+https://civicnotice.net/bot)A one-off read of a portal's own public settings, before we build anything.
civic-notice playbook (+https://civicnotice.net/bot)A health check that confirms the links we hand readers still open, and re-reads a year of a meeting list to confirm we recognise every meeting it marks cancelled.

We never disguise the client, and we never send a browser's User-Agent to look like a person. If a request claims to be us and does not carry one of these, it is not us.

What we fetch

Only what your site already publishes to anyone: your meeting list, the published agenda for each meeting, the adopted minutes, and the list of who sits on the body. Where you publish a roll call, we read that too.

  • Nothing behind a login, a paywall or a session. We have no account with any government. The only keys we hold are the free public API keys that Congress.gov and Open States issue to anyone who asks.
  • Nothing you have not published. We follow your own published paths and do not walk private directories. One exception: Congress.gov's API lists a House roll call some time after the vote, so to find new ones we ask for the next roll number until the API answers that it does not exist.
  • No writes, and no searching your site. Reads only. One platform, eScribe, answers its meeting calendar only to a date-range request sent by POST, and that is the one form post we make.

How much traffic that is

Small, and deliberately slow. Each government is re-read on its own schedule, usually every three hours for a city or county and once a day for a state, and the work is a handful of requests per new or changed meeting, not a crawl of your site.

  • One request at a time against a given host, with a pause between each wherever the host has asked for one or needs one. Two exceptions: Congress.gov's API takes three at once, the rate our own sweep measured as reliable, and the public ArcGIS map layers we read council districts from are queried one at a time without a pause.
  • A shared budget per host, not per government. Several cities often sit on one vendor's server, so the pacing is set for the server, and adding a city does not double the rate against it.
  • Published files instead of crawling, where they exist. State legislatures' records come mostly from Open States' downloadable archives rather than from legislatures' own sites; the one legislature site we read is California's bill pages, for the Legislative Counsel's Digest. Each day we read Open States' public list of which archives have changed, and about once a week per state we download a newer one: one request to its API for the file's address, then the file.
  • Your Crawl-delay is the ceiling wherever your robots.txt declares one. On one vendor that means 60 seconds between requests, and our reader is built around that rather than against it.
  • No rendering, no asset fetching. We do not load your images, fonts, stylesheets or scripts.

robots.txt, and how we actually use it

We read it, and it decides whether we read you at all. Being straight about the mechanism matters more than sounding compliant, so:

A person reads your robots.txt before we build anything for your government, and its rules — including Crawl-delay — are written into the reader for that platform. We do not re-fetch it on every run. So a rule you add today is not something our software notices by itself. Email us and we will stop the same day; that is the reliable path, and blocking the User-Agent at your edge works immediately.

If your robots.txt disallows an unnamed client from the paths your agendas live on, we treat that as a no and do not build. We have walked away from a portal for exactly that reason rather than read it anyway. If the rule turns out to be your vendor's default rather than your decision, we would rather ask you than assume either way.

If you want us to stop

Say so: support@civicnotice.net. No reason needed, no argument back. We will stop reading you and the app will show your government as not covered rather than leaving a stale copy behind — which is the outcome that actually serves your residents, and the reason we would rather hear from you than be blocked silently.

If you would rather just block it, this is the name to match:

WhereWhat
WAF / edge ruleDeny where User-Agent contains civic-notice. Takes effect at once.
robots.txtUser-agent: civic-notice → Disallow: /. Correct, and read by a person on our next look, not by our software on the next run — so pair it with an email if you need it today.

A 403 or a 429 is a clear answer and we take it as one. We do not rotate addresses, change the User-Agent, or route around a refusal.

What we do with what we read

The app is a reading tool. It shows your record and does not interpret it:

  • Your titles, verbatim. We never re-word an agenda item or a measure, and we never write a summary. Where you publish your own plain-language summary, it appears as yours.
  • Every item links to your page. We do not republish your documents; we point readers at them. In practice the app sends traffic to your portal rather than taking it.
  • No scores, grades or ratings for any official, body or government. Ever.
  • Gaps stay gaps. If your minutes record an outcome without naming each member's vote, the app says that and stops. It does not infer a vote nobody cast.
  • We say when we last read you, and that your own record is the authority if the two disagree.

If the app has something wrong about your government, that is the most useful mail we can get and we treat it as a defect, not feedback: support@civicnotice.net.

If you want us to read you

Also welcome, and also just an email. Tell us where you publish agendas. If you run records software we already read, adding you is quick. If your portal blocks unnamed clients and you are happy for us to read it, a line saying so is all we need — or your vendor can name civic-notice in the allowed list.