Column: Chartering a Course Through Data
At the Ann Arbor city council’s Feb. 16 budget committee meeting, committee members were introduced to the city’s new data catalog. Even though it is only February, I think this will be the most significant project undertaken by the city in all of 2010.
Ann Arbor police service calls for Jan. 3, 2010. This map was built by The Chronicle in about 15 minutes using data from the city's online catalog. (Image links to fully interactive map hosted at http://www.batchgeocode.com)
At the same meeting, the budget committee also continued its discussion about the content of the monthly financial reports that the city charter requires the city administrator to provide to the council.
What ties these issues together is the idea that there’s information the city will be routinely pushing out, without anyone needing to make a special request for it.
In the case of the data catalog, it appears at first glance that the project is a kind of bonus for the citizens of Ann Arbor. That is, it could be thought of as something the city is not required by law to do, but which it’s doing anyway in the interest of transparent government.
That’s different from the monthly financial statement, which the charter explicitly requires. That issue came to the surface during the budget committee meeting, during a verbal exchange between Christopher Taylor (Ward 3) and the city’s chief financial officer, Tom Crawford. The exchange found Taylor appealing to an English word only rarely deployed as a verb: “I guess I’d stickle.”
On Stickling
CFO Tom Crawford is required by the city charter to provide to the council a monthly statement via the city administrator, Roger Fraser:
SECTION 5.6. The Controller shall be the chief accounting officer of the City. The Controller shall:
(6) Submit to the Council, through the City Administrator, by the tenth working day of each month, a statement showing the balances at the close of the preceding month, in all funds and budget items, the amount of the City’s known liabilities and budget items to which the same are to be charged, and all other information necessary to show the City’s financial condition;
After a period during which the city council apparently did not expect such a monthly report, the council’s budget committee has now begun to talk about what information should be contained in the monthly financial statement.
At the Feb. 16 budget committee meeting, Crawford said that he’d provided to councilmembers in that month’s statement the information that he interpreted the charter to require. It included encumbrances – funds that are committed for a specific use. But Crawford suggested not including encumbrances in future statements. In context it was clear that the concern was based on clean formatting and readability of the document, not a desire to shield that information from the public.
Christopher Taylor (Ward 3), however, wanted to know on what basis that information could be stricken from the monthly statement. Hadn’t Crawford just told the committee that he interpreted the charter to require its inclusion? Crawford suggested that the content of the report could reflect a consensus from council about what they wanted to see in the report: “It depends on how much of a stickler you’ll be.”
And at that, Taylor allowed: “I guess I’d stickle.” And Taylor is right to stickle. As I wrote in a previous column on the city charter:
That is, the city council cannot waive a charter requirement. And any citizen has legal standing to file suit on a charter violation to demand relief.
The ensuing discussion among the budget committee touched on the idea that the monthly statements would be made available online. Perhaps formatting and readability issues could be addressed through linking to a separate document – as opposed to embedding the information on encumbrances directly in a document.
That seems like a practical approach to take: Focus on providing the information underpinning the statement, not so much on the formatting of the statement document. Can the statement “show” the encumbrances, if there is only a link from the statement to a separate document containing the encumbrances? Probably so – that’s the kind of issue the council can work out with city staff.
But it’s not up to city staff and the council to agree to strike the encumbrances from the set of information that’s required by the charter to be provided.
Data Catalog: Records and the City Charter
At the Feb. 16 meeting, the budget committee agreed that once they are satisfied with the format of the monthly statement, the statement will be conveyed directly to councilmembers, without intermediation by the committee. Conveying it to the council – and the public – by posting it online would be a good approach.
It’s a good approach because it would easily satisfy a charter requirement that all city records, like the monthly financial statement, be public:
City Records to be Public
SECTION 18.2. All records of the City shall be public, shall be kept in City offices except when required for official reasons or for purposes of safekeeping to be elsewhere, and shall be available for inspection at all reasonable times. No person shall dispose of, mutilate, or destroy any record of the City, except as provided by law, and any person who shall do so contrary to law shall be guilty of a violation of this charter.
Of course, the city could also be compelled to produce those monthly financial statements under the Freedom of Information Act (FOIA) to anyone who asks.
By posting the monthly financial statements online, the city reduces the resources that would otherwise be required to respond to requests under the FOIA, or the city charter, that those records be produced.
The same principle applies to the information the city is providing as part of its new data catalog.
Last month, the city council’s budget committee had been told the data catalog would be coming online soon. [Chronicle coverage: "Ann Arbor's Budget Data to Go Online"] So last week’s implementation of this first draft of the data catalog – which contains much more than just financial transactional data – was expected and welcome news. In addition to financial data, for example, the catalog also contains mapping data and public safety information.
Washtenaw County government is moving toward a similar goal. Commissioners discussed their “transparency of government initiative” at a Feb. 18 working session.
The city’s data catalog is a tremendous stride forward for transparency of our local government – but it should not be analyzed as an “extra” or a “bonus” for citizens. The FOIA requires that city documents – with few exceptions – be produced on request. And independently of the FOIA, the city charter requires that city records be available for inspection at all reasonable times.
By providing availability 24/7 on the Internet, the city would certainly cover “all reasonable times.”
The data catalog, then, should be seen as a way for the city to use technology efficiently to respond globally to potential requests for access to information under the FOIA or the city charter – which citizens could already legally demand on an individual basis.
As the city looks to add to the data catalog, then, one guiding question should be: Is this information record required to be public under the charter or the FOIA? If the answer is yes, then the information is a candidate for inclusion in the data catalog. Otherwise put, everything is fair game for inclusion in the data catalog.
Is there any reason why some city records shouldn’t be prioritized for inclusion in the data catalog? Absolutely. Those data sets that would require intensive ongoing staff resources for production of the data should be a lower priority.
As head of the city’s information technology, Dan Rainey, told the budget committee on Feb. 16, the data sets that are included in this initial phase are those that can be produced in automated fashion. They’ll be on a production schedule, with no human intervention required beyond the initial setup.
Data versus Records/Reports
What’s included in the data catalog are data sets, not reports/records. The idea is that by providing information in a relatively raw state, individuals – citizens or councilmembers – who are interested in building their own reports can do so fairly easily, without introducing an additional burden to city staff.
Kevin Eyer, senior applications specialists in the IT department with the city of Ann Arbor, gives the city council budget committee a quick tour through the data catalog. (Photo by the writer.)
For example, the city’s data catalog includes a comma-delimited file containing police service calls. There are data fields for date, location, type of call, and the street address.
Why don’t they provide that data presented as a map? It’s partly because you can make your own map out of the data, if you need a map. Besides, whatever kind of map the city might create, there will always be someone who’d prefer a different kind of map – maybe someone wants to see the police calls for a specific day, like Jan. 3, 2010.
That’s what I wanted – for demonstration purposes – so I took the city’s data, and headed over to http://www.batchgeocode.com and within about 15 minutes generated the map shown at the top of this column. A few tips on preparing the police call data for mapping at batchgeocode.com:
- Add columns for city and state and fill each cell with “Ann Arbor” and “MI,” respectively
- In the street address field, replace the word “block” with a blank – the city provides the address information by block, not the specific address.
- In the street address field, insert spaces around the “&” for addresses specified by intersection.
- The batchgeocode.com process takes whatever column you name “group” and assigns the colors of the map push pins based on that. Clicking on the pushpins in the map legend causes just those color push pins to appear.
- Read through the documentation at batchgeocode.com
- If you’ve never done this before, it’ll take longer than 15 minutes.
It’s not that the city doesn’t like maps, though. Among the data sets in the catalog are various KML files that open in Google Earth. For example, there’s a perfectly drawn map of Ann Arbor’s city boundaries, which can be used as a layer in other maps that people might be interested in creating.
The demonstration of the data catalog at the budget committee meeting prompted an exclamation from Marcia Higgins (Ward 4): “It seems really user friendly!”
It is.
But the city will be looking for feedback on the data catalog – the email address DataCatalog@A2gov.org is one way to do that. And as Chronicle readers provide that feedback, I’d encourage you to bear in mind the difference between data and reports – as well as the idea that the data catalog provides information that could legally already be demanded from the city.
Dave Askins is editor of the Ann Arbor Chronicle.
A note for the mapping project for the police service calls -
The data is mapped at the block level, so “400 block of Mushroom Court” might mean 401, 435, or 499.
The dataset I saw mapped street numbers greater than 1000 into blocks that were 1000 big, not 100 big, so “2000 block of Washtenaw” might be 2001 and it might be 2999, a mile or so away.
The visual impact of this is to make downtown look like it has more crime, and the outsides of town look like they have less crime; that’s an artifact of the coding used, not an accurate portrayal of reality. You can see this most clearly in northeast Ann Arbor where there are way fewer records along Plymouth Rd visible (since they are all bunched up at 2000 plymouth)
A plausible remedy to this would be to randomize the address mapped; so when you got a record that said 400 block of Mushroom Court you’d randomly pick 435 some day and 453 some other day. It’s more useful along blocks that are bigger.
This article is a very nice explanation of the difference between reports and raw data. Raw data are more flexible, but not everyone will be able to make use of it. For those folks, the monthly crime map in the Ann Arbor Observer is a useful alternative. Otherwise, an interested citizen might have to find someone to help them massage the data, but that’s preferable to waiting for a FOIA request.
This article was easily worth my monthly volunteer subscription.
Curious that the police still use the hundred block for location. Who does that benefit? I guess this is the way that police have always done this and only now that citizens get the data is there a need to change. Does the Ann Arbor Observer need to cross check all of the police reports with reality (sounds like a hell of a service on their part–which should be unnecessary).
Thanks for another well reported out article (I guess the IT budget that a2politco.com rails about has benefits).
The “hundred block” approach to providing the crime statistics probably provides some form of privacy for the victims. I’m not sure I’d want the whole world to know that my particular house was broken into. That might give others some ideas. The “thousand block” limitation as highlighted on Plymouth Road might be a bit of over kill if victim privacy is the intention.
I went back over Dave’s sample data set and didn’t see Plymouth Road thousand blocks, so it’s not on this particular map; I was able to confirm, though, that all but one of the 1548 records in the data set available tonight at http://a2gov.org/data that includes the word BLOCK that have a 4 digit address in front of the block have either 1000, 2000, or 3000. The odd exception is a single call to an Ypsilanti address in the 7000s.
It made me laugh that there’s an apparent drug bust in that hotbed of narcotics activity, Loch Alpine. I think Google Maps got confused by “S WAGNER OR PARK LAKE AVE”–that’s probably supposed to be Dolph Park.