FOIA Update: Printed vs. Electronic Records

Legal and technological issues explored
attempted redaction with magic marker that says out of public discussionattempted redaction with magic marker that says out of public discussion

Example of attempted redaction of printed electronic record with magic marker by the city of Ann Arbor.

As we reported in connection with a recent column analyzing the possible legal and ethical implications of email exchanges among Ann Arbor city councilmembers, The Chronicle requested additional email records from the city.

Our purpose in requesting additional records was to explore more fully the workings of the city council as reflected in emailed communications – and to compare that with the public deliberations at the council table. That public discourse is something we already describe in a fair amount of detail in our meeting reports.

The request was for a substantial amount of material, dating back to the launch of this publication in September 2008. We provided a voluntary extension to the city of two weeks – beyond the three weeks within which an agency is required to comply with a Freedom of Information Act request. That extension ended last Friday.

On Friday we picked up the email records and paid for them (around $380 total). The records had been printed out, in some parts redacted with a black pen, then photocopied. The Chronicle’s contention is that we are legally entitled to copies of the electronic files – appropriately redacted.

FOIA-ed material from the city of Ann Arbor

FOIA-ed email records from the city of Ann Arbor.

We will be pursuing  the dual paths of (i) convincing the city to provide the requested records in electronic form, and (ii) working – more slowly than we could with the electronic files – with the printed material we have to explore topics like the 601 S. Forest brownfield vote, the arguments for postponing the City Place PUD, plus how and why specific committee assignments on city council were made for the current round of appointments.

In this updater, we’ll confine our focus to laying out some of the technology issues surrounding electronic versus printed records.

What the FOIA Law Says

When a government agency complies with a Freedom of Information Act request, it must do so within certain constraints. Some of these constraints are there to prevent agencies from creating a de facto denial of access to information by raising economic barriers. In relevant part the FOIA law states:

(3) In calculating the cost of labor incurred in duplication and mailing and the cost of examination, review, separation, and deletion under subsection (1), a public body may not charge more than the hourly wage of the lowest paid public body employee capable of retrieving the information necessary to comply with a request under this act. Fees shall be uniform and not dependent upon the identity of the requesting person. A public body shall utilize the most economical means available for making copies of public records. [emphasis added] A fee shall not be charged for the cost of search, examination, review, and the deletion and separation of exempt from nonexempt information as provided in section 14 unless failure to charge a fee would result in unreasonably high costs to the public body because of the nature of the request in the particular instance, and the public body specifically identifies the nature of these unreasonably high costs. A public body shall establish and publish procedures and guidelines to implement this subsection.

To The Chronicle’s knowledge, the city of Ann Arbor has not published procedures and guidelines as required by the FOIA law. What we understand of the city’s policies in this regard comes from verbal conversations with the city attorney, Stephen Postema, together with the physical items provided by the city in response to our request.

In the first conversation with Postema, he raised the specter of charging The Chronicle as much as $25,000 in order to comply with our request – which included records of emails among councilmembers and staff for the days on which the city council met dating back to September 2008. In subsequent conversations, he clarified that he had meant to associate that figure with what it would have cost to undertake a full “disaster recovery” of all deleted files – something we had not requested.

In calculating the charges to The Chronicle, Postema said, the city was construing each of the meeting dates as separate requests. That worked out to The Chronicle’s benefit, he explained, because it was the city’s policy not to charge for the first four hours of labor associated with examination, review, and redaction associated with a FOIA request. That means that for each date on which email records were requested, the city allotted four hours of labor that it did not charge to The Chronicle.

It appears that the practical outcome of this policy was that The Chronicle was charged only for the cost of photocopies.

Electronic Format for Requested Records

Because the records we requested were stored in electronic format, that is the format we explicitly requested from the very start and in every subsequent communication we had with the city. It is The Chronicle’s position that the city of Ann Arbor is required to produce those records in their original electronic format.

Benefits of Electronic Format

Independently of legal requirements, there  are clear benefits to both the city and to The Chronicle (and its readers) for the city to provide records in electronic format.

  • Reduced Material Cost: The cost of a single CD-ROM is lower than several thousand sheets of paper, many of which bear an admonition to “Think Green!” and to print out the the email only if necessary.
  • Reduced Time Cost: The time required for copying electronic files is less than a process that entails first printing, then redacting, then photocopying again (presumably two copies – if someone else were to request the same records, it would not be efficient to undertake the same redaction).
  • Improved Accuracy and Convenience in Use: If the records are provided in electronic form, then copying and pasting from the records themselves helps ensure accuracy in re-publication of the information. If paper documents are provided, there’s a step in converting printed text (back) to digital text where errors can creep in. It’s also more convenient. Finding material among the records is more convenient with electronic documents because of the machine-searchability of electronic text.
  • Improved Accuracy and Convenience in Compliance: If redaction  is undertaken using an electronic editor on electronic files, it’s easier for the person doing the redaction – any material that appears multiple times can be deleted “all in one go.” Quality control of digital redaction is also easier than when undertaken electronically. For example, running a simple “diff” analysis on redacted versus original files would allow a reviewer to focus just on the redacted material to ensure that whatever had been deleted really met the criteria for redaction under FOIA.

Case Law on Format

While there are strong practical benefits to provision of documents in electronic format, a practical benefit does not rise to the level of a legal requirement. For that The Chronicle bases its contention that the city is required to provide the records in their original format on a 15-year-old lawsuit brought against the city of Ann Arbor. From the court records of the case.

In the fall of 1993, Plaintiff, Jon Zeeff, made a Freedom of Information Act (“FOIA”) request to Defendant, City of Ann Arbor, for current data used to produce “Traffic Condition Section Condition Reports” and “Property Information” reports. Defendant offered to provide Mr. Zeeff paper copies of the Traffic Condition Section Condition Reports and microfiche copies of the Property Information reports. Mr. Zeeff, however, requested that the data be provided to him in computer format. [Emphasis added.] Defendant declined to provide the data in the desired format. and Plaintiff initiated this action to compel its production. Defendant motions for Summary Disposition pursuant to MCR 2.116(C)(8) and (C)(10). Plaintiff responds and requests that the Court render judgment in his favor pursuant to MCR 2.116(I)(2). At a hearing held on February 23, 1994, the Court granted judgment in favor of Plaintiff as to his request for the data to produce the Traffic Condition Section Condition Reports. [Emphasis added.]

The records were ordered to be released and attorney fees of $25,000 were awarded.

The City’s Position on Electronic Documents

In conversations with The Chronicle, the Ann Arbor city attorney has taken the position that the Zeeff case does not apply because it did not involve redaction. He cited concerns that if electronic data were redacted electronically, the redacted material could be recovered through data forensics by the requester.

Based on conversations with coworkers at the Workantile Exchange, plus affiliates with the University of Michigan College of Enginering Computer Science department –  a PhD student,  an assistant professor, and a full professor – the risks of inadvertently providing electronically redacted material to a FOIA requester can be dealt with in a straightforward way.

First, it’s important to recognize the real risk of electronically editing a document, copying that document electronically and inadvertently providing to a FOIA requester a history of the changes undertaken. In ordinary, non-FOIA circumstances, provision of a document’s previous editing history is sometimes not inadvertent, but intentional – many Chronicle readers will no doubt be familiar with MS Word’s “track changes” functionality.

But email records are not as inherently complex as MS Word documents are. They can be reduced in straightforward fashion to a simple “text file” – a file that contains just the characters you see on the screen, with no fancy fonts or underlines, or any other “word processing” features. The text format used for storage of email files is commonly mbox.

Using a simple text editor, redaction of any material meeting the criteria under FOIA can be replaced with something like “[Redacted]“.  For its plain text editing needs, The Chronicle uses TextWrangler, which is available free.

For the sake of argument, let’s say the file  FOIAfile.txt is stored on drive C. If the plain text edits are undertaken to FOIAfile.txt and saved to drive C, and drive C is handed over to the FOIA requester, then there’s still some risk that the redacted information could leak to the requester.

How? When a file is saved, the computer file system could reallocate where exactly the file’s data is physically stored on the disk, so that the parts of the old version of the file could still be lying around on the disk somewhere. That is, the “redacted” data would be on the disk, just not labeled by the file system as part of the file. But because it would still be there, it could be read – though reading it would require tools somewhat more sophisticated than a text editor like TextWrangler.

That’s why it’s important that the files be saved to a different file system – like a newly formatted CD. By using a different file system to save the files to be provided under FOIA, the risk described above is eliminated.

Key to this approach is that the text files in question really are text files. That is, their size measured in bytes, should match the number of characters in the file. That’s straightforward to verify. Here’s how we verified it for a sample text file. First, we asked TextWrangler to count the characters in the file by clicking on the “I” button: 72.

Screenshot of character count from TextWrangler

Screenshot of character count from TextWrangler.

Then we asked the computer’s operation system to tell us how many bytes the file took up: 72.

screenshot of get info from MacIntosh computer

Screenshot from "Get Info" on Macintosh computer.

So the file really is a plain text file.

Where From Here?

We’ll begin working on the the printed materials we have, but it will be a slog. It’s not clear that a first step of just scanning everything and running an optical character recognition program on the whole kit and kaboodle is feasible. Some intial sorting is probably in order.

We recognize that the use of a text editor is not necessarily as universal a skill as using a black marker. But in both cases, the real skill lies in the judgment of whether the material meets the criteria for redaction, not the operation of the tool. Based on some of the batches of records provided early in the process, it’s clear that things can go wrong, even with the relatively simple technology of a black pen.

As the lead art for this article shows, the city’s attempts at redaction were in some cases unsuccessful because the pen was not dark enough – and probably also because the photocopier applied an autosharpening process to help bring out the contrast between the marker and the letters underneath. When we noticed that phenomenon, we provided an example to the city, so that the issue could be addresssed in subsequent batches.

We hope  that the city of Ann Arbor will recognize the importance of providing electronic records in electronic form when requested under FOIA.

16 Comments

  1. By hospadaruk
    June 30, 2009 at 9:39 pm | permalink

    To: Dave
    From: Bob
    Subj: What a waste of $380

    I for one, grow weary of this… “issue”? Can we move on to taking the dam down or world peace or something?

  2. By Alan Goldsmith
    July 1, 2009 at 6:07 am | permalink

    This is NOT a waste of funds. This issue isn’t about moronic insulting exchanges via email while ignoring public speakers at council meetings. It’s about how this city is governed and it’s interesting how the people who want these documents to remain hidden from the public view sound much like the Bush Whitehouse did.

    Regardless of what comes out from the documents released as part of this FOIA request, this is world class journalism and Dave Askins and the A2 Chronicle deserve a tip of the hat. The Chronicle is doing the kind of reporting the Newhouse owned News should have done for years but didn’t have the guts to do.

    Thank you!

  3. By Tom Whitaker
    July 1, 2009 at 9:00 am | permalink

    My neighbors and I would love to know what Council was typing while we were standing in front of them, pleading our case against City Place, last January. What was the mystery email that Mr. Greden announced he had received that prompted him to request a tabling of the City Place PUD? Who sent it and what did it say?

    With the hideous, ordinance violating, new version of City Place coming before Planning Commission next Tuesday, and before City Council on July 20, we would like to know what (and with whom) our Councilmembers were secretly communicating the last time this item was before them. We have gotten precious little response to our recent correspondence with Council on this matter and have been given the impression that several Councilmembers have already made up their minds about it. Any additional insight about what they are really thinking would be helpful.

    I hope The Chronicle will find a way to “slog” through the stack that will provide timely information about issues that are still up in the air, like City Place.

  4. July 1, 2009 at 9:47 am | permalink

    I found this highly instructive and informative, even without regard to the question of emailing and twittering during council meetings. Good, tough reporting, Mary & Dave.

  5. By Christina Lirones
    July 1, 2009 at 9:50 am | permalink

    The issue of public access to public documents is crucial to open, transparent, and democratic government. While I have had no difficulty receiving documents from the City of Ann Arbor (in fact, I recently sent in a compliment to the Mayor on the good work of their FOIA coordinator), I have had terrible problems with FOIA requests made to Clerk Alan Israel in Pittsfield Township. Previous Clerks (Judith Walter, me, and then Feliziana Meyer) would often provide documents free of charge at the counter or electronically by email, or at a charge of 5 cents per page for paper copies for larger requests. Labor was rarely charged, except in the case of very large requests. I remember spending large amounts of time compiling documents for FOIA requests, both as Clerk and as Treasurer, and never charging for my own time; after all, I was elected to work for the public. Now, Mr. Israel, with the ratification of the Board, has raised the cost of paper copies to 50 cents per page, and labor costs are common and excessive. For example, a copy of an audiotape of a meeting used to be just the cost of the tape, since it takes only a few minutes to put the tape in the deck and flip it as needed. The last tapes I requested cost me nearly $30 for each meeting, where previous costs were $2.50 (or less, when Ms. Meyer was the Clerk – the cost of the tape itself was recently increased). Labor was charged for the entire length of the tape, as if the individual sat and watched the wheels spin on the machine while the tape duplicated (although the first half hour of time was deducted – the Township still retains one of our FOIA policies, where requests that take less than half an hour are not charged labor, so the Clerk’s Office does deduct that half hour from the length of the tape.). For the retrieval of a handful of letters from personnel files, which should have been charged at the HR Generalist’s rate (if at all – it would have taken the HR Generalist, who filed the letters in the first place, about 15 minutes to pull them out), I was charged an hour and a half at the rate of the new Administrative Services Director. In numerous cases, the Township automatically waited 5 days to respond to requests, then automatically requested an extension of ten days, for the simplest requests. Our former Clerk requested a two page job description that had been posted on line, and the Clerk’s Office attempted to charge her nearly $40. After a lengthy email discussion, the cost was ultimately reduced to $1.00. The costs are so excessive and arbitrary, I instead now opt to seek documents from other agencies that do business with the Township whenever possible. After explaining how difficult it is to obtain public documents from the Township, I have been able to receive the documents at little or no cost from other agencies. While I have considered hiring an attorney to fight the Township’s violations of the FOIA, the costs of the court battle to recoup the amounts I have been overcharged seem overwhelming. While I am deeply saddened that the FOIA is not being followed, I don’t know how anyone can afford to defend the public’s rights to public documents. The consequence of these arbitrary and excessive fees is that I simply do not request information I would like to have, and that is a very sad situation at the level of local government.

  6. By Bill Merrill
    July 1, 2009 at 10:25 am | permalink

    Dave-

    Do you know if the documents you receive as a result of a FOIA need to be provably official documents? I’ve heard–but don’t completely understand the issue–the reason we get PDFs documents from the city website is the documents need to be (apparently) editable and not containing revealing extra data.

    PDFs can be a pain to consume but they appear to fit the need.

    I found my self wondering about a combination of text and cryptographic signature. I realize that the tools for crypto signing are well beyond either a black marker or a text editor, but it could provide us with usable and trusted information.

  7. By Brad Thompson
    July 1, 2009 at 10:45 am | permalink

    The heavy lifting done by journalists is rarely recognized. To properly do the job it requires an amazing amount of time and tedium to drill down to the information you’re trying to find.

    Good work. I have no doubt that the AA News wouldn’t expend the effort, but this absolutely has to be done. Unfortunately many people do not understand how important an independent, inquisitive fourth estate is to a strong functioning democracy.

  8. By Susan
    July 1, 2009 at 10:46 am | permalink

    I believe the $380 was money well spent and made a small donation to The Chronicle to help defray the cost. If you disagree, perhaps you’d like to make a donation to the campaign of the councilperson you think will be harmed most by these documents. And good luck with that, I say.

  9. By Susan
    July 1, 2009 at 12:14 pm | permalink

    re: #6, I think he means the documents need to be UNeditable.

  10. By a2eastsider
    July 1, 2009 at 12:14 pm | permalink

    It is disheartening to hear how many people automatically assume that elected officials are dishonest, incompetent liars who spend their time gleefully conspiring behind closed doors to “get” people. That kind of behavior really is rare though is always exploited because people love a great conspiracy theory more the mundane reality of slogging through business. The majority of people who serve are concerned citizens who are trying to do what they believe is the best thing for the people they represent. They do know more about the issues that are being raised and the parties involved because it is their job to know that. They spend a great deal of time reading meeting packets and notes and answering email from their constituents. They also get to hear from a wide range of people at meetings; some of whom show up prepared and on topic with well thought out statements, questions, and concerns and others who definitely do not.

    It looks like the Chronicle is more interested in finding out how things flow in the decision making process than they are in witch hunting and name calling, which is good. We have plenty of sources for sensationalist non-news.

  11. July 1, 2009 at 12:55 pm | permalink

    In re. # 10 — that’s not at all what I’m assuming. I’m glad the Chronicle is testing my assumptions by looking at the record, not just repeating hearsay.

  12. By Liz Margolis
    July 1, 2009 at 1:29 pm | permalink

    I am the FOIA officer for the Ann Arbor Public Schools. It has been an interesting journey for the past 6 years to wade through the many FOIA requests that come into the district. I am not at all opposed to FOIA requests and frequently find them very useful to communicate information that is not normally releasable to the public without this. I too find The Chronicle’s study of what happened in the city very interesting as far as process and flow and not the content which I personally think has been blown way out of proportion. As someone who sits through long board meetings it is hard sometimes to resist the urge to email about various topics but resist you must! Lesson learned!

    What I do have issue with is the FOIA request that feels like it is issued to create work and hardship for the organization when a simple discussion can usually prove fruitful. While FOIA law does not allow for the creation of any requested document the advent of email as the main communication tool can take hours and hours of staff time to fulfill the request.At the Ann Arbor Public Schools we waive the first $20 of costs for both time and copies for the request. While I perform the work we charge the lowest hourly rate of any district full-tiime employee ($9.27/hour) and .10cents per page. It takes a lot to go beyond the $20 waiver but believe me we sometimes produce boxes and boxes for these requests (thank you AA News!!).

    FOIA is an important tool when used properly to keep the needed checks and balances on publicly funded organizations.

  13. By a2eastsider
    July 1, 2009 at 1:35 pm | permalink

    re. #11 – I’m also glad The Chronicle is checking out how the things gets done in Ann Arbor. They seem to care about getting the story right rather than getting the right story.

  14. By Greg
    July 1, 2009 at 2:00 pm | permalink

    As Ms. Margolis alludes to, I suspect that FOIA also has an underlying goal of protecting privacy and the money of the entity’s taxpayers, particularly against frivolous FOIA requests. My guess is that many local governments and school districts regularly have to deal with harassing FOIA requests that would be never-ending if it were not for FOIA’s guidelines. Those with political motives would find FOIA otherwise particularly helpful to benefit their own cause.

  15. July 1, 2009 at 2:45 pm | permalink

    From #10: “They do know more about the issues that are being raised and the parties involved because it is their job to know that.”

    I agree. I believe that I and others, certainly including council members, do know more–let’s change that to being more informed–than most about the issues we address. And when we lack humility in our deliberations it can be seen by others as lacking integrity–i.e., “dishonest, incompetent”, etc.

    Knowing more is not the same as knowing enough. (And, of course, this is pretending that we can actually “know” anything, hence the above substitution of “informed” for “know”.)

  16. By Peter Zetlin
    July 2, 2009 at 7:46 am | permalink

    Congratulations to the Chronicle for a first rate job of reporting. Even if you believe this particular controversy is overblown, FOIA is important because it enhances government accountability. Dave’s clear, well researched piece documents how this process works. If you’re interested in government, this is a good piece to read.