Dealing for data:
 The ‘ins and outs’ of negotiating for electronic records




Just as with dealing with paper documents, getting electronic information from a government can be difficult. But following a few strategies and learning the “ins and outs” of data can help make it a little easier. Here are some things you should know when accessing electronic records:

Know the law. Know how your state treats (or doesn’t treat) electronic information and what the exemptions to the open records act are.

Know what information you want. Don’t ask an agency to provide you everything it has. Make sure your request is narrow and specific. Also, you don’t always have to ask for the data first. Request a list of its databases. Then ask for record layouts for specific databases. Then ask for the data. One helpful tactic is to request the agency’s Y2K compliance reports, which often list all databases affected. Some of that information may now be outdated, but it could be a good starting point.

Know how the information is kept. Try to find someone in the information systems department at an agency who knows how the information is kept. The public affairs officer that you usually deal with may not know the details of the databases.

Know what the appropriate cost should be. In most states, you should really have to pay only duplication costs. If they hit you with a high price, ask them to itemize the costs — frequently this request alone will decrease the fee.

Know who does the data entry. The best resource to any database are the data entry clerks.. They can tell you details about how the information is input and how often it is updated.

Know who administers the data. The persons in charge of the database can be much more helpful that the agencies’ public relations persons. Sometimes they can be pretty excited to have someone take an interest in their database. Long before you ever need the data, tour the agency’s data processing center — get to know the folks you need to meet. Go to software users groups — there are usually some data processing folks from government agencies there. Get hard copy summary reports. Summary reports will help you to check your data and may give you hints about what data the agency keeps.

Know how many records or pieces of information are in the database. When you get the database, make sure you have the right number of records.

Increasingly, reporters face two major hurdles in gaining access to databases. Agencies will claim that release of data on named individuals intrudes upon their personal privacy. And agencies will try to sell public record data for profit.

It is increasingly difficult in many states to get any data on individual people — and there is little public support for access to such data. Be persistent at getting the data while it is still available and get together with other news organizations to fight impending closure of these databases.

Insist that you should only pay for reproduction costs. Neither government agencies nor outside companies who process their data should be profitting from the sale of public data. Wealthier news organizations may sometimes pay more than cost just to get the data more quickly, without challenging unfair costs. However, there is a danger that those sales set a precedent that hurts overall access to public information.

In some cases, the law does not clearly say whether you have a right to a particular database. That does not mean you should not try to get it. In fact, news organizations have done important stories with data that was not necessarily a public record. When a Miami Herald found voter fraud in the mayoral election, the records were not open records at that time.

The North Andover Eagle-Tribune in Massachusetts found public officials who were getting welfare checks by analyzing a welfare database that was leaked to them by a source.

In other cases, you may be able to argue that you are entitled to at least portions of the database. Find out whether a researcher or another government agency has ever gotten the records.

Argue that the public has an interest in the disclosure. Identify important important stories that have been done using the records you seek or similar records in other jurisdictions. Remind officials that they are accountable to the public. A politician interested in “cleaning up government” might release records. Find similar records that have been released.

Making your request

Determine whether an agency will be cooperative before using tactics that might be seen as confrontational. A records custodian may provide records you need without requiring a formal request.

If the agency makes electronic records available to other agencies, it may produce them for a requester. Many agencies now make computer terminals available for public use.

Develop a good “paper trail.” As you appeal an adverse decision, good documentation of your request and the agency response will be valuable.
Show how your request is reasonable by including information you learned earlier. If state statutes support your request, cite them. If any court decisions from your state support your request, cite those as well. Alternatively, cite other state or federal cases, especially those from jurisdictions that are nearby or have public records laws similar to yours.

Address fees. Because of the potential costs of computer access, it is important to discuss fees in your request letter. Ask that the agency incur no costs until it advises you, or state the maximum you are willing to pay. Ask for fee waivers where appropriate. Offer to discuss your request with agency officials to avoid delays.

Follow up any denial with oral negotiations. Ask a records custodian if you can narrow a request to satisfy the custodian’s concerns or reduce fees. Try to negotiate lower fees if necessary.


Determine if an administrative appeal is available. Some states allow, or even require, review by another public official before an appeal to a court. Explain in detail why your request is, or ought to be, covered by the state open records law and why a denial is unlawful or unreasonable.

Pursue a judicial appeal if necessary. Your case may warrant an appeal to a court. Seeking legal representation is best, but if you cannot afford an attorney, you may be able to pursue your own appeal.

All evidence you have gathered will be important at this stage. You must also address cases that weaken your position. If a case is contrary, demonstrate how your situation is different from that considered in the earlier case.

Overcoming common excuses

The previous guidelines are all dandy, until you actually ask for the data. Many times you will run into excuses. Reporters may, more often than not, just accept the excuses agencies give them for keeping the data to themselves. But often sheer persistence will pay off.

Our database is too complicated.
Knowing the lingo will pay off. If you understand databases and the officials can see that - you will have more success getting a database.

Our computer system can’t output a file.
It would be unusual for this statement to be true, but it does happen. If you can’t find someone at the agency who can figure out how to export the data, find out what software they use. You might have to call a vendor directly to get information on exporting from its software.

Some reporters run into cases where an agency uses software with which they are familiar and can talk the agency through the exporting process.

The person who knows how to do that is on vacation for two weeks /doesn’t work here anymore.
If you are not willing to wait, you may have to find another tactic. Does another agency keep the same data? Could the agency’s software vendor help you? If they realize that you are not going to go away, they may be more willing to work with you.

It will cost you $20,000.
Know the law and how fees can be assessed in your state. Ask for an itemized estimate of charges. Find out what it really costs agencies to reproduce the data.

Offer to pay reasonable programming fees. In most states, you should only have to pay the programmers their hourly rate, not overhead.

If they do programming, ask for a copy of the program or at least have them put in writing that they will save the program in case you need the data again next year. See if there is a rate charged by state agencies to other agencies. Provide your own tapes, disks or CDs. Ask for a backup tape.

In some cases, this actually might be the price charged to that agency from a larger state data processing center. For example, in California many agencies are charged such fees from the Teale Data Center.

The database is not public record.
The burden is on the agency, not you, to show where in the open records law the information is exempt from disclosure. If only part of the information is exempt, most states require that they redact the confidential information, and give you the rest.

We don’t like what you plan to do with it.
Most open records laws do not require that you disclose what you plan to do with the data. Some states ask reporters to complete data disclosure forms asking what the purpose of a request is. In these cases, be as vague as possible. You may not know everything you plan to do with the data.

We don’t keep that on computer.
Make sure that is true. If you have a computer-generated report from the agency, the agency probably does keep the underlying data on its computer. However, you may actually run into cases where records are not computerized. In that case, you may want to have a data entry firm input the information or enter it yourself. Some journalists scan data, but scanning numbers can introduce errors into your data. For instance, it is very easy for a “2” to become a “7” in the scanning process.

That uses proprietary software.
You do not want the software, you want the data. If agencies do not know how to copy to a file or print to a file, find out who their vendor is.

We don’t mind giving you a few records, we just don’t want to give you the whole database.
When the San Jose Mercury News sued for a pet license database, the judge asked the agency’s attorney: “You mean if they wanted ONE record from ONE person it would be okay?” The attorney answered: “Depends on the person.” The Mercury News lost the suit on a finding that disclosure would invade pet owners’ privacy. However, in most states, if one record is a public record, all of them should be.

Learn to talk like a nerd

Megabytes, density, ASCII. Mention these terms and many journalists turn pale. But knowing a few computer terms can help unlock the door to a wealth of information. Suppose you were interested in investigating federal contracts granted to companies in your area.

You could examine summary reports created in an agency office, but you might want to look at the records themselves. Imagine sorting through thousands of documents. Making sense of such information would take a very long time. Looking at that same information on a computer makes it easier, if you know what you’re doing.

The information in your computer is a bunch of little on and off switches that form different patterns to stand for different things. Each switch is called a “bit” — that stands for Binary DigIT. It’s how the computer stores information: 1 or 0, off or on, yes or no.

Information is just like your old secret decoder ring where a certain pattern would stand for a certain letter. For example, J51 might have stood for the letter A. Your computer does the same thing with different patterns of bits.

In order to have enough combinations for all the possible letters, numbers and symbols, your computer needs eight slots — that group of eight on/off switches is called a byte. That secret decoder language that your personal computer uses is called ASCII. So, for instance, the bits 01001000 make up a byte with the value of 72; the ASCII value of 72 is “H”

Another type of language is EBCDIC. That takes more work to translate, but if it is all you can get, take it.
And, for those of you who want to sound especially techno-savvy in your negotiations: ASCII stands for American Standard Code for Information Interchange and EBCDIC stands for Extended Binary Coded Decimal Interchange Code.

You also may have the opportunity to get data in a variety of formats including CD-ROM, DVD, tape cartridges, 9-track tapes and diskettes. You will need to make sure your computer can read some types of media such as 9-track tapes.

In addition to the data, you also will need some hard-copy information. You must have a record layout — that is the map to your data. You also need to know some format information about the data.

Once you have determined if your file is ASCII or EBCDIC (or is one of those rare files already in a database or spreadsheet), you need to know if the information is arranged in fixed or delimited fields.

A fixed length file has everything in columns; delimited files typically have commas separating each piece of data, so your database or spreadsheet program knows which piece to put into the first column, second column and so on.

With this information, you can import the information into a spreadsheet or database program and, using your record layout information, start compiling useful information from what started as a bunch of computerized bits.