Open by Default [beta]
An open data playbook to help you get started in your city or county.
When public data is made freely available in open, standardized formats, it can drive transparency, community engagement, and accountability. Governments around the country are building a culture and commitment to openness in City Hall across departments by making government data openly and easily available to citizens — and supporting open data with process and technology.
In this guide, we’ll explain some of the practical tactics and considerations you’ll need to know to get an open data initiative off the ground in your city or county, drawing upon the experience and expertise of other governments who have successfully done so.
This guide is in beta and we'll be continuing to develop it. Help us improve this resource by submitting an issue or pull request on GitHub, or email us with comments and suggestions: gov-staff [at] codeforamerica [dot] org.
Introduction: What is open data, and why bother?
Government data is a valuable public resource that, when accessible by every community member, can be a powerful tool to support the goals and values of the community. Cities and counties around the country are making an effort to ensure the data they hold is freely and easily available to the public by embracing and institutionalizing the practice of “open data.”
What is open data? Basically, open data refers to data (such as documents, databases, records, or transcripts, including those managed by outside vendors) released by a government or organization that is:
- freely available to be used, shared, and reused by anyone for any purpose, commercial or otherwise.
- available in digital, machine-readable formats (such as .csv) so that it can be used in combination with other data and applications.
- available in its entirety — and able to be downloaded “in bulk” and not just manually retrieved record-by-record.
There are many benefits of opening data. By opening data in machine-readable formats, governments can drive internal efficiency, spark community engagement, and fuel a civic tech ecosystem —to name just a few of the reasons your government might decide to pursue an open data initiative.
Many individuals and organizations contributed their knowledge and expertise to this guide. In particular, we would like to thank: Jim Craner, Peter Koht, Amy Mok, Dave Guarino, Tim O’Reilly, David Eaves, Laura Meixell, Andrew Crow, Mark Headd, Rebecca Williams, Mark Leech, Tim Welsh, Jenny Park, Tim Moreland, Daniel Hoffman, Millie Crossland, Ian Kalin, Jack Madans, Mike Migurski, Garrett Jacobs, the Sunlight Foundation, and the Open Knowledge Foundation.
Laying the Groundwork for Open Data
Define the goals of your open data initiative
In order to be successful, open data initiatives must be clearly aligned with larger strategic goals and objectives. Opening data just for the sake of “doing open data” is a recipe for frustration, confusion, and lack of long-term impact.
So, before getting started, it's key to define your goals for open data. Being clear about this from the beginning will make it easier to articulate the value and get other stakeholders on board. Ask yourself: Why is open data important for your government? What do you hope to accomplish? How can open data support the existing needs and priorities of your government and leadership?
Common goals and outcomes for open data
Depending on a combination of resources, priorities, and values, cities may have any number of goals for an open data initiative. Common goals include:
- Increase interoperability between systems and data-driven applications within City Hall and with other jurisdictions.
- Save staff time and create efficiencies by increasing information sharing between departments.
- Provide citizens information to better understand their government's activities and participate in improving the quality of life and promoting economic development.
- Provide greater awareness of — and availability of — data for data-driven decision-making throughout City Hall.
- Seed the environment for a local civic technology ecosystem.
- Capitalize on applications and services built on open data standards in other cities.
- Increase communication and demonstrate openness to build citizens’ trust in government.
- Reduce the burden on government staff to respond to repetitive public records requests by making frequently-requested information available to residents in a self-serve format.
Further reading: See this section for examples of how open data has achieved these goals in different communities.
Aligning it with organizational goals and priorities
Showing how open data can drive progress on high-priority issues can help generate initial buy-in from leadership, establish quick wins, and generate momentum to get a longer-term open data initiative off the ground. When crafting your rationale for open data, you should consider how open data can strategically support high-level policy priorities specific to your local context — such as creating jobs, reducing vacant and abandoned properties, or increasing government transparency. You might want to review recent high-profile speeches from your city’s leadership, such as an annual State of the City address, to identify key issues.
Be creative and push your thinking about how open data could have an impact in unexpected areas. For example, in Boston, Mayor Menino stated in his 2012 State of the City address that he wanted to the improve school selection process for families. Using open data, the city then built a simple web interface to help parents easily see what schools their child was eligible for and compare them on a range of dimensions like test scores, after-school programs offered, and travel time from their home.
What to expect: How open data has worked for cities of all sizes
There’s a way to do open data on any budget. Cities and counties with populations (and budgets) of all sizes have launched successful open data initiatives. The strategy and structure you choose to pursue should be informed by an understanding of your government’s resources, political environment, and priorities. Cost, demands on staff time, technology needs, and team structure can vary widely.
Other cities have paved the way and can serve as a model for what to expect as you launch your own open data initiative. Here are some examples of how other cities have done it, told by the government staff who made it happen.
Louisville, KY: Open Data
Tim Welsh, Deputy Director of Technology for Louisville Metro Government, describes open data in Louisville.
Chattanooga, TN: Open Data
Jenny Park and Tim Moreland, Open Government Specialists for the City of Chattanooga, answer some questions about their open data initiatve.
Montgomery County, MD: Open Data
Montgomery County was one of the first counties in the United States to enact an open data policy. Daniel Hoffman, Chief Innovation Officer, explains how they have implemented open data.
Pittsburgh, PA: Open Data
Pittburgh's open data initiative emerged from a strong Mayoral directive, and now the city is working to implement processes and practices across departments to make open data the norm. Laura Meixell, the Analytics and Strategy Manager for the City of Pittsburgh (and a former Code for America fellow), is helping lead the charge.
Albuquerque, New Mexico: Open Data
Rather than going the typical route of using an open data catalog provider like Socrata or even open source CKAN, Albuquerque has pursued a scrappy do-it-yourself strategy. Mark Leech, Application Development Manager has been leading the City’s open data initiative, ABQ Data.
Kansas City, MO: Open Data
Millie Crossland, KCMO's technology project liason, explains the three types of stakeholders they found when implementing open data: "Data Go-tos, Process Innovators, and Connectors."
Further reading: Beyond Transparency, edited by former Chicago Chief Data Officer Brett Goldstein with Code for America, is an anthology featuring stories and essays from over twenty experts and practitioners about open data's impact in cities. The book is available to read for free online. You can also purchase a hard copy.
Build departmental support and executive buy-in
A sustainable long-term open data initiative requires cooperation between multiple departmental stakeholders, and support from executive leadership to put resources towards open data. The next step to creating a successful open data initiative is to get buy-in within city hall. You’ll want to start by identifying a small group of supporters who can be the initial champions, and gradually work to build momentum and consensus more broadly. Every city’s internal structure and circumstances are different — there’s no “one size fits all” method to getting political support for open data — but here are some general guidelines to get you started.
Who needs to be at the table?
Executive leadership: A successful open data project often involves coordination and cooperation between multiple city departments; having support from City leadership can help ensure the project receives enough resources to be successful. Depending on your city, this might be the Mayor, City Manager, or another executive team member. The executive leadership advocate should be high enough up in the organization to help resolve differences between departments and agencies if they arise (and they probably will!).
Internal champion: In addition to executive leadership level sponsorship, it’s important to have a mid-level “boots on the ground” advocate who is invested in driving the open data initiative forward. This internal champion should serve as point person for questions, help define processes and coordinate stakeholders, and support the departmental representatives during the rollout process. Ideally, they will host weekly or bi-weekly “Open Data Office Hours” where folks from other departments can come to for help.
There’s no "ideal" job title for this person -- in other cities, it has been anyone from the Chief Technology Officer to a communications manager to Open Government Specialist. The important thing is their commitment to making sure the open data initiative succeeds. If you’re reading this guide, the internal champion might be you.
IT leader: Because an open data project can involve changes to the administration of existing data-based workflows, data storage repositories, etc., close cooperation of an expert on your municipality’s data infrastructure is essential.
GIS Specialist: A great deal of municipal data is place-based, tied to a specific geographic location. Having the assistance of a GIS specialist can ensure that data is provided to users in a useful format (e.g., interactive online maps are more helpful to the general public than downloads of raw GIS data files) and that geographic data is accurate.
Departmental Stakeholders: Even if your open data initiative is being driven by members of the IT department, getting support from multiple departments across City Hall who own the data you want to release is critical to success (more on prioritizing data for release later). A supportive departmental representative can help spur adoption of open data publication initiatives within their department and while also providing valuable subject matter expertise to the leaders of the city-wide open data project.
Further reading: The Socrata Open Data Field Guide includes additional tips about assembling an open data team.
Here's some tips for getting departmental stakeholders on board with open data:
- Explain direct benefits to the department (“we’ll reduce the number of citizen calls requesting this information”)
- Assuage fears (“the goal is to increase services provided while reducing costs, but without reducing headcount”)
- Point out opportunities for increased innovation and efficiency (“your rainfall report will be automatically sent to the Water Department and the Street Maintenance Department”)
One tactic is to prepare department-specific handouts for the director and leadership of each city department. The handout should specify examples of the types of data that department might publicize and included examples of applications and services built on similar data in other cities. Demonstrating examples of successful projects can help persuade decision-makers to get on board with the plan.
Demonstrating Value: Open data success stories
Here are some quick stories and proof points about how other governments have used open data to help you persuade key stakeholders of the value of open data.
San Francisco: Crime Spotting Map
San Francisco Crimespotting is an interactive map of crimes in San Francisco and a tool for understanding crime in cities, built using open data from the city. Crimespotting lets residents find out what's going on in their neighborhood with interactive maps and customizable RSS feeds of crimes in areas that they care about.
New York City: Targeting Illegal Building Conversions Inspections
In New York City, dangerous illegal building conversions are a big problem and public safety risk. The government opened and integrated data that had previously been siloed within by several departments and analyzed it to increase accurate inspection targeting and better allocate scarce resources where they will do the most good.
Asheville, North Carolina: Empowering Startups
CIO Jonathan Feldman started “Open Data Day” to bring together citizens journalists, business people, elected officials and others together making opening data a collective issue. Led in partnership with Venture Asheville, the event gave Asheville a chance to highlight business opportunities that arise from publishing open data. One local employer with a staff of 35 uses publicly-available data from municipalities, adds their analysis, and sells it to insurance companies.
Charlotte, NC: Helping Local Organizations Unlock Funding
The City of Charlotte publishes detailed neighborhood-level data about a number of quality of life indicators. Local non-profit organizations use the dashboard to find sources and statistics to support grant applications.
Chicago: Flu Shot Locations
During an OpenGov Hack Night event, Tom Kompare, a local hacker heard the Chicago Health Department talk about making it easier for people to find where to get flu shots. Kompare had a prototype site out within a week of the initial event. Another local organization was able to provide feedback on the app and help arrange a meeting with the City to discuss it. After support from the city to get legal and communications departments to sign off, the official app launched within a month.
Health and Human Services
Louisville: Restaurant Inspection Scores on Yelp
Louisville Metro Government published restaurant inspection data in an open, standardized format set by a cohort of other cities, third party organizations, and industry. Yelp was then able to consume the data easily and publish it alongside its popular online restaurant reviews, making the data more useful and accessible to the average citizen.
San Mateo County: Aggregating Community Services
San Mateo County worked to create an open database of all local community organizations and the services they offered, and then created a centralized, searchable site called SMC-Connect. This has made it easier for residents and case workers in need to find services they or their clients are eligible for in times of need by providing a single, comprehensive point of reference.
Internal Cost Savings and Efficiency
Albuquerque: Reducing Transit-Related 311 Calls
Opening data can reduce the number of phone calls for citizens to obtain information. People calling about Transit issues in Albuquerque were usually waiting for a bus and wanting to know where it was right then. Based on this analysis, the City decided to open up their transit data. Making these datasets available saved about $180,000 in 2013 in calls to the 311 call center alone.
Oakland: Streamlining Public Records Requests
Oakland launched RecordTrac, a user-friendly system to track and submit public requests that makes all past requests open and publicly available. The app displays every message or record uploaded before prompting users to make new submissions. It has helped the City of Oakland process over 2,000 public requests across all city departments in its first year of use — requests that won’t need to be processed again since the information is now easily available to residents.
Chicago: Eliminating 311 Redundancies
Chicago opened up its 311 data in a standard format, allowing it to easily integrate the existing 311 system with reporting tool SeeClickFix (a web and mobile app that lets residents report, track, and follow up on non-emergency issues). Doing so eliminated the need for city employees to redundantly, manually enter online requests into the City’s 311 system. With the Open311 integration, online requests are sent automatically and entered into the system.
Transparency and Accountability
Boston: Increasing Trust Between Government and Residents
In Boston, a Harvard Business School study showed that when the City released data to show citizens the status and progress on open 311 service requests, citizens expressed a more positive attitude toward government.
Opening and Publishing Data
Prioritizing data for release
Where should you start? We suggest starting small: focusing on a few key datasets for your initial release and ensuring that there is a sustainable publishing process in place, rather than doing a huge one-time data dump that will be hard to keep up to date. Former Philadelphia Chief Data Officer Mark Headd recommends starting with the "Three Bs": Buses (transit data), Bullets (crime data), and Bucks (budget and expenditure data).
As your open data initiative matures, you can (and should) continually reevaluate and add more data at a later point.
Eighteen recommended datasets
Code for America, in collaboration with the Sunlight Foundation and Open Knowledge Foundation, has identified eighteen of the most important datasets that we recommend every municipality publish. Of course, you might not be ready to release all of these right away, but it’s a good place to start. View the full list here.
Other ways to prioritize
Here are some other ways to identify high-priority datasets to release.
- Start with your goals: First of all, you should return to the goals you outlined for your open data initiative. Which datasets will be most conducive to achieving your goals?
- Look for low-hanging fruit: What data sets would be easiest to release? Perhaps some departments already publish certain data online (as PDFs, shapefiles, spreadsheets, or other formats) that would be easy to aggregate in the open data catalog. These can be low-barrier fodder to get your open data catalog up and running, and help build momentum to open up more data later.
- ...And quick wins: There are many free, open-source apps that are built to use open municipal data . By prioritizing the release of data sets that can be used by these existing apps, you can demonstrate the immediate tangible benefits of open data to citizens and city staff. For example, by publishing the location of storm drains or fire hydrants in your city, you can fairly easily stand up the Adopta application, making it simple for the average citizen to use and consume that data in a way that’s relevant to them. See this section for other examples of apps using municipal data.
- Look for existing demand: Examine city website traffic and public records requests to see what data is most often sought out by the public. By making the most-often requested datasets easily available in an open format, you can satisfy public demand and alleviate burdens on departments who frequently have to supply the same information over and over again.
- Ask the public what they want: The public, in particular your local civic hacker community, likely has ideas about what data would be the most valuable and useful to them. Invite citizens to suggest data sets for publication by conducting a survey or requesting input via social media or tools like Mindmixer or Textizen. This in no way obligates you to publish the requested data, but it can be useful information when deciding what data to prioritize.
- Take cues from other cities: Many US municipalities, large and small, have launched open data catalogs over the last few years. Take a look at their open data catalogs to see what the most viewed or most downloaded of their data sets are.
- Include data resources from parent jurisdictions (county, state, federal): Your local data platform might be a local citizen’s first entry into the open data world -- so include links to locally-relevant data sets available in other jurisdictions, such as the federal Census tract maps and data for your city. Check out data.gov and your state’s open data catalog for geographically-targeted data that your citizens might find useful.
Further reading: The Open Data Handbook includes other useful tips on how to choose datasets to include.
Select an open data platform
Before you can publish data, you’ll need somewhere to put it: an open data catalog or open data portal. This is essentially a website where the city can upload and update datasets, and the public can search and download them.
As with many software choices, your options can be boiled down to:
- Build your own custom data platform solution
- Buy an off-the-shelf data platform (or rent via an SaaS model)
- Borrow from existing open source offerings
Every city’s requirements are different, but there are two core features integral to any data platform: 1) ability for staff to upload data sets, and 2) ability for visitors to browse, search for, and download data sets.
In addition, other features are common in many platforms and can be considered high-priority:
- Integrated data manipulation (“transformation”) tools
- Integrated visualization tools
- API access for integration and interoperability
- Connections to other relevant open data sources (county, state, etc.)
Important criteria to consider when choosing an open data platform include:
- Feature satisfaction
- Total cost of ownership
- Sustainability of solution by IT (support, maintenance)
- Availability of support options (community, vendors, consultants)
There are two main models for open data platforms: software-as-a-service (SAAS) and self-hosted deployment. There are benefits and drawbacks to each. SaaS options include support and hosting from the vendor, which allows government staff to focus on publishing data, not maintaining the application. Self-hosted options allow for flexibility. There are no maintenance contracts or vendors to manage, but staff resources will be required to sustain the portal.
Comparing major platform options
CKAN -- the Comprehensive Knowledge Archive Network -- is open source software powering open data platforms across the world. Provided by the Open Knowledge Foundation in the UK, CKAN is used at the local, regional, national, and international levels of government as well as in academia. The CKAN application is written in the Python programming language.
CKAN is used for over two dozen official government data portals around the world. Data.gov, the United States Federal Government’s open data catalog, chose CKAN to power its platform. There are pre-deployed instances of CKAN that are available to use free of charge to governments and other data producers at CivicData.com (powered by Accela) and DataHub.io.
DKAN is a Drupal-based implementation of CKAN that offers an easier installation and support burden while preserving API compatibility.
- Catalog and repository and visualization platform
- Ideation and community features
- Stable and relatively large developer/support community
- Extensive documentation available
- Documented instructions for deployment to VirtualBox or Amazon EC2
- Proven installation base in nations and regions throughout Europe and hopefully soon US
- Relatively complex application environment (Python/pylons, nginx, elastic search, etc.)
- Limited existing US municipal or state success stories
- Requires ongoing support resources internally to maintain the software unless you use one of the pre-deployed instances mentioned above.
OpenDataCatalog (ODC) is open source software originally created by Azavea for the city of Philadelphia. The software is available for download on GitHub and a community of developers and implementers continue to work on improving the software. ODC includes options for data owners to submit data for consideration and for registered public users to nominate a type of data they would like to see openly available to the public. ODC is written using Django, Python and PostgreSQL.
- Multiple success stories in US, creating a community of support among implementers in different cities.
- Moderately complex application environment (Python base)
- Limited commercial support options
- Relatively immature feature set
- Requires styling out of the box
- No repository capabilities
- Requires ongoing support resources internally
OpenDataSoft is a new generation Commercially-off-the-Shelf (COTS) SaaS data platform that can be used for open data, closed internal data and Smart Cities programs. The OpenDataSoft back end is provisioned and maintained by OpenDataSoft, and the front end of OpenDataSoft is fully open source through HTML, CSS, AngularJS Widgets, some AngularJS and Bootstrap elements. Spend your time on full editorial control of the look and feel and presentation of the data to your citizens without the need for enterprise development of the back end.
OpenDataSoft has been adopted by the City and County of Durham, NC (https://opendurham.nc.gov), The Town of Cary, NC (https://data.townofcary.org) and the Town of Chapel Hill, NC (https://www.chapelhillopendata.org)
- Open Source front end.
- Allows you to develop data stories and performance dashboards easily with no additional fee.
- Data transformation processors, data harvesters and scheduled dataset updates remove most of the manual labor expense from your program.
- OpenDataSoft does not provide consulting services so implementation partners are needed if you need assistance in developing your open data program
- Limited commercial support options
Socrata is the most popular commercial data platform provider in the United States. Socrata offers a turnkey SaaS cloud-hosted data catalog, paid for on a subscription basis. The Socrata platform includes API abilities and sitewide analytics that track consumption and engagement metrics.
- Well-supported and relatively turnkey option
- Lots of experience with municipal government
- Add-on services like a performance management tool
- Depending on options, can be more costly
- A more complicated interface
Junar is another cloud-based SaaS option that includes RESTful API abilities and analytics. Pricing plans for organizations range start at $750 per month.
Junar is used by the City of Palo Alto.
- Tiered pricing plan starts below $5,000, making it under the procurement threshold for many cities.
- Easy-to-use interface, helps users easily collect and create datasets by pulling previously disorganized information from a computer
- White label branding
- Smaller capacity for API calls than Socrata
- Limited search capabilities
- Does not provide community participation features
ArcGIS Open Data is a hosted web application provided by ESRI that allows any organization using ArcGIS Online to easily open up data stored in that system. Data providers can easily enable open data, specify open data groups, and create and publicize their open data without hosting their own site.
- Many governments already have an existing relationship with ESRI.
- Easily integrates with existing systems.
- May be best suited for geographic data.
- The offering is currently in beta as of June 2014.
Further reading: 2014 Code for America Fellows compiled this summary which further compares features and costs for some of the most common open data platform providers.
Publish your data
After choosing what data you are going to publish and choosing where you're going to put it, there are a few steps to make it open. In order to be considered truly open, government data should be:
- Open by default
- Freely available in non-proprietary formats
- Openly licensed
- Downloadable in bulk
- Published in common formats, when available
- In digital form and machine readable (such as .csv)
- Placed in centralized, easily accessible location (such as open data portal)
- Accompanied by metadata that attests to the data’s structure, accuracy, assumptions, ownership, completeness, and currency
Further reading: The 8 Principles of Open Data lays out an in-depth definition of open data.
While that might seem like a long, complicated list of requirements, there’s just a few steps to get most of the way there. If you don’t check every one of those boxes, it’s okay! You are still making meaningful progress towards open data. The reality is that there’s always going to be room for improvement. Here’s how to get started once you’ve chosen a dataset to open.
First of all, do some due diligence to make sure you’ve covered your bases. Do a review of the data to account for its completeness, currency, and accuracy. If it is not perfect, again, that’s okay! It’s better to release some data than nothing at all, as long as it isn’t misleading or factually inaccurate; just make sure to clearly note any omissions or limitations in the metadata (see below) so data consumers understand what they’re looking at.
If you haven’t already, make sure that any key stakeholders or data owners are aware that the data is going to be released (not only will this keep you out of hot water, but they also might have valuable suggestions about how to structure or position the data). Double-check to make sure there are no restrictions or privacy concerns surrounding the release of the data.
Adding metadata, or descriptive information about the dataset, is another important step to ensure that users know what the information is, when it was last updated, and what it includes. The Philadelphia Open Data Guidebook has suggestions about basic metadata to include with any published dataset.
Next, choose and apply an “open license” that makes the data legally open and freely available for anyone to use and share. The Open Knowledge Foundation Open Data Handbook has some good guidelines on how to choose a license.
Now, you need to make the data technically open and publish it on your open data platform. The exact steps for this will depend on what format your data currently exists in, and what format you want it to be. Often it will involve an ETL, or “Extract, Transfer, Load” — a bit of code that gets data out of one system, converts it to the desired format, and puts it into another system. Here’s some useful information and resources about ETLs from former Code for America fellow Dave Guarino (note: this is targeted at technical folks).
Planning for Sustainability
Create an open data policy
Eventually, you'll want to make open data publication standard operating procedure via a city ordinance or executive order. This is a key step in becoming truly “open by default.”
As of June 2014, thirty cities and three counties in the United States have passed open data policies, and the number is growing rapidly. There are many existing examples that you can build off of as you begin drafting an open data policy.
Further reading: the Sunlight Foundation has created a living set of open data policy guidelines to address what data should be public, how to make data public, and how to implement policy, including sample policy language for each principle.
When is the right time for a policy?
In some cities, an open data policy is the first step towards open data. Passing the policy creates an executive mandate that drives momentum around implementing open data. In Los Angeles, for example, Mayor Garcetti issued an Executive Order shortly after entering office in 2013, directing all departments to collect data that they generate and prepare it for release in early 2014. This led to the launch of the city’s open data portal in early 2014. This can be a useful strategy if you have supportive executive leadership, but need leverage to generate buy-in from departments.
In other cities, the policy actually follows implementation of open data. Louisville Metro Government, for example, had a long-standing open data program and nationally acclaimed data-driven performance management initiative before finally making open data official policy in 2013. This can be a useful strategy if you have strong grassroots and departmental-level support and can “bootstrap” an open data initiative; after you have demonstrated value to leadership, passing a policy can be a finishing touch to establish formal, ongoing commitment.
Here are some examples of different kinds of open data policies that you could adapt for your city:
- City of South Bend Executive Order No. 2-2013
- City of Louisville Executive Order No. 1, Series 2013
- City of Austin Resolution No. 20111208-074
Further reading: For more examples, the Sunlight Foundation maintains a comprehensive list.
Appoint staff to be responsible for data management
For open data to “stick,” there needs to be staff responsible and accountable for implementation. This doesn’t necessarily mean hiring additional staff; responsibility can be delegated to existing employees. In some cities, the internal open data “champion” continues to drive and organize open data throughout the city, even if that’s not necessarily one of their official core job functions, often leading a cross-departmental open data working group. Other cities create a dedicated position to oversee open data implementation, often called a Chief Data Officer. Others require that each department designate a staff member to be their point-person for open data.
Chief Data Officer
The primary responsibility of a Chief Data Officer is to operationalize open data across the government. The White House defines the role as “part data strategist and adviser, part steward for improving data quality, part evangelist for data sharing, part technologist, and part developer of new data products.”
Not all governments establish a formal Chief Data Officer position, and there are many successful open data initiatives operating without one. However, if it’s possible, appointing a senior-level official to oversee the implementation and execution can go a long way towards making open data useful, sustainable, and actionable.
Here are a few job descriptions for government Chief Data Officers:
- Sample CDO job description from the White House’s Project Open Data
- Philadelphia Chief Data Officer job description
- San Francisco Chief Data Officer job description
An Open Data Coordinator (ODC) is a government employee designated by a department or agency to coordinate open data efforts for that department. Many open data policies mandate that an ODC be designated in each department and participate in an ongoing, city-wide open data working group chaired by the open data “champion” (whether that is a Chief Data Officer or somebody else).
Responsibilities might include identifying and publishing of data sets to open data portal, addressing feedback from the public on the department’s public data, and reporting on progress to the broader working group. Some departments have had success in giving this responsibility to the person who is already managing public records requests for the department, as these roles are closely related.
Making open data useful
Use common open data formats
We've seen large cities with large budgets do amazing things with in-house technology and predictive analytics teams but making your data useful can be as easy as opening data in common formats.
Code for America's recommended formats make open data easier to access, use, and move to the applications people already use to make decisions. Aligning with these common standards can be as simple as reformatting data before publishing it.
When cities across the country format their data in the same way, technologists can build tools that scale across cities at a fraction of the effort and cost. Open data formats streamline the process of finding and using data across city agencies. That means that civic apps can come from technologists with talent and drive, not just big companies with cash on hand.
A number of high profile tech companies with broad reach want to help government do more with open data:
- General Transit Feed Specification (GTFS): Google thinks data on mass transit helps people stay on time and get where they want to go.
- Housefacts Specification: Trulia and Zilllow want to share your residential building inspection data with the 90 million people who visit their site per month because they also want to help people make good choices about their next home.
- Local Inspector Value Entry Specification (LIVES): Yelp is eager to share the results of restaurant inspections because they want to help people find restaurants that aren't going to get them sick.
Further reading: Learn more about open data standards that Code for America supports.
Hold a hackathon
Civic hackathons serve as a great community engagement tool, bringing together technologists and civic leaders to brainstorm and develop applications using open data. These coding marathons allow people in the civic space and citizens to work with developers to creatively use civic data to address community issues. The success of hackathons, like National Day of Civic Hacking, is contingent on several factors, one being the availability of open data. The applications developed at hackathons are often great outcomes, but hackathons are best approached as a means to increase resident participation and provide a platform for community collaboration.
Successful hackathons tap into both the civic and local developer communities. So, marketing them is of the utmost importance. Socrata offers a comprehensive guide to running, marketing, and managing a hackathon, which suggests contacting local developers, posting in high-traffic forums, and posting fliers at local universities in computer science departments to generate interest. Reaching out to groups like Code for America and Socrata can be key as well, since each group is tied to a large network of technologists in the civic space.
Support from the local government is key in citizen engagement. Announcing the hackathon at council meetings and getting members within the city government to encourage participation among their constituents is a good way to create a buzz outside of the developer community. Excitement about a hackathon within the local government can serve as a hub of interest for the rest of the community.
Further reading: Socrata has a step-by-step guide to how to plan a hackathon using open data.
Deploy apps that use open data
Citizen-facing apps are a great way to make open data useful for non-technologists. By providing a user-friendly interface to the data through a simple web or mobile app, you get city information directly into the hands of residents. Other cities and civic hackers have built many free, open source civic apps that are available for governments to redeploy for their own communities. (“Open source” means that the software code is freely available for others to use without restrictions, much like open data.)
Further reading: Open source expert Ben Balter has put together a guide for government employees who want to learn more about open source and participate in the open source community.
It’s important to note that just because the code is free doesn’t mean there won’t be costs associated with redeployment. Technical resources are needed to customize, launch, and maintain the app. Still, open source civic apps can be useful quick wins to demonstrate the value and use cases of open data. Here are a few recommended open source apps that use open data which you can customize and redeploy for your community.
Further reading: More apps using open civic data (both free and paid) can be found on the Code for America Products page.
This guide is in beta and we'll be continuing to develop it. Help us improve this resource by submitting an issue or pull request on GitHub, or email us with comments and suggestions: gov-staff [at] codeforamerica [dot] org.