Beth Noveck: think about how to open up the API of government

Former White House deputy CTO for open government Beth Noveck recently gave a TED Talk on a “more open source government. For more perspective in the vein of “open,” read CNN’s TED summary, “What if you could make anything you wanted?” Noveck’s talk is embedded below:

“…start by teaching young people that we live, not in a passive society, a read-only society, but in a writable society, where we have the power to change our communities, to change our institutions, that’s when we begin to really put ourselves on the pathway towards this open government innovation”

Are “Commons 2.0” and participatory urbanism hype or hope?

“…armed with low-cost phones and an Internet connection, people are using civic-minded apps like ForageCity to tackle everything from public safety to potholes. The question is whether the technology will have the long-term effect that some foresee, or whether the “commons 2.0” and “participatory urbanism” will become empty marketing slogans.”

-Angela Woodall, writing in the Oakland Tribune about a new mobile application from Oakland’s Youth Radio that is designed to help people redistribute extra fruit and vegetables to people in need.

Forage City app
[Image Credit: Susan Mernit]

Woodal asks good questions and, as it happens, posed them to me last week in a phone interview. (I’m quoted in the article.)

Here’s a couple of thoughts that didn’t make it in. Mobile applications that civic developers are creating around the world — like ForageCity — are making it increasingly possible for more people to interact more easily and for less cost where ever and whenever they wish. That does lead to giving more power to more people to connect to one another and solve problems, or at least discuss them.

The potential for such apps to connect and, crucially, scale is particularly significant when there is a shared standard for the open government data that fuels, as with the standard for transit data (GTFS) that now exists in 450 different cities. Around the U.S., cities are slowly working with one another to define more such standards — but it’s a complicated process that doesn’t happen overnight, or even years.

The question is whether the technology will have the long-term effect that Code for America founder Jen Pahlka described to Woodall. On that count, I tend to give Pahlka — and my publisher, Tim O’Reilly — the benefit of the doubt.

As I said to the reporter, the potential for civic apps is enormous — but these the tools are only as good as the people who use them and adapt them. The tools can be quite good on their own — full stop — but many network effects will only take place with broad, mainstream adoption.

Smartphones can now be used for finding shelter, improving medical care and documenting riots — but the same devices are also used for gaming, pornography, celebrity gossip and shopping. While the apps used to find city services are generally not the ones used to surveil citizens, in practice the mobile device itself may be an agent of both actions.

Working out how to both protect the rights of citizens and empower citizens using mobile devices will be a difficult and crucial need in the years ahead.

It’s not immediately clear, at least to this observer, that state governments, Congress, regulators and law enforcement are up to the challenge, but it’s hard not to hope that they rise to the challenge.

Jay Nath on how San Franscisco is working to get its Gov 2.0 groove back

Back in January, Govfresh founder wrote about how San Francisco can “get its Gov 2.0 groove back,” offering six recommendations to the city government to use technology better.

[Image Credit: Fog City Journal]

When asked for comment, San Francisco chief innovation officer Jay Nath (@Jay_Nath) responded to Fretwell’s suggestions via email. While I’ll be sharing more from Nath and SF CIO Jon Walton over at the O’Reilly Radar civic innovation channel, in the meantime I’m publishing his specific responses to those recommendations below.

Build the best mayoral website in the world

Nath: We can always improve how we communicate with our constituents. If we were to undertake an effort to redesign the Mayor’s site, we should take a holistic approach and not just focus on the Mayor’s site. The approach NYC took to invite their design community is one that I think is very smart and something that SF should consider.

Use “Built in SF” technology

Nath: We agree and launched our City Hall iZone concept where we pilot great local technologies and services. We frequently meet with great companies like Square, Twitter, Uber, Yammer and invite each of them to work with the City. Specifically, we’re actively exploring Yammer, Zendesk, Get Satisfaction, Cozybit and 802.11s mesh, Google+ hangouts, and others. Additionally, we’re already using local tech like WordPress (which powers our innovation site), Twitter via Open311API, and Instagram.

Go back to the (data) fundamentals

Nath: We have an open data roadmap to strengthen our leadership in this area. It’s in our 2012 innovation portfolio as well. Our goal is to structurally change how we share data so that our default position is one of sharing. One idea is to require that all software purchased that stores structured data to have a public API. As we secure staffing for this effort, we will invite the community to help us shape the final form and execute.

Leverage the civic surplus

Nath: I would argue that we’ve done a great job in this area. Last summer, we partnered with Gray Area Foundation for the Arts (GAFFTA) to produce the “Summer of Smart.” This series of hackathons produced over 20 prototypes, 500 participants and 10,000 hours of civic engagement. We’ve continued our efforts this year with the City’s first unhackathon around taxi dispatch and real-time mass communication. Our Mayor and transit director both attended the event and thanked our community for their efforts to make SF a better city.

Additionally, we launched our citizen engagement platform, ImproveSF, in a very big way in April.

Open source the infrastructure

Nath: While we can do more to increase open source software adoption, I want to
recognize our efforts to date:

  • open source policy
  • SFPark Android/iPhone app
  • Enterprise Addressing System
  • SmartPDF
  • LAMP as an option for internal customers
  • Pligg (DataSF)
  • Several Drupal applications

Additionally, the idea of moving our City from the existing CMS (Vision) to WordPress is not just about open source technology. We, as a City, made the decision to utilize Vision CMS a couple of years ago and the switching costs to migrate to WordPress currently outweigh the benefits. I will encourage the City to strongly consider WordPress, Drupal, etc for consideration when Vision no longer meets our needs.

Give citizens a dashboard

Nath: This is more than just adopting the IT Dashboard. We have to implement the governance and project management model to ensure that the data is accurate. This is something we need to do but requires time and culture change. I agree that we need to increase access to high value datasets like expenditures. This is part of our open data roadmap and will receive renewed focus in 2012.

SahelResponse.org showcases the power of open data and neocartography

On January 9th, I wondered whether 2012 would be “the year of the open map.” I started reporting on digital maps made with powerful new software and open data last winter, in the context of open government.

In the months since, I’ve seen many more maps emerge from the work of data journalists and government, including a beautiful one made with TileMill and open data from aid agencies at SahelResponse.org. You can explore the map in the embed below:

Nate Smith, who works at DevelopmentSeed, the makers of MapBox and TileMill, blogged about SahelReponse.org at PBS Mediashift.

To bring key aid agencies together and help drive international response, the SahelResponse.org data-sharing initiative maps information about the ongoing food crisis in the Sahel region of West Africa. More than 18 million people across the Sahel are at risk and in need of food assistance in the coming months, according to the United Nations. Recent drought, population movements, and conflict have created a rapidly changing emergency situation. As in any crisis, multiple agencies need to respond and ramp up their coordination, and access to data is critical for effective collaboration. In a large region like the Sahel, the band of mostly arid land below the Sahara Desert stretching across the continent, effective coordination and collaboration are critical for responding effectively.

Thanks to new technologies like TileMill, and an increased adoption of open data, it was possible to put all the key data about the crisis — from relief access routes to drought conditions and population movements — in one place, openly available and mapped to give it further context.

More than half a year later, on other words, I think the prediction that 2012 will be the year of the open map is being born out. The adoption of OpenStreetMap by Foursquare was a notable data point, as was StreetEast moving to OpenStreetMap from Google Maps. In response to the challenge, Google slashed its price for using the Google Maps API by 88%. In an ideal world, the new competition will result in better products and more informed citizens.

The City of Quebec launched an open data site

Up in the currently not-so-frozen north, the City of Quebec has stood up an open data directory online. There are currently 26 datasets listed, spanning a variety of data formats, from .CSV to .XML to .XLS to to .KML to .SHP. (The latter two are GIS files, of interest to folks who like to make maps.)

The city published the video embedded below last night, in addition to a “demarche” (or statement) on the open data website about the project.

ExpoTI-GVQ – Projets étudiants, CÉGEP Limoilou from E-Gouv Québec on Vimeo.

Hat tip @Data_BC

UPDATE: As Richard Ackerman pointed out on Twitter, this open data site went live in February. While the video is new, the site is not.

MIT Civic Media conference examines the success and failures of open government in the U.S.

The 2012 Civic Media Conference featured two full days of conversations about (what else?) the future of civic media and democracy. One conversation is particularly worth calling out and sharing with the Govfresh audience: a panel assessing what’s gone wrong and what’s gone right with open government in the United States over the past three years. The discussion was moderated by Susan Crawford, currently of the Harvard Law School and Kennedy School (and formerly a special advisor at the White House) and featured Mike Norman of Wefunder.com, Mark Headd of Code for America and Chris Vein, Deputy United States Chief Technology Officer for Government Innovation in the White House Office of Science and Technology Policy. I’ve embedded the video below:

Watch live streaming video from knightfoundation at livestream.com

You can read an excellent, comprehensive liveblog of the open gov panel at the Civic Media blog.

What is smart government?

Last month, I traveled to Moldova to speak at a “smart society” summit hosted by the Moldovan national e-government center and the World Bank. I talked about what I’ve been seeing and reporting on around the world and some broad principles for “smart government.” It was one of the first keynote talks I’ve ever given and, from what I gather, it went well: the Moldovan government asked me to give a reprise to their cabinet and prime minister the next day.

I’ve embedded the entirety of the morning session above, including my talk (which is about half an hour long). I was preceded by professor Beth Noveck, the former deputy CTO for open government at The White House. If you watch the entire program, you’ll hear from:

  • Victor Bodiu, General Secretary, Government of the Republic of Moldova, National Coordinator, Governance e-Transformation Agenda
  • Dona Scola, Deputy Minister, Ministry of Information Technology and Communication
  • Andrew Stott, UK Transparency Board, former UK Government Director for Transparency and Digital Engagement
  • Victor Bodiu, General Secretary, Government of the Republic of Moldova
  • Arcadie Barbarosie, Executive Director, Institute of Public Policy, Moldova

Without planning on it, I managed to deliver a one-liner that morning that’s worth rephrasing and reiterating here: Smart government should not just serve citizens with smartphones.

I look forward to your thoughts and comments, for those of you who make it through the whole keynote.

The Empire (State) Strikes Back (Against Corruption)

This week, New York Attorney General Eric T. Schneiderman launched NYOpenGovernment.com, a new website that his office touts as a means for “voters, the media and government watchdogs hold state government accountable” by providing the public online access to government data on campaign contributions, lobbying, and state contracts.

“Secrecy breeds corruption, while transparency generates confidence,” Attorney General Schneiderman said, in a prepared statement. “New York Open Government will help the public keep an eye on what their government is doing in order to deter corruption and increase confidence in the public sector. This site is a one-stop-shop for New Yorkers demanding up-to-date and comprehensive information about their government.”

The launch of the new site fulfills a commitment that Schneiderman made as a candidate for Attorney General. NYOpenGovernment.com is an expansion of Project Sunlight, which went online in 2007 under then NY Attorney General Andrew Cuomo.

The citizens of New York could use a boost to confidence about their state government. According to a release from the NY AG’s office, at least 20 current or former elected members of the legislative and executive branches of the New York State government were either accused or convicted of crimes per the last decade.

“It’s hard not to be enthusiastic about this launch,” said Laurenellen McCann, national policy manager for the Sunlight Foundation, when asked for comment. “NYOpenGovernment.com demonstrates a genuine commitment to public oversight that more states should seek to emulate. Without the online release of information about campaign contributions, lobbying, state contracts, and other “influence data”, no government can really claim to be fulfilling its promise to be open or to provide open data.”

The data on the new website is sourced directly from the relevant state agencies. Campaign finance data come from the Board of Elections, lobbyist data from the Joint Commission on Public Ethics, state contract data from the Comptroller’s office, state corporation data from the Department of State, and bill information from the legislature. According to the attorney general’s office, the AG receives raw data, in different formats, from the agencies when they update their own respective websites.

“What’s worth noting about New York’s new platform is that it not only releases this important accountability data, it also provides contextualization for it, allowing citizens to access the info through centralized searches,” said McCann. In fact, this is the primary approach behind sites like Ethics.gov, or Sunlight’s Influence Explorer.com and one that we consider a best practice.”

It’s also worth noting that the site’s Web design is clean, uncluttered and loads quickly on a mobile device, if not in a mobile-optimized version.

If media and citizens have requests for data or questions about quality or accuracy, the AG’s office established a primary point of contact: Jason Ortiz, the director of special projects, and provided an official phone number (212-416-8743) and email address: Jason.Ortiz@ag.ny.gov.

The introduction of site was parsed by numerous members of New York’s good government community:

“With New York Open Government, Attorney General Schneiderman is showing clear leadership in making to government more transparent and accountable,” said Andrew Rasiej, founder of Personal Democracy Media and Chairman of the NY Tech Meetup, in a prepared statement. “By updating New York Open Government online tools and features A.G. Schneiderman is demonstrating that in the 21st century, the public’s access to information regarding how their government officials act must be easily searchable and accessible online.”

“We’re excited by Attorney General Schneiderman’s New York Open Government website,” said John Kaehny, Executive Director of Reinvent Albany, in a prepared statement. “We applaud the attorney general’s efforts to harness the immense power of the internet to increase government transparency and accountability. We look forward to working with A.G. Schneiderman to help New York Open Government achieve its full potential as a potent tool for restoring trust and confidence in our state government.”

“New York Open Government is an important resource for New Yorkers who want to know how their government works,” said Susan Lerner, Executive Director, Common Cause/NY, in a prepared statement. “We applaud Attorney General Schneiderman for helping to bring New York’s information services into the 21st century, and significantly improving access to publicly available data. Government transparency is essential to an engaged electorate.”

“In the information age, New Yorkers want and expect access to the hard data that shows what their government is up to,” said Russ Haven, legislative counsel for NYPIRG. “Attorney General Schneiderman’s New York Open Government website provides a ‘one-stop shopping’ place for average New Yorkers as well as sophisticated researchers to find information about elected officials and those seeking to influence them. The features will make it easier to access, organize and ultimately make sense of information as never before. This is an important resource for New Yorkers trying to keep tabs on government.”

What’s next?

There’s an additional bright spot here, with respect to cost to taxpayers: no expensive contract to a systems integrator was involved. The office of the Attorney General built the site in house, with no consultants. According to the NY attorney general’s office, hat they’re committed to sustaining and enhancing the site, including adding more datasets, improved search and a trackers for the most viewed data.

If, in the future, it may be possible for citizens to share information about government programs, practices or officials into the their social networks, it will a step ahead for networked accountability. “We’ve seen the power of social media for democracy movements around the world,” said NY Attorney General Schneiderman, in a prepared statement. “By making this tool compatible with multiple media platforms, we hope to empower our own citizens to hold their government accountable.”

The AG’s office looks at the website like an example of “living, breathing and evolving public accountability,” and emphasized that they will listen to its users and implement their suggestions “when it makes sense” to do so.

Here’s one suggestion, from this native of upstate New York: set up a data.nyopengovernment.com so that citizens, media, developers, advocates and state employees can see, browse and download the data in bulk. Currently, a user can search for an individual and then view all the relevant records, as for Mario Cuomo, with the capacity to download the data as a .CSV, Excel file or XML.

While New York should and is being lauded for this step forward to make open government data available in open, structured form only, its public officers could help to enable an ecosystem of networked accountability through enabling the creation of Web services, not just new websites. The next evolution in open government is not to encourage citizens to visit a website but to release the data that site is built upon so that it finds them, when they use search engines, social networks, media websites or civic applications like OpenStates.

Citizen Audit: Which federal agencies have published open government plans 2.0 online?

If you look at the White House open government dashboard, you might be forgiven for thinking that all agencies had complied with the mandate to post plans, publish “high value” datasets to Data.gov and implemented them. It’s all “green” and “yellow.”

The reality is more complex. Measuring the outcomes of open government initiatives effectively is a task that will require the attention of watchdogs, inspector generals, economists and academics. (See the work of the Sunlight Foundation on building ClearSpending.org to evaluate the quality of federal spending data on USASpending.gov, for example.)

What I could do was to simply assess whether agencies have updated their 2010 plans and published them online. When I visited the websites of 29 federal agencies, I found that less than half of them (13) had posted a “2.0” version of their plans.) 9 had not updated them since 2010. Based upon that audit, there should be a lot more red on the White House’s /open dashboard.

UPDATE: As of May 7th, several agencies had updated and published their plans online. (I’ve updated the spreadsheet below.) The remaining laggards: Defense, Labor, OMB, USTR, Veterans Affairs, National Archives, NSF, NRC, ONDCP and White House OSTP.

UPDATE: As of September 7th, Labor and OMB, ONDCP still had not updated their plans. Amy Bennett, director of Open The Government, brought the lag to the public’s attention. More context follows after the dashboard.

This issue came up this past week at the “Open Government Reality Check” at Information Week’s Government IT Forum. Given the prompt, perhaps the administration will put more pressure on agencies to update, publish and share their plans, including sharing them with the public on social media. (NASA, Energy and USDA have led the way on the latter count.) Applying pressure from OMB to implement against them is another matter, though some agencies continue to move forward. The Department of Transporation, for instance, recently launched Safety.Data.gov.

White House officials might also look to updating their own plans: the open government plan from the White House Office of Science and Technology Policy doesn’t appear to have been updated since April 7, 2010.

If and when the White House does update this dashboard, additional metrics might be added for each agency: “Number of Freedom of Information Act (FOIA) requests made,” “FOIA requests fulfilled” and “average time to response.”

A story this week on airline passenger complaints by Michael Grabell at ProPublica, for instance, showed that a FOIA request to TSA took four years to fulfill.

Since this data is, in theory, being posted at FOIA.gov, integrating and displaying these metrics would offer both a broader and more granular insight into the administration’s open government performance. Changes in culture will naturally be harder to measure.

UPDATE: