Data.Gov.Uk

Tim Berners-Lee launches UK public data website Web pioneer Sir Tim Berners-Lee has launched Data.gov.uk, giving access to a wealth of public data, and the Guardian launched a website to make finding that data easier Kevin Anderson Thu 21 Jan 2010 09.56 ESTFirst published on Thu 21 Jan 2010 09.56 EST Shares 2 Comments 3 Why the launch is important Sir Tim Berners-Lee, the inventor of the world wide web, launched data.gov.uk, a new British government website offering free access to a huge amount of public-sector data for private or commercial reuse. While taking a page from the US with the launch of its own public data hub, the UK data portal offers almost three times the number of data sets available on the US site. Berners-Lee laid out the thinking behind the site in a "manifesto" published on the Guardian: Data underpins our economy and our society - data about how much is being spent and where, data about how schools, hospitals and police are performing, data about where things are and data about the weather. Yet until recently not many non-technical people concerned themselves with data and how it could be used better. Data.gov.uk had a beta launch for developers last October, and Berners-Lee said that the official launch was a beginning, not an end goal. we hope it never is: there will always be scope to add more data as it is collected and to give more help to those using it to make economically or socially valuable applications. MP Stephen Timms, minister for Digital Britain, kicked off the launch. Data.gov.uk launched with more than 2,500 data sets, and he gave a sense of the range of data available: Whether a company exists or is solvent, we have data on that topic. If you need to know whether the house that you are about to buy is prone to flooding, we have data on that. I am told that we have how many fish there are in the English Channel. As the Prime Minister Brown pointed out in a liberty speech in 2007, data does not belong to government but public, Tims said. The developer preview of data.gov.uk launched and worked with 2000 developers and to find out how it could be more useful. Timms praised the leadership of Berners-Lee and Shadbolt, and he also praised the involvement of the developer community. The features reflects feedback from the 2000 developers involved in the preview, and the site continues to have features so that developers can continue to contribute their ideas. The site would help showcase British developer talent. He reiterated pre-launch comments from Berners-Lee and Shadbolt who said that this was not the end of the process. He said: We have agreed to an open licence for free reuse of the data. We have agreed to release data, weather, transport and public finance. We have started public consultation to open up data at Ordnance Survey. Timms said that this was part of strengthening the relationship between citizens and civic society, and he said that data.gov.uk would help the government become more efficient as public finances were under pressure. In addition to showcasing British developer talent, he said the project would: • Open up government • Improve public services • Increase economic growth and social value Unexpected green light Societies create lots of data, Berners-Lee said. Scientific data. Weather data. Social graph. One important bit of data is government data, collected by governments and paid for by tax payers. It's mostly under-utilised. Prime Minister Gordon Brown asked Berners-Lee how the UK should make best use of the internet. Berners-Lee said that the UK government should put all of its data online. Much to his surprise, the prime minister said: "Yeah, let's do it." I don't normally get such a direct response, Berners-Lee said. 3.30pm: Berners-Lee said: "Why data?" If you put data looking after something in Britain. they can compare things over time. They can run statistical analysis and see trends. People can put things on a map. 3.35pm: Berners-Lee said: "The people who generated this data should get credit, kudos and respect. We must track whose hard work it has been that has led to this." 3:35pm: Nigel Shadbolt then gave more detail about the project. Highlighting the use of data to by John Snow discover the source of cholera outbreak in 19th Century London, he said, "We history of improving public services by getting the facts out there." 3:37pm: Shadbolt said that the project did not just engage the developer community but pushed and prodded by them. 3:39pm: They also created URIs, uniform resource identifiers, across government so that data was more easily shared, Shadolt said. 3:40pm: Data.gov.uk was built using open-source technology by 10 people over six months. During the launch, the UK Director of Digital Engagement tweeted: #datagovuk built with #linux, #drupal, #mediawiki and other good stuff. (UK Gov OSS policy at http://bit.ly/639bw0) 3:41pm: The site also has a place to highlight some of the latest applications built using the data. 3:41pm: Shadbolt next talked about the licencing terms of the data. He stressed what a watershed the launch was in terms of licencing. The data might have been available before, but it was available as PDFs, which didn't allow easy reuse of the data, or was available under restrictive licences. I don't think that we should underestimate new Crown Copyright for Data consistent with Creative commons. 3:43pm: The government is still engaged in the public consultation on how to licence geographical data from the Ordnance Survey, data which Shadbolt described as the "crown jewel" of UK government data. He said: We have to get that data to the greatest number of people with the least amount of friction. 3:45pm: The site has already got rave reviews from sites such as Web 2.0 news site, ReadWriteWeb, which highlighted the fact that data.gov.uk had almost three times more data than data.gov in the US. Shadbolt said that he hoped competition amongst governments to release their data might set off "a race to the top". 3:47pm: In the hands of talented programmers, Shadbolt said that they could develop interesting applications with the data in just a few days. Developers with access to the preview access to the site have already built applications using the data, and Simon Rogers, editor of the Guardian's Datablog, highlighted them earlier today. 4.15pm: To follow up on Simon's post highlighting some of the applications, here are a few of my favourites from the presentations: • An easy way to compare house prices across the UK • A newspaper with information based on a postcode • And the Billion-Pound-o-Gram, helping us understand some of the huge numbers we run across daily especially in terms of government finance. 4.25pm: Here are the Guardian, we launched a government data gateway, bringing together not only data from the new site here in the UK, but also making it easy to find data from New Zealand, Australia and the United States. 4.28pm: The event ended with a panel session called the Power of Apps. After the iPhone, everyone seems to be obsessed with apps, but this was about data-driven apps. Mike Bracken, the Guardian's technology director, gave this slightly humorous introduction of Berners-Lee: He's an Oxford graduate, who went on to create the world wide web Not too many people have that on their CV. 4.31pm: Bracken started off by asking, "Where do we go now?" "This is very much the beginning. Hopefully, this is the tip of the iceberg," Berners-Lee said. "There is a whole lot more to do." First you do the easy bits, but after they finish those, they will probably run into government departments that are more resistant to releasing the data. "The most important thing is to get this used," he said. 4.33pm: What next? Shadbolt said that we have to maintain momentum. Government has an "imprecise" understanding of how much non-personal data it has, which is a nice way of saying that it doesn't have records of all of the data that it has. 4.34pm: Shadbolt said a big challenge would be finding "the join points" where the data sets are connected, and they want to focus on local government data as well. "Now that the baby has been born, we don't want to let it die of neglect," he said. There are some interesting lessons that they learned in building the project. 4.35pm: Is structural change still needed in central government, Bracken asked. Richard Allan, the director of public policy at Facebook, said that the government needed the confidence to release data. Much of their experience with data was when CDs or laptops full of personal data go missing. When it comes to local government, we would challenge a "large local authority which shall remain nameless" on their copyright statement, they said that a junior web officer thought they needed one so they cut and pasted it from another website, Allan said. 4.38pm: There was a big focus on local data on the panel. Chris Taggart, of OpenlyLocal, said that our biggest connection was to our local community. People were interested where the money went in their area. People were concerned about why their roads hadn't been fixed and who was the biggest developer in their area. 4.40pm: What new services do you expect to see? Bracken asked. What new enterprises will start? Shadbolt said that was a tough call with many answers. "Where will it get monetised? What will be the business model?" "It's a very embryonic landscape," he said, but past experience with services such as geo-spatial say that there is a lot of money to be made. 4.42pm: Bracken put the same question to Allan. Right now, Allan said that most of the applications that are making money are social, and he pointed to the recent acquisition of social games maker Playfish, which sold to games giant EA for hundreds of millions of dollars. However, government data apps might have a much smaller market, but he still saw opportunities. 4.44pm: What about privacy? "We define our privacy as a common and reasonable expectation of privacy," Shadbolt said. You can't as a society say as information cannot be used in these ways. Social conventions will evolve. It wasn't a statement that privacy doesn't matter, but rather that it was an evolving, ongoing debate fed in part by changing norms. 4.46pm: Taggart said that it was too early to talk about business models. Things are moving too fast. Open data is an enabling technology, and we don't know what will be built on this technology. With the launch of OpenlyLocal, he wanted to "correct the asymmetries in access to data". Much of this data was available previously, but it was only available at a cost. Opening up data more publicly allows a wider range of people to have access to that information, not just people with more power and more open. He didn't know what would happen in terms of business, but hopefully, he said, open data would make the world a bit fairer. 4.49pm: There was a question from a ZDNet journalist about privacy and licencing. Allan said that, in terms of privacy, there is a difference between something you do and something that is done to you. The government is tied in to only releasing data that is non-personal. However, there is another side. People are sharing personal data because they perceive value in doing so. That's different than government doing things on data that it holds. It leads to caution (possibly over-caution, was the implication) on the government side in terms of releasing data. 4.51pm: 4.51pm: Shadbolt said that in terms of privacy, this is really about what we decide as a society is acceptable behaviour. This isn't about the technology but rather about a public debate. In terms of licencing, they worked with alternative copyright organisation, Creative Commons, to develop an open licence. Before this, there wasn't a UK-wide Creative Commons licence but rather one for England, Scotland and Wales, and they had to work with Creative Commons to develop a UK-wide licence. 4.53pm: There was a question about mobile being the platform of the future and whether this would cause problems in terms of building apps because of fragmentation in the mobile market. Berners-Lee said that web standards existed and were being ported to mobile phones. Things have changed greatly in the past few years in terms of the web experience that people expect. In the bad old (well, sadly recent) days, mobile web browsers didn't follow standards, making it difficult to easily port desktop apps to mobile phones. That was changing. 4.56pm: How time consuming will it be to create linked data? Berners-Lee said: That's a really interesting question that takes us back to the creation of language and the Tower of Babel. 4.57pm: Berners-Lee said there are some elements of language such as degrees Celsius, but there are other things like data, where you pull up a table and are recognised internationally and some only recognised locally. Using potholes as an example, which lack an international data standard unlike temperature, it might become valuable enough for people who want their potholes fixed to create a data standard if it helped them get fixed faster. The point being that people standardise when there is an incentive to do so. 5.00pm: Someone from City University said that he had poster on his wall saying: "It's the postcodes stupid!" On one day, Gordon Brown says that we'll release the postcodes, but the next day, someone from the Royal Mail, says that no one will release the postcode data. Shadbolt said that one problem were issues with the postcode system itself. There are also issues of franchising in terms of postcodes. The Guardian building itself is not findable by its postcode using mapping software, Allan said. (The building is a year old, but common online maps or SatNav systems don't have the postcode for Kings Place in their database.) 5.03pm: Greg Hadfield, of the Telegraph until last week now of CogApps, asked how do we improve data literacy because right now people rely on politicians, civil servants and others to interpret. Taggart said with data you can badly analyse it or twist it, but with open data, at least you have the opportunity to interrogate yourself and say that you got it wrong. 5.06pm: In terms of data literacy, Berners-Lee said that the UK had a high degree of map literacy because Ordnance Survey maps are taught as part of the curriculum. Likewise, it may be that countries with more data will become more data literate. 5.08pm: Shadbolt said, "Tim and I have heard a lot of excuses for not releasing data in the past few months such as: 'But, they will be able to interpret it in different ways'". There were lots of noises of approval from the room. 5.09pm: Tony Hirst of Open University (and the author of the wonderful OUseful blog on data and data mashup tools) asked what education were needed in terms of local government. Allan admitted that this was not at the top of the agenda for local government, especially with shrinking budgets. Standard processes needed to be adopted. "The more you make it 'noddy' the easier it will be," Allan said. Berners-Lee said the next phase is very much about spreading the word. He suggested workshops where people would share what software they used and how they built their apps. "A community can help teach itself," he said. There is a wiki for each database where people can ask questions about projects they want to do. 5.12pm: There was a question about the possible release of time-sensitive data. "The timetabling data is one of those causes to get agitated about," Shadbolt said. Berners-Lee said that in the US you couldn't just take cable television schedule data and put it on any site. It's private and viewed as valuable in the US. In the UK, television scheduling data is not private. He hoped to create an atmosphere of "competitive disclosure", where groups (and not just government groups) saw it as competitively advantageous to release data. 5.18pm: Speaking of competitive advantage, Allan highlighted a possibility for developers. A government data app that works well in the UK might work well in the US or Australia, and British developers might be able to sell the apps they built for data.gov.uk to use in other markets, including the large US market. 5.20pm: One last point, data.gov.uk is not one large database, Shadbolt said. It's a hub linking together government data. It's a decentralised model, not one huge database. Shadbolt mentioned this several times, probably to head off misconceptions that the data.gov.uk was a one, huge, Orwellian database chock full of data on British citizens. Just in case you're confused. It's not that. 5.24pm: The point of data.gov.uk was to shift the default in UK to publishing data if it is collected, Shadbolt said. "It's shifting the default from 'why?' to 'why not?'", he said. In Australia, they have added their open data strategy into their freedom of information policy, Allan said. One way to cut down on FOI requests was simply to publish the data in the first place, he added. 5.29pm: What's the time line for all of this work? Berners-Lee said, "I'm not inclined to guess. I'm more inclined to declare that this time next year more than 50% of this will be on the web." Shadbolt laughed uncomfortably and turned a bit red. He added, "Tim and I are appointed until June." There are different milestones in the project, Shadbolt added. He talked about league tables to compare different branches of government. It sounded like the kind of competition in data release that Berners-Lee suggested. Topics Tim Berners-Lee Technology blog Internet Government data Free our data