Open data
Open data is the idea that some data should be freely available to everyone to use and republish as they wish, without restrictions from copyright, patents or other mechanisms of control. The goals of the open-source data movement are similar to those of other "open" movements such as open-source software, hardware, open content, open education, open educational resources, open government, open knowledge, open access, open science, and the open web. Paradoxically, the growth of the open data movement is paralleled by a rise in intellectual property rights. The philosophy behind open data has been long established, but the term "open data" itself is recent, gaining popularity with the rise of the Internet and World Wide Web and, especially, with the launch of open-data government initiatives such as Data.gov, Data.gov.uk and Data.gov.in.
Open data can also be linked data; when it is, it is linked open data. One of the most important forms of open data is open government data, which is a form of open data created by ruling government institutions. Open government data's importance is borne from it being a part of citizens' everyday lives, down to the most routine/mundane tasks that are seemingly far removed from government.
Overview
The concept of open data is not new, but a formalized definition is relatively new. Conceptually, open data as a phenomenon denotes that governmental data should be available to anyone with a possibility of redistribution in any form without any copyright restriction. One more definition is the Open Definition which can be summarized in the statement that "A piece of data is open if anyone is free to use, reuse, and redistribute it – subject only, at most, to the requirement to attribute and/or share-alike." Other definitions, including the Open Data Institute's "", have an accessible short version of the definition but refer to the formal definition.Open data may include non-textual material such as maps, genomes, connectomes, chemical compounds, mathematical and scientific formulae, medical data, and practice, bioscience and biodiversity. Problems often arise because these are commercially valuable or can be aggregated into works of value. Access to, or re-use of, the data is controlled by organisations, both public and private. Control may be through access restrictions, licenses, copyright, patents and charges for access or re-use. Advocates of open data argue that these restrictions are against the common good and that these data should be made available without restriction or fee. In addition, it is important that the data are re-usable without requiring further permission, though the types of re-use may be controlled by a license.
A typical depiction of the need for open data:
Creators of data often do not consider the need to state the conditions of ownership, licensing and re-use; instead presuming that not asserting copyright puts the data into the public domain. For example, many scientists do not regard the published data arising from their work to be theirs to control and consider the act of publication in a journal to be an implicit release of data into the commons. However, the lack of a license makes it difficult to determine the status of a data set and may restrict the use of data offered in an "Open" spirit. Because of this uncertainty it is also possible for public or private organizations to aggregate said data, protect it with copyright and then resell it.
The issue of indigenous knowledge poses a great challenge in terms of capturing, storage and distribution. Many societies in third-world countries lack the technicality processes of managing the IK.
At his presentation at the XML 2005 conference, Connolly displayed these two quotations regarding open data:
- "I want my data back."
- "I've long believed that customers of any application own the data they enter into it."
Major sources
In science
The concept of open access to scientific data was institutionally established with the formation of the World Data Center system, in preparation for the International Geophysical Year of 1957–1958. The International Council of Scientific Unions oversees several World Data Centres with the mandate to minimize the risk of data loss and to maximize data accessibility.While the open-science-data movement long predates the Internet, the availability of fast, ubiquitous networking has significantly changed the context of Open science data, since publishing or obtaining data has become much less expensive and time-consuming.
The Human Genome Project was a major initiative that exemplified the power of open data. It was built upon the so-called Bermuda Principles, stipulating that: "All human genomic sequence information should be freely available and in the public domain in order to encourage research and development and to maximize its benefit to society'. More recent initiatives such as the Structural Genomics Consortium have illustrated that the open data approach can also be used productively within the context of industrial R&D.
In 2004, the Science Ministers of all nations of the Organisation for Economic Co-operation and Development, which includes most developed countries of the world, signed a declaration which essentially states that all publicly funded archive data should be made publicly available. Following a request and an intense discussion with data-producing institutions in member states, the OECD published in 2007 the OECD Principles and Guidelines for Access to Research Data from Public Funding as a soft-law recommendation.
Examples of open data in science:
- The Dataverse Network Project – archival repository software promoting data sharing, persistent data citation, and reproducible research
- – Open data about scientific artifacts from the University of Muenster, Germany. Launched in 2011.
- – Open scientific datasets encoded as Linked Data. Launched in 2011.
- – Open scientific datasets related to wildlife classified by animal species. Launched in 2015.
In government
Some make the case that opening up official information can support technological innovation and economic growth by enabling third parties to develop new kinds of digital applications and services.
Several national governments have created websites to distribute a portion of the data they collect. It is a concept for a collaborative project in the municipal Government to create and organize culture for Open Data or Open government data.
Additionally, other levels of government have established open data websites. There are many government entities pursuing Open Data in Canada. Data.gov lists the sites of a total of 40 US states and 46 US cities and counties with websites to provide open data; e.g. the state of Maryland, the state of California, US and New York City.
At the international level, the United Nations has an open data website that publishes statistical data from member states and UN agencies, and the World Bank published a range of statistical data relating to developing countries. The European Commission has created two portals for the European Union: the EU Open Data Portal which gives access to open data from the EU institutions, agencies and other bodies and the PublicData portal that provides datasets from local, regional and national public bodies across Europe.
In October 2015, the Open Government Partnership launched the International Open Data Charter, a set of principles and best practices for the release of governmental open data formally adopted by seventeen governments of countries, states and cities during the OGP Global Summit in Mexico.
In non-profit organizations
Many non-profit organizations offer more or less open access to their data, as long it does not undermine their users', members' or third party's privacy rights. In comparison to for-profit corporations, they do not seek to monetize their data. OpenNWT launched a website offering open data of elections. CIAT offers open data to anybody, who is willing to conduct big data analytics in order to enhance the benefit of international agricultural research. DBLP, which is owned by a non-profit organization Dagstuhl, offers its database of scientific publications from computer science as open data. Non-profit hospitality exchange services offer trustworthy teams of scientists access to their anonymized data for publication of insights to the benefit of humanity. Before becoming a for-profit corporation in 2011, Couchsurfing offered 4 research teams access to its social networking data. In 2015, non-profit hospitality exchange services Bewelcome and Warm Showers provided their data for public research.Arguments for and against
The debate on open data is still evolving. The best open government applications seek to empower citizens, to help small businesses, or to create value in some other positive, constructive way. Opening government data is only a way-point on the road to improving education, improving government, and building tools to solve other real world problems. While many arguments have been made categorically, the following discussion of arguments for and against open data highlights that these arguments often depend highly on the type of data and its potential uses.Arguments made on behalf of open data include the following:
- "Data belongs to the human race". Typical examples are genomes, data on organisms, medical science, environmental data following the Aarhus Convention
- Public money was used to fund the work and so it should be universally available.
- It was created by or at a government institution
- Facts cannot legally be copyrighted.
- Sponsors of research do not get full value unless the resulting data are freely available.
- Restrictions on data re-use create an anticommons.
- Data are required for the smooth process of running communal human activities and are an important enabler of socio-economic development.
- In scientific research, the rate of discovery is accelerated by better access to data.
- Making data open helps combat "data rot" and ensure that scientific research data are preserved over time.
While the human abstraction of facts from paper publications is normally accepted as legal there is often an implied restriction on the machine extraction by robots.
Unlike open access, where groups of publishers have stated their concerns, open data is normally challenged by individual institutions. Their arguments have been discussed less in public discourse and there are fewer quotes to rely on at this time.
Arguments against making all data available as open data include the following:
- Government funding may not be used to duplicate or challenge the activities of the private sector.
- Governments have to be accountable for the efficient use of taxpayer's money: If public funds are used to aggregate the data and if the data will bring commercial benefits to only a small number of users, the users should reimburse governments for the cost of providing the data.
- The revenue earned by publishing data can be used to cover the costs of generating and/or disseminating the data, so that the dissemination can continue indefinitely.
- The revenue earned by publishing data permits non-profit organisations to fund other activities.
- The government gives specific legitimacy for certain organisations to recover costs.
- Privacy concerns may require that access to data is limited to specific users or to sub-sets of the data.
- Collecting, 'cleaning', managing and disseminating data are typically labour- and/or cost-intensive processes – whoever provides these services should receive fair remuneration for providing those services.
- Sponsors do not get full value unless their data is used appropriately – sometimes this requires quality management, dissemination and branding efforts that can best be achieved by charging fees to users.
- Often, targeted end-users cannot use the data without additional processing – if anyone has access to the data, none may have an incentive to invest in the processing required to make data useful.
Relation to other open activities
- Open access is concerned with making scholarly publications freely available on the internet. In some cases, these articles include open datasets as well.
- Open content is concerned with making resources aimed at a human audience freely available.
- Open knowledge. Open Knowledge International argues for openness in a range of issues including, but not limited to, those of open data. It covers scientific, historical, geographic or otherwise Content such as music, films, books Government and other administrative information. Open data is included within the scope of the Open Knowledge Definition, which is alluded to in Science Commons' Protocol for Implementing Open Access Data.
- Open notebook science refers to the application of the Open Data concept to as much of the scientific process as possible, including failed experiments and raw experimental data.
- Open-source software is concerned with the open-source licenses under which computer programs can be distributed and is not normally concerned primarily with data.
- Open educational resources are freely accessible, openly licensed documents and media that are useful for teaching, learning, and assessing as well as for research purposes.
- Open research/open science/open science data means an approach to open and interconnect scientific assets like data, methods and tools with linked data techniques to enable transparent, reproducible and transdisciplinary research.
Funders' mandates
- to deposit bioinformatics, atomic and molecular coordinate data, experimental data into the appropriate public database immediately upon publication of research results.
- to retain original data sets for a minimum of five years after the grant. This applies to all data, whether published or not.
Non-Open data
Several mechanisms restrict access to or reuse of data. They include:- making data available for a charge.
- compilation in databases or websites to which only registered members or customers can have access.
- use of a proprietary or closed technology or encryption which creates a barrier for access.
- copyright forbidding re-use of the data, including the use of "no derivatives" requirements.
- patent forbidding re-use of the data.
- restriction of robots to websites, with preference to certain search engines.
- aggregating factual data into "databases" which may be covered by "database rights" or "database directives".
- time-limited access to resources such as e-journals.
- "webstacles", or the provision of single data points as opposed to tabular queries or bulk downloads of data sets.
- political, commercial or legal pressure on the activity of organisations providing Open Data.