Open Web Index

Open Web Index (OWI)

The main Idea of Open Web Index is to set up a publicly funded, global, searchable index of the Web that is open to competing companies, institutions and civil society actors. The Open Web Index is a European-based approach for a competitive and data-protecting digital infrastructure. It aims to build a basis for genuine competition in the digital platform business.

Vision

To build a public library of the Web.

Technical idea

Separate the index from the services that use the index.

Public vs. private

While the index should be public, the services can be proprietary.

Possible applications

Search engines, web analytics, artificial intelligence, ...

Structure for separating index and services

Separating the index and the servicesallows for a multitude of services, whether existing as search engines or otherwise, to be run on a shared infrastructure. The figure shows how the public infrastructure is responsible for crawling the web, for indexing its content, and for providing an interface and application programming interface (API) to the services that are built upon the index. While services are allowed to do their own further indexing to prepare documents, some advanced indexing is also provided by the open infrastructure. Furthermore, as modern search engines rely heavily on usage data, these data (most prominently search queries routed to the index) are collected and made available for reuse.

Some possible applications

A web index puts unstructured and heterogeneous data of the Internet into a structured shape. This enables setting up Internet services, which process this data for different kinds of usage.

Publications

Lewandowski, D. (2019). The web is missing an essential part of infrastructure: An Open Web index. Communications of the ACM, 62(4), 24–24. https://doi.org/10.1145/3312479

Lewandowski, D. (2016): Perspektiven eines Open Web Index. Information Wissenschaft und Praxis 67(2016)1, 15-21. https://doi.org/10.1515/iwp-2016-0020

Lewandowski, D. (2014): Why we need an independent index of the Web. In: König, R.; Rasch, M. (eds.): Society of the Query Reader. Amsterdam: INC, 2014, pp. 50-58.

Journalistic articles

Stefan Krempl: Missing Link: Offener Web-Index soll Europa bei der Suche unabhängig machen. Heise Online, 22 January 2023.

Dirk Lewandowski, Jutta Haider & Olof Sundin: ”Stater och myndigheter kan inte förlita sig på Googles goda vilja.” Sydsvenskan, 25 May 2021.

Dirk Lewandowski and Olof Sundin: How an Open Web Index Could Help Solve the "Search Problem". Medium, 13 April 2021.

Christian Kunz: Europäisches Gegenmodell zu Google? Der Open Web Index. SEO Südwest, 8 November 2015.

Dirk Lewandowski: Warum wir einen freien Web-Index brauchen. iRights.info, 12 May 2014.