Frequently Asked

What does CRIS stand for?

The acronym CRIS stands for Crowdsourcing Research Questions in Science. CRIS is part of the project Open Innovation in Science, which will run over several years. Tell us is the name of this crowdsourcing project, and it is initiated by the Ludwig Boltzmann Gesellschaft. From the research conducted, we aim to increase the direct benefits that can be achieved for society as a whole. You can get more information on www.redensiemit.org

Last update on 17.09.2015 by Roland Schütz.

What is crowdsourcing?

Crowdsourcing describes the way individual problems and tasks of an organization can be solved respectively, taken care of, with the help of crowed-based external knowledge carriers. This includes the use of various online platforms.

Core elements of crowdsourcing are:

  1. An organization/institution/individual has a problem or a task for which a solution is required
  2. A community (crowd) would like to solve this problem or the task voluntarily
  3. An online platform brings both sides together and enables interaction
  4. The bearer of the problem, as well as the crowd, benefit from solving the problem or the task

(Based on the definition by Daren C. Brabham, Crowdsourcing (MIT Press Essential Knowledge), 2013)

Last update on 09.10.2014 by Roland Schütz.

What is the difference between crowdsourcing and citizen science?

Crowdsourcing, citizen science – also known as crowd science – basically describes public participation in scientific research. Frequently, it deals with statistic surveys or projects where laypeople report on observations, evaluate data according to instructions, or carry out measurements. The crowdsourcing process CRIS, of the Ludwig Boltzmann Gesellschaft goes far beyond this and offers citizens the opportunity to directly influence generating research questions. In CRIS, laypersons can be creative; the aim is to inspire a discussion on an equal footing by all those involved from which later on, hypotheses and new research questions can be derived.

Last update on 09.10.2014 by Roland Schütz.

What is the difference between crowdsourcing and crowdfunding?

The goal of the Ludwig Boltzmann Gesellschaft’s CRIS crowdsourcing project is to collect the concerns of those affected, as well as problems and proposed solutions. Crowdfunding, on the other hand, describes a type of financing which has no relevance for CRIS.

Last update on 09.10.2014 by Roland Schütz.

Why is crowdsourcing deployed for such a specific theme?

The field of psychological illnesses was selected in a carefully chosen process in which, besides secondary data, international expert interviews were asked included.

Basically, psychological illnesses are highly relevant on a health policy and research level. Generally, the (international) high prevalence is faced with a significantly lower research activity, as 20 per cent of all persons develop psychological illness in their lifetime.

Last update on 10.12.2014 by Roland Schütz.

How was the crowdsourcing project organized?

First, ideas, problems and/or proposed solutions were collected from heterogeneous stakeholders. In a second step, the themes and problem areas were clustered. This step was followed by the evaluation, and subsequently the selection of individual themes that will serve as an impulse for new research processes. You can find more information here.

Last update on 09.10.2015 by Roland Schütz.

When did the crowdsourcing by CRIS took place?

We asked participants to submit their unsolved questions and problems in the field of mental health issues on the platform www.redensiemit.org. The platform was open from 16 April until 6 July.

Last update on 09.10.2015 by Roland Schütz.

How does it differ to conventional patient platforms / forums?

Conventional patient platforms and forums serve as an exchange for patients amongst themselves or are web portals from which medical advice can be obtained. Tell us aims at gathering impulses and concerns associated with psychological illnesses from those affected and interested, in order to communicate these to the research process and attain new, socially-relevant research issues.

Last update on 09.10.2015 by Roland Schütz.

Are the contributions anonymous?

It was not possible to submit a contribution without previous registration. All contributions were treated confidentially and were, of course, subject to data protection. At no point in time were submitted contributions accessible by other participants. Therefore, no participant was able to read or comment on contributions submitted by others. 

Participants could delete their profile after submitting their contributions. In this case the contributions remained in the system, but could not be associated with the former profile.

Last update on 21.01.2016 by Roland Schütz.

How were the results sorted and processed further?

The evaluation and ranking of the results occured through a two-stage process. In September, the newly defined research topics were presented on the plattform. For tow weeks registered participants had the possibility to vote for the topics they considered most relevant. At October 22, an international expert-jury finalized the ranking.


Last update on 21.01.2016 by Roland Schütz.

Is such an amount of data manageable? And how can one ensure the quality of the crowdsourcing results?

We were expecting a multitude of submissions to CRIS. A trained team went through an extensive analysis process, which was computer-aided, and supported by experts from the field of mental health. This ensured that all contributions received due attention and care.

Last update on 21.01.2016 by Roland Schütz.

What about the protection of the ideas (intellectual property)?

This crowdsourcing was not about generating ideas, but rather having participants submitted contributions about unsolved issues, unanswered questions and personal experience. Thus, the question of protection of intellectual property did not arise.  All Contributions were treated confidentially and are, of course, subject to data protection.

Last update on 21.01.2016 by Roland Schütz.

Are there best practice examples for successful crowdsourcing in science?

To date, crowdsourcing in science has only been used in very few cases. An example is the Harvard case in which the question "What do we not know about how to cure type 1 diabetes?" was asked and answered in a comprehensive crowdsourcing process. The Harvard Medical School not only wanted to find new solutions regarding type 1 diabetes, but to generally examine how open innovation could be used in a scientific research process, i.e. concerning the formulation of research questions or the evaluation of research applications.

To a large extent, the project was financed by the National Institute of Health, the national research agency in the USA. The use of tax revenues for such a project, in combination with one of the most renowned universities in the world – suggests that the theme open innovation in science is of top relevance internationally. 

Last update on 09.10.2014 by Roland Schütz.

What happens to the crowdsourcing results?

The Ludwig Boltzmann Gesellschaft will distribute the results to the national as well as the international research community and make them available to Austrian science policy makers and to the EU.

Last update on 19.02.2015 by Roland Schütz.

How was the theme "psychological illnesses" selected?

The theme was chosen in an extensive evaluation process by the Ludwig Boltzmann Gesellschaft. Several criteria were used:

  • Psychological illnesses in relation to other illnesses (i.e. cancer) are subject to little research
  • High prevalence, and thus widespread concern in the population (both those directly affected and relatives, all age groups)
  • All age groups and social classes are concerned with psychological illness
  • Highly relevant to health policy (the results of psychological illnesses have drastic effects on society, i.e. early retirement, long-term sick leave, etc.)

Last update on 20.02.2015 by Roland Schütz.

Who chose the theme?

With reference to the analysis of secondary data and interviews with international experts, the theme was recognised as a priority by the Ludwig Boltzmann Gesellschaft and chosen for the Open Innovation in Science initiative.

Last update on 09.10.2014 by Roland Schütz.

How did you ensure that all relevant stakeholders were involved?

An exact, comprehensive stakeholder mapping was carried out. Informing the relevant stakeholders was carried out by the project team, by the Advisory Board as well as the project partners.

Last update on 21.01.2016 by Roland Schütz.