UZH Reproducibility Day 2019

A day devoted to transparent and reproducible research practices.

Tuesday, February 5, 2019, KOH-B-10.

Open to all members of UZH: students, researchers, administrative and technical staff

No registration required.

About UZH Reproducibility Day

Get information on issues of reproducibility, learn about solutions and offers at UZH. Take the plunge and practice with experts from CRS in hands-on workshops. Presentations and workshops by experts from across UZH as well as invited speakers:

Joachim Wagner, chief editor of The International Journal for Re-Views in Empirical Economics and Nathalie Le Bot, senior editor at Nature will give the publisher's perspective on replication and reproducibility.

Organized by

Satellite Workshop

Software Carpentry Workshop on February 7 and 8, 2019

Aiming to help researchers get their work done in less time and with less pain by teaching them basic research computing skills. This hands-on workshop will cover basic concepts and tools, including program design, version control, data management, and task automation. Participants will be encouraged to help one another and to apply what they have learned to their own research problems.

More information and registration for waiting list

9:00 Start

9 : 00 - 9 : 30

Introduction

By Leonhard Held (CRS)

9 : 30 - 10 : 00

Reproducible Qualitative Social Science: A Plea for Action

By Marco Steenbergen (CRS)

Using Qualitative Comparative Analysis (QCA) as a case study, the first half of this lecture shows that reproducibility is a problem in qualitative social science. Numerous decisions enter the approach, many of them having direct consequences for subsequent findings. In many cases, the choices made are highly subjective but often hidden. This hampers reproducibility. The second half of this lecture focuses on practices that could improve reproducibility. Those practices go beyond QCA and can also inform other forms of qualitative inquiry.

10:00 Break

10 : 00 - 10 : 30

Self-catered coffee break

10 : 30 - 10 : 45

Replication Studies

By Martina Grunow, managing editor of The International Journal for Re-Views in Empirical Economics (program changed: Joachim Wagner, editor in chief had to cancel)

10 : 45 - 11 : 00

Reproducibility at Nature

By Nathalie le Bot, senior editor of Nature

11 : 00 - 11 : 15

Reproducibility Matters

By Lawrence Rajendran/Tamara Zaytouni, Science Matters

11 : 15 - 11 : 30

Open Access and Open Data at UZH

By Andrea Malits (HBZ)

11 : 30 - 12 : 00

Podium and General Discussion

By Martin Grunow, Natalie le Bot, Lawrence Rajendran/Tamara Zaytouni, Andrea Malits

12:00 lunch

12 : 00 - 13 : 15

Self-catered lunch break

13 : 15 - 13 : 45

Standards in Clinical Research and What We Can Learn from Themw

By Regina Grossmann (CTC), Ulrike Held (EBPI, CRS), Stefanie von Felten (EBPI)

13 : 45 - 14 : 00

Reproducibility in Animal Research

By Paulin Jirkof (Animal Welfare UZH)

Reproducibility issues with animal studies have become a highly discussed topic in the scientific community. The ethical construct commonly used to justify the use of animals in research is that of the “greater good” that can be achieved with results of animal experimentation. Results of animal based research are the foundation and reason for further animal experiments and are used to provide efficacy and safety determinations for clinical studies in humans. This talk will give a short overview on the ethical and animal welfare considerations that arise when a study using animals is not reproducible and how the scientific community is trying to tackle the reproducibility issue.

14 : 00 - 14 : 15

Open Science Principles

By Simon Schwab (CRS)

Open science principles are part of good research practice. This talk will give a short overview of the most important principles, such as pre-registration, open methods, and data sharing . The presentation will also address why the focus must shift from significant results to the moment prior to the analysis: the research question and the methods.

14 : 15 - 14 : 45

The Digitalization of Science — Opportunities and Challenges for Reproducibility

By Abraham Bernstein (DSI, CRS)

The Digitalization for Science is well on its way. On one side, scientific processes are increasingly automated and artificial intelligence techniques are “invading" almost any research domain. On the other side, technology allows for a previously unprecedented scale of collaboration. Based on examples from Computer Science and other related domains this talk introduces a number of these innovations and reflects on their risks and opportunities for ensuring the reproducibility of results.

14:45 break

14 : 45 - 15 : 15

Self-catered coffee break

15 : 15 - 15 : 30

iPortal: Automated and Reproducible Computing at UZH

By Lars Malmstrom (S3IT)

Reproducible computing and data analytics can be challenging. At S3IT, we have developed iPortal, an integrated data and workflow manager that enables users to upload and analyze their data using a web browser. The iPortal workflows achieve reproducibility by using singularity containers and generates Jupyter Notebooks that can be extended by the user.

15 : 30 - 16 : 05

Hands on: Preregistration Platforms and Templates

By Nicolas Langer (Institute of Psychology, CRS)

Please bring a laptop!

16 : 05 - 16 : 40

Hands on: Sample Size Planning with R

By Carolin Strobl (CRS)

Please bring a laptop!

Sample size calculation is an important prerequisite for planning scientific studies. This presentation will review the statistical basics of how sample size and other factors determine the power of significance tests (bring your laptop to use an online app for the illustration). Afterwards, sample size calculation for standard methods like t-test and regression by means of the R package pwr is illustrated (install R and the pwr package on your laptop to actively follow the example; it is also fine to just watch). In the end, the presentation will discuss critical choices that need to be made for sample size calculation in practice and give a short outlook on sample size calculation for more advanced methods like multilevel models.

16:40 break

16 : 40 - 16 : 55

Self-catered coffee break

16 : 55 - 17 : 30

Hands on: Design of Replication Studies

By Leonhard Held, Manuela Ott, Charlotte Micheloud, Samuel Pawel (CRS)

Please bring a laptop!

It is conventionally thought that a replication of a significant experiment will have a high probability of resulting again in statistical significance. However, even in the absence of publication or other biases of the original study effect, this "replication probability" is substantially lower than expected. We will illustrate through examples how it depends on the p-value and the uncertainty of the effect estimate from the original study. This in turn implies that the replication sample size has to be quite large to ensure a significant replication result with high probability. We will illustrate this with a Shiny App, which can be used by the participants to design their own replication studies.

17 : 30 - 18 : 05

Hands on: Reproducible Analysis Pipelines with R Notebooks

By Simon Schwab, Eva Furrer (CRS)

Please bring a laptop!

R Notebooks can largely improve the reproducibility of data processing and statistical analyses. Notebooks are electronic documents containing code chunks that can be executed independently and interactively, with output directly visible next to the code. We will give a brief introduction into R Notebooks and R Markdown and demonstrate its technicalities and strengths. Afterwards, participants analyze themselves an example dataset with a provided R Notebook to produce publication-ready figures and tables. RFully reproducible analysis pipelines not only strengthen the credibility and transparency in research, but also improve the efficiency of the researcher in case an analysis needs to be modified or redone, for example, during the peer-review process.

18:05 Apéro

18 : 05 - 19 : 00,

Apéro

Contact Info

logo

UZH CRS, EBPI, Hischengraben 84