Apps @ CRSU

DETAILED LIST OF APPS FEATURES

Below is a list of common features/principles that we try and adhere to when developing the CRSU apps.

• The apps have all been developed using R and {shiny} (with JAGS and STAN for Bayesian analysis)
• Where possible we utilise existing R packages. This is definitely a “standing on the shoulders of giants” principle. There are many meta-analysis packages for R and some of them are extensive and fantastic. We try and leverage the power of the best ones (and acknowledge the ones we use in the apps). In more recent apps (e.g. MetaBayesDTA and MetaCNMA) it has been necessary to write more of the analysis code ourselves due to the lack of packages with the desired functionality.
• All our apps are free to use and open source with Github pages with the code and re-use licences. Our apps are beginning to be embedded within other’s systematic review software with our permission and we may receive a royalty licence for such usage but we plan to keep the stand-alone apps free, and hosted by the CRSU online, for as long as we can afford to do so.
• We aim to make the interfaces as simple and accessible as possible utilising “point and click” principles. The R software and all the user developed packages for meta-analysis are a fantastic resource but we understand there are barriers to entry.
• Where appropriate, the apps have built in example datasets. We have looked at many apps without this feature and it can be very frustrating. A built-in dataset allows the user to “test drive” the app straight away to establish whether it is appropriate for their needs as well as illustrate the data format(s) the app requires.  
• It is not our intention for the apps to replace expert statistical input, but we appreciate statistical input can be in short supply and for those who know what they want to do but not how to our apps should be helpful, although input from a statistician is recommended. Having said this, it is hoped that the apps have educational value and we know they are used in teaching around the world.
• Sensitivity analysis is used to assess the robustness of the results to the assumptions made in the analysis. For example, there may be concerns that some studies have been carried out with less rigour, and, therefore, are more susceptible to bias, but this has been ignored in the initial analysis, essentially making the assumption that all studies are free of bias. The impact of this can be explored by removing the lower quality studies to assess how this potentially impacts the overall findings of the meta-analysis. Whilst such investigations are not technically difficult to conduct, they can become fiddly and tedious to code, disincentivising the busy researcher from carrying them out. We have aimed to facilitate such analyses allowing study exclusion, via a simple “click to select” approach which we hope encourages such analyses to be carried out.
• Whilst we aim to provide all results in numeric formats, we aim to put an emphasis on providing visualizations of results where possible. We hope this facilitates both interpretation and understanding. We introduce interactivity to visualisations to further assist with result exploration and provide options where results are complex/multifaceted.
• All numeric and graphical output is readily available for exporting into documents etc. This includes providing the researcher with plots of publication quality.
• Finally, we have recently attempted to conform to standards required by open science / FAIR data principles. This can be challenging, for example, it requires that all analyses conducted in the apps are transparent and reproducible. We added functionality to MetaPairwise as a test case for providing the underlying R code require to conduct the identical analysis from with R. This is demanding to programme, but we hope to offer this functionality for other apps in the future. For Bayesian analyses generic JAGS/Stan code has been included in the apps for some time.