PREPARING FOR A SYSTEMATIC REVIEW: PLANNING THE PROCESS transcript 2 00:00 --> 00:14 Welcome to the 3rd video in our Conducting a Systematic Review, planning the process, class. In this video, we will address items from the PRISMA-P checklist's methods section. 3 00:15 --> 00:36 Item eight: eligibility criteria. These are aspects of the records you're going to be screening that would qualify them for inclusion or exclusion from your review. You should be explicit in stating these criteria, because you want the least room for subjectivity when screening. All team members should be on the same page. 4 00:37 --> 01:25 You can easily build eligibility criteria from your PICO. For example, here is the PICO for the acute pancreatitis review we discussed in the last video, as well as the criteria listed in their protocol on PROSPERO. Their population of interest was hospitalized adults with acute pancreatitis. For their eligibility criteria, they specified the age range as adults over 18 years of age. For eligibility criteria related to the intervention they specified the types of feeding they were interested in. They also specified what constitutes early feeding; feeding initiated promptly within 48 hours. They similarly elaborated on timing of feeding for the comparison, delayed feeding. 5 01:26 --> 01:46 For criteria related to outcomes, their main outcomes of interest were length of stay, readmissions, and mortality. They also have a lengthy list of secondary outcomes, though they state that there may be other outcomes beyond this list. When screening, you may come across outcomes that you had not anticipated, but the more you specify in advance, the better. 6 01:47 --> 02:14 Item nine: information sources. Here you list all of the places that you're planning to search for literature. Usually, reviews pull sources from a minimum of three databases. Standard bibliographic databases, such as PubMed or Web of Science, primarily index formally published literature- journal content, sometimes book chapters. If you are working with a librarian, they can help to guide you to the appropriate databases. 7 02:15 --> 02:54 In addition to bibliographic databases, standards recommend grey literature sources. Grey literature is anything not formally published. This includes posters, government reports, dissertations and theses, and entries in clinical trial registries. Searching such sources helps to reduce publication bias. There is a tendency to assume that if something wasn't published, that it is somehow deficient. That's not necessarily true. For example, some dissertations or posters just haven't been published yet. Perhaps the team presented preliminary research and the study is ongoing or they're just not done with the final manuscript. 8 02:55 --> 03:17 Clinical Trial registries are an extremely important source of grey literature. There's a lot of great research out there that doesn't make it to publication. Take, for example, a clinical trial for a pharmacologic intervention. Negative or neutral results are much less likely to be published. Consequently, if you only use the published data on your topic, your review will be skewed. 9 03:18 --> 03:33 Item 10: search strategy Include a draft of your planned search strategy for a single database. You can update this once you have a finalized search. Ideally, you will be working with a librarian who will create this strategy for you. 10 03:34 --> 03:50 Item 11a: data management. Here you explain your plans for managing data and records throughout your review. This includes documenting all search strategies and results, filing and folder systems, etc. 11 03:51 --> 04:24 One aspect of data management that the library can assist with are Endnote, which we generally use for deduplication and full text retrieval in systematic reviews. If you are working closely with a librarian, they would manage this aspect of the review. Otherwise, we offer monthly Endnote workshops and one-on-one training by request. Our DataLab offers a class on data management and data sharing, which covers file naming and documentation, metadata and data best practices, and data sharing tools. 12 04:25 --> 04:54 Item 11b: the selection process Here you will specify your method for selecting studies (such as how many reviewers will screen records and how conflicts will be resolved). You will need to select a screening tool. Some people use Excel or Endnote, but they aren't ideal. I highly recommend that you use a specific systematic review screening tool. Our users most commonly use Rayyan or Covidence, both web-based platforms. 13 04:55 --> 05:47 Covidence is available for a fee. It is more intuitive than Rayyan and offers more features, including built in data extraction tables and an auto-populated PRISMA diagram. Rayyan, however, is free and is still a very useful tool. Both tools will allow your team members to screen records independently, blinded to what other reviewers are doing. When each team member is done screening, Rayyan and Covidence will identify conflicts for your team to resolve. Both also allow you to enter keywords related to your inclusion and exclusion criteria, and will highlight those keywords in green (inclusion terms) and red (exclusion terms). They also allow you to assign reasons for exclusion, which will be helpful when it comes time to report the number of records excluded for each reason in your manuscript. 14 05:48 --> 06:05 You should also report whether you will perform pilot testing, in which all reviewers screen a small subset of records and discuss conflicts. This helps to flush out any areas of confusion before you embark on the full title/abstract screening. 15 06:06 --> 06:24 I mentioned the PRISMA flow diagram before. Here is the template. A screening tool will track the numbers you need to fill out half of this diagram: records screened, records excluded, full text articles assessed for eligibility, and full text articles excluded, with reasons. 16 06:25 --> 07:15 Item 11c: data collection. These are your methods both for extracting data from included documents and any processes for obtaining additional data from investigators (which you might need to do when the record in question is a poster or conference abstract). There is no standardized form for data extraction. You can create a form from scratch or adapt an existing form. If you go to the Tools page on our Systematic Reviews GalterGuide, we link to a Data Extraction Form adapted from the Cochrane Collaboration. Whatever data points you choose to extract, you also need to select in what format you will record them. You could use a simple Word document, Qualtrics, RedCap, Excel, Google Forms/Sheets, etc. 17 07:16 --> 08:02 Item 14: risk of bias. You will assess the risk of bias for each record included in your review. There is no standardized risk of bias checklist. There are a lot of different tools, some of which are listed on the Tools for Reviewers page of our Reporting Research GalterGuide. RoB 2, which is Cochrane's risk of bias tool for randomized trials, and ROBINS-I, Cochrane's risk of bias tool for non-randomized studies of interventions, are commonly used. There are also subject specific tools, such as the PEDro Scale for physiotherapy trials. Choose what works for your team and the types of studies that you will be including in your review. 18 08:03 --> 08:32 Item 15: data synthesis Here you state your planned methods for quantitative or qualitative analysis. If you require guidance, you can locate statistical books through Galter, several of which are listed on our Reporting Research GalterGuide. You can also reach out to the Biostatistics Collaboration Center. They offer an initial one hour consultation for free. Lengthier collaborations or in-depth support or available for a fee. 19 08:33 --> 08:59 Item 17: confidence in cumulative estimate How will you assess the overall strength of the body of evidence. In the follow-up class to this (Part 2 – Tools & Resources), we discuss the distinction between this and risk of bias assessment in detail. The GRADE approach, Grading of Recommendations Assessment, Development, and Evaluation, is commonly used, though there are other options. 20 09:00 --> 09:16 That's it for the PRISMA P checklist. If you would like to look at a couple of sample protocols, here are two that were created with the guidance of Galter librarians. The first is registered on PROSPERO and the second was published in the journal Systematic Reviews. 21 09:17 --> 09:42 Systematic review standards recommend having a librarian on the team. At Galter Library, we provide two levels of support. We can work with you as a consultant: we would meet with you once for systematic review education, assistance refining your search question, and search training. The second level is full collaboration, in which the librarian receives co-authorship credit on your published review. 22 09:43 --> 10:14 In this model, the librarian will help formulate your research question, search for existing reviews on your topic, develop and run the search strategy, deliver deduplicated results, assist with full text retrieval, document the search strategy and process, and write up search methods for the manuscript. For this level of support, we require that teams have a protocol. Consult with a librarian at the start of your project. Our librarians usually require a minimum of one month to create search strategies and process results. 23 10:15 --> 10:52 These are the works referenced during this video series. More resources are listed on our Systematic Reviews GalterGuide. If you require assistance with a systematic review, please submit an online request form, which is provided on the Galter Services page of our Systematic Reviews GalterGuide, or reach out to your liaison librarian. To determine your liaison librarian, go to our homepage, click Research Services in the main menu, then Find my Liaison Librarian. Liaisons are listed here for each department, division, and institute. 24 10:53 --> 11:12 This concludes our "Conducting a Systematic Review: Planning the Process" video series. To learn about selected review tools and resources, take our "Conducting a Systematic Review: Tools and Resources" class or watch the corresponding video series.