Replied in thread
I argue that there is some understanding in the #psychometrics and professional communities that use #TestScores that evidence of quality is tentative as any statistical result.
And in my personal view, the different methodologies #COSMIN has developed (e.g., https://link.springer.com/article/10.1007/s11136-018-1798-3) or is currently pushing forward (https://doi.org/10.1186/s13643-022-01994-5) are tools to aggregate available evidence exactly for this reason: each individual study and estimate offers an incomplete picture.
SpringerLinkCOSMIN guideline for systematic reviews of patient-reported outcome measures - Quality of Life ResearchPurpose Systematic reviews of patient-reported outcome measures (PROMs) differ from reviews of interventions and diagnostic test accuracy studies and are complex. In fact, conducting a review of one or more PROMs comprises of multiple reviews (i.e., one review for each measurement property of each PROM). In the absence of guidance specifically designed for reviews on measurement properties, our aim was to develop a guideline for conducting systematic reviews of PROMs. Methods Based on literature reviews and expert opinions, and in concordance with existing guidelines, the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) steering committee developed a guideline for systematic reviews of PROMs. Results A consecutive ten-step procedure for conducting a systematic review of PROMs is proposed. Steps 1–4 concern preparing and performing the literature search, and selecting relevant studies. Steps 5–8 concern the evaluation of the quality of the eligible studies, the measurement properties, and the interpretability and feasibility aspects. Steps 9 and 10 concern formulating recommendations and reporting the systematic review. Conclusions The COSMIN guideline for systematic reviews of PROMs includes methodology to combine the methodological quality of studies on measurement properties with the quality of the PROM itself (i.e., its measurement properties). This enables reviewers to draw transparent conclusions and making evidence-based recommendations on the quality of PROMs, and supports the evidence-based selection of PROMs for use in research and in clinical practice.