BlueHealth researchers have published a new paper explaining how they tested the reliability of the BlueHealth Environmental Assessment Tool, or BEAT for short.
Led by Dr Himansu Mishra, this work was done to better understand how blue spaces may influence health-promoting behaviours. In order to achieve a good state of knowledge about such spaces, the BEAT had to be developed to be effective and reliable.
Testing the reliability of the tool required a two-stage approach. In Stage 1, one common and several different expert assessors rated 16 sites independently and their results were compared. In Stage 2, two assessors rated 21 sites independently and their results were compared. The Inter-class correlation coefficient (ICC) was calculated to assess inter-rater reliability for both stages. Stage 2 results showed greater reliability after enhanced training of the assessors. To demonstrate the effectiveness of the tool at revealing differences between sites and for identifying health promoting affordances we carried out intra and inter-site comparisons of a subset of six sites for the Stage 1 and 18 sites for Stage 2.
The researchers found that the tool performs consistently and compares well to the reliability shown by other similar tools. They also found that deeper training and use of the BEAT guidance improved tool’s reliability and the level of agreements. Once evaluators had been through such training, the BEAT was found to successfully reveal similarities and differences between different blue spaces of a broadly similar character.
Importantly, the results presented in the paper demonstrate conclusively that the tool can be used reliably (with training and guidance) and that it provides meaningful data to help planners and designers assess the environmental characteristics of different blue space sites.
