“Outdoor Air and Health Atlas” project: Explorative web review and quality assessment of web open data Atlases of air pollution effects on health

Background

The World Health Organization (WHO) estimated that 2.4 million people die from causes associated with air pollution every year.^1,2 This number makes air pollution the most important environmental risk factor for mortality and morbidity. Air pollution shortens life expectancy, affecting different organs in the body, and its effect has been assessed and quantified in terms of overall mortality, and for specific causes, such as respiratory and cardiovascular mortality, as well as for lung cancer.^2,3 Given the importance of this issue for the public and decision-makers, some national and international environmental protection agencies made available to citizens and consumers information about the quality of outdoor air in the community and its effects on health.^4,5 The aim was to increase public awareness, make policy makers accountable, and promote participatory decision making. 

The Internet has become a central mass medium for interactively providing information and reference resource for people. Via the World Wide Web, it provides access to a substantial volume of health-related information that might not be readily available through traditional sources.⁶ The amount of health information accessible on the Internet and the number of consumers gaining access to it have significantly increased over the years.⁷ Therefore, it is fundamental that the information published on public websites of agencies is easily understood by professionals, decision makers, and people who are unfamiliar with technical terminology.⁸

Nowadays, several websites provide interactive maps and infographics which depict the impact of air quality on health. The most well-known examples include: the Global Burden of Disease (GBD),⁹ the European Environment Agency (EEA), and The Global Health Observatory.^5,10 GBD estimates air pollution as the cause of over 6.5 million deaths each year globally, and this number is increasing.¹¹ Likewise, the EEA defines air pollution as the major cause of premature death and disease and the single largest environmental health risk in Europe.⁵ Thus, these websites might be useful tools for informing and raising awareness on air pollution consequences.

The Italian Ministry of Health funded the project “ATLAS – Air and Health: an integrated atlas to support decisions and research” aimed at developing a national interactive atlas which relates air quality data and health outcomes and whose contents are understandable and accessible to both professionals and lay users. Therefore, the project coordinators performed a review and analysis of websites available on the internet which present data on the effects of air pollution on health outcomes as a preliminary step to design and develop the Italian atlas. The aim is to be as systematic as possible in searching and retrieving existing websites presenting data on the health impact of air pollution variations in time (trends) or space (maps) or both. Furthermore, the review aims to describe and classify the contents and level of interactivity, as well as evaluate the technical, scientific, and communicative quality of retrieved interactive websites.

Materials and methods

Population-Exposure-Comparator-Outcomes (PECO) frame of the search question

The study aimed to find and analyse websites that provide interactive maps and infographics depicting the impact of air quality on health. Websites may include empirical data or estimates of data obtained from empirical measures that are up to date and show geographical (maps) or temporal (trends) patterns of the health outcomes of air pollution.

To better define the focus of this analysis and the inclusion and exclusion criteria of the review, the Population-Exposure-Comparator-Outcomes (PECO) question framework was adapted to meet the study needs. PECO is the frame used in the fields of environmental, nutritional, and occupational health to formulate health questions about the causality of associations between exposures and outcomes. Therefore, it has a pivotal role in informing the study design or inclusion and exclusion criteria for a review.¹²

The resident population has been considered as Population in PECO framework, since the residence of the subject is usually assumed as the primary reference place for being exposed to air pollution.¹³ As Exposure, the focus was on the main pollutants, including PM₁₀, PM_2.5, NO₂, and O₃. These pollutants are considered of potential interest for the Italian Atlas. Synthetic indicators based on particulate or NO₂, plus eventually other pollutants, have been included. Additional pollutants can be considered if included in addition to one of the exposures of interest. Lower levels of concentration or of exposure to the pollutants of interest, empirically observed or a priori defined counterfactual levels of pollutant concentration, was considered as Comparator. Finally, the Outcomes included were:

• all-cause mortality;
• mortality due to specific causes for which association with air pollution exposure has been demonstrated;
• hospitalization and access to first aid for all causes and for specific air pollution-related causes;
• incidence or prevalence of chronic disease with demonstrated association with air pollution;
• qualitative health and risk indicators.

Eligibility criteria

To be included in the review, websites had to:

• Contain empirical data or estimates of data obtained from empirical ones;
• Be dynamic and up to date;
• Be free to view;
• Not require a password for access;
• Contain maps or infographics;
• Provide comprehensive information which relates air pollution and health outcomes.

Set inclusion or exclusion criteria were not set for the strategies of exposure assessment as comparators, since environmental pollution is typically characterized by high space-time variability and spread.¹³ 

Sources and web search engines

Sites were searched between September and December 2024 by one investigator. The common strategy users employ to search for information on the Internet is through the use of a search engine.¹⁴ Thus, the website search was performed on three of the most popular research engines worldwide (Google, Bing, and Yandex) and in Italy (Google, Bing, and Yahoo), which collectively accounted for between 96% and 98% of all search engine traffic.¹⁵

Search strategy

The included keywords were: air, pollution, health, atlas, interactive maps, infographic. These terms were entered in English in different groupings to facilitate a comprehensive search. The first 200 links/websites for all possible keyword combinations were screened and evaluated, for a total of 1,000 links. If in the first 200 records a saturation were observed – i.e., the last 50 websites were irrelevant, duplicates or overlapped in the search engines or presented in a previous search –, the screening was stopped. If, after the first 200 links, any reasonable saturation was noticed or if in the last 50 had emerged at least one new site, the search continued until saturation was obtained. 

Data extraction

From eligible websites, three reviewers (ED, OD, and PGR) independently extracted website descriptive characteristics and technical and qualitative variables using an electronic data extraction form and stored all the data in an Excel database. Then, the level of concordance in data extraction was evaluated and the final decision about the variables to consider was made by consensus or majority among reviewers. 

Descriptive characteristics of the website include all the data that outline what information is contained within the website and how it is provided.16 In the present study, the considered descriptive characteristics were:

• Site activation year;
• Promoter/editor;
• Data source;
• Target/aim;
• Geographical coverage;
• Time coverage;
• Exposure type of pollutants;
• Measure applied (daily value, annual average, ect);
• Time window (annual average or on-time);
• Type of outcome (qualitative/quantitative, short/long term);
• Time window (annual average or on-time);
• Whether cause specific mortality was considered;
• Hospitalisation;
• All cause and specific cause hospitalization (as mortality);
• Measure applied (attributable/foreseeable fraction, incidence/prevalence, number of attributable deaths).

On the other hand, technical and qualitative variables include all the information describing the quality of website characteristics and the data stored therein. The considered qualitative variables were:

• Design and interactivity of websites;
• Readability;
• Accessibility;
• Completeness/comprehensiveness/coverage/scope and transparency;
• Speed/efficiency.

Methods to assess these qualities are detailed below.

Technical and qualitative variables may be considered closely associated with each other, since the quality of each website contents is strictly related to the type of entered data.

Finally, eligible websites were divided into two categories: 

1. websites providing dose/effect data on the impact of air pollution on health outcomes, such as mortality and hospitalization;

2. websites where air pollution is recognized as detrimental to health, but without a specific dose/effect quantification.

Tools for technical and quality criteria and scores

To identify the main characteristics that influence the quality and functionality of the websites, we constructed a theoretical framework (Figure 1). This diagram illustrates how the technical and qualitative variables act on the information flow from the website providers to the final users. It reports the considered variables, along with the corresponding measurement scales applied. 

Technical and qualitative variables

Design and interactivity. Interactivity in website analysis refers to the degree to which users can actively manipulate and explore the information presented, rather than passively consuming static content. In the context of the websites analysed by this paper, interactive features mainly concern the data displayed (example, year, exposure, type of pollutant, and health outcomes). Higher levels of interactivity allow users to personalize their experience, generate tailored insights, and, therefore, lead to a more effective interaction with complex data sets, improving both understanding and decision-making. In fact, an analysis of the content of public health websites has shown that interactivity is essential for engaging users,¹⁷ especially in the context of widespread health problems. Moreover, the design features such as websites, visual aspects, and layout, and the communicative effectiveness of the visual model were evaluated.

The method applied to evaluate this parameter involved a manual analysis of the level of customisation available to the user in the data visualizations of the reference site, conducted by an UX/UI designer specialized in the field of data visualization (LB). Specifically, the more parameters can be configured, the greater is the potential for interaction. For evaluation purposes, the visual model itself was also examined to highlight its degree of interactivity and ability to adapt to user needs. This approach is reflected in the visual analytics literature, where direct user customization – such as filtering, direct view manipulation, and interactive selection – is considered a key indicator of the interactive effectiveness of a visual system.¹⁸ 

Readability. Readability is the evaluation of the literacy level needed to read the content of the website. It considers the complexity and length of words and sentences, as well as the use of technical terminology that may not facilitate the correct comprehension of the topic. The main text of each site was identified and extracted by one researcher (ED). It was evaluated using the Flesch-Kincaid Grade Level (FKGL) Index, a readability formula that assesses the approximate reading grade level of a text, based on the complexity and length of words and sentences.¹⁹ The index starts from defining the Flesch Reading Ease (FRE) score for the text, which ranges from 0 (unreadable) to 100 (very easy to read); then, this can be translated in the FKGL, which classifies the scores in groups corresponding to years of schooling in the United States. Table 1 reports the FRE score ranges alongside the associated reading-difficulty level and the corresponding FKGL values. The tool applied was Word Stats.

Accessibility. Accessibility refers to compliance with standards such as the Web Content Accessibility Guidelines (WCAG), which ensure usability for people with disabilities and compatibility across multiple devices. A recent review highlighted that many websites dedicated to healthy behaviours vary widely in terms of readability and interactivity, with scores often ranging from poor to adequate. Furthermore, the WCAG framework provides the definitive accessibility standard against which to evaluate such sites.²⁰ In addition, WCAG provides the Web Accessibility Evaluation Tools List: a list of online tools that can be used to measure accessibility and compliance with their guidelines. Among these, www.accessibilitychecker.org has been selected, which provides a result in percentage form.

Completeness/Comprehensiveness/Coverage/Scope and Transparency. According to the theoretical framework (Figure 1), the working group defined the areas of interest to assess the quality of the content presented by each website. 

Websites were considered to be complete since they include as many key elements as possible to ensure a good and comprehensive understanding of the topic. The ability of the websites was evaluated to provide clear and detailed information about key elements such as air pollution, health outcomes, and the impact of air pollution on health. Moreover, transparency is the ability of a website to disclose the source of information and the algorithms used and to make data accessible to users.

Completeness and transparency of each website were analysed with the Soot Score, which measures the coverage of predefined topic areas, weighting some areas as being more important than others.21 Therefore, the proportion of a-priori-defined key elements covered by websites was calculated in a range from 0 (poor) to 100 (excellent).

The topic areas and the related key elements considered were:

• Outcomes: type of outcome, short and long-term effects, mortality and morbidity data, details by specific cause, and extra information (e.g., insights by population type-population effects);
• Pollutants: type of pollutant, how the exposure estimate is reported, data resolution, and extra information (population effects);
• Counterfactual scenario: one or more scenarios are reported;
• Time horizon: on-time data, retrospective data, and projections into the future;
• Metrics applied: how the results are provided;
• Transparency of the website: data source, model applied, how data is presented, possibility to download data.

For each area, a weight was set on the basis of consensus reached among the members of the working group during one of the plenary meetings described elsewhere.²² The selected topic areas and the raw and weighted scores attributed to each are reported in Table 2. Two reviewers (SB and PGR) rated each site for each domain, the average was computed, eventual major discordance were discussed for consensus.

Speed and efficiency. Speed and efficiency refer to the technical performance of a website, including loading times, responsiveness of interactive tools, stability of navigation, and compatibility between devices/browsers. A slow or unstable website not only frustrates users but also diminishes trust and reduces engagement.²³ For example, user experience research shows that delays in page loading can cause stress reactions comparable to those experienced when watching a horror movie, underscoring the importance of fast performance. Furthermore, a marketing case study has shown that improving Core Web Vitals leads to a measurable increase in user engagement and satisfaction.²⁴ 

There are numerous online tools available for this parameter that allow us to measure the speed and efficiency of a website. Among these, PageSpeed Insight was chosen, a tool developed by Google for Developers which provides a parameter called “Performance” in its website analysis report, a rating in cents that is useful to evaluate efficiency. Real user experience data in PSI is based on the Chrome User Experience Report (CrUX) dataset. PSI records real user experiences related to First Contentful Paint (FCP), Interaction to Next Paint (INP), Largest Contentful Paint (LCP), and Cumulative Layout Shift (CLS) over the previous 28-day collection period. PSI also reports experiences for the experimental metric Time to First Byte (TTFB).²⁵

Results

In total, 1,000 websites were screened and 15 were identified as potentially relevant. The in-depth analysis led to the exclusion of 7 of these, because not fulfilling inclusion and exclusion criteria; 8 were included in the review. The list of the seven excluded websites and the reasons for their exclusion are reported in Figure S1 (see online Supplementary Materials). All the eight included web sites were identified by the first engine search (Google); no web site was identified by only one search engine.

Description of the websites

The assessment of the website’s descriptive characteristics was based on the judgment of the data extractor through a scale including yes, no, not applicable. A list of specific information provided by the websites was included for all the descriptive characteristics, and it is shown in Table 3 and Table 4. 

Three out of the eight websites are operated by governmental agencies, three by academic institutions, with public funding, and two by private foundations, one based in US, the Health Effects Institute, that receives funds from both public and private energy industry, and one based in China, the ‘World Air Quality Index’ project, that is funded by donations and advertising. All the websites provide geographical coverage at the global level, except the European Environmental Agency and the UK website that cover Europe and UK, respectively. All the websites included in the analysis were launched in the 2000s, and most provide data updated within the 2019-2022 time range. An exception is the website managed by the ‘World Air Quality Index’ project, which provides real-time data with daily updates since 2014 (Table 3).

Most of the websites are targeted at both decision/policy makers and the general public, with the primary aim of informing decision-makers and the public about the state of environment, climate change, and wider sustainability issues. However, the language used is sometimes relatively complex and may not always be fully accessible to a non-expert audience. By contrast, the ‘World Air Quality Index’ project website explicitly aims to foster awareness of air pollution by providing unified, global, and real-time air quality information using language that is accessible to lay users; accordingly, it is primarily designed for public consultation by citizens. Two websites differ in terms of their intended end users. The ‘Our World in Data’ platform is mainly addressed to the academic community, with the mission of producing academic research on the world’s largest problems based on the empirical analysis of global data. In contrast, the Global Burden of Disease (GBD) website primarily targets policy makers and experts in the sector, with the aim of delivering timely, relevant, and scientifically robust evidence to inform and improve health policy and practice (Table 3).

Furthermore, the content of each website was examined in terms of exposure and health outcomes data (Table 4). The analysis showed that all websites report exposure data on particulate matter (PM_2.5 and/or PM₁₀), while only some provide information on NO₂ and O₃. Except the ‘World Air Quality Index’ project, which displays real-time data based on daily pollutant levels, maps generally present pollutant concentrations derived from annual averages of daily measurements.

Considering health outcomes, six out of eight websites include quantitative dose-response information on the health effects of air pollution. In the remaining two websites, air pollution is acknowledged as detrimental to health, although without providing explicit dose-response estimates. Notably, the platform developed by the Medical Research Council and Public Health England reports data on the overall incidence of air-pollution-related impacts across 14 diseases, whereas the ‘World Air Quality Index’ project website represents air-quality levels using emoticons as qualitative indicators of the short-term impact on health. All the other websites present quantitative data mostly focusing on to long-term effects. Most platforms report all-cause mortality, while only two, The Global Health Observatory and the Global Burden of Disease sites, provide also cause-specific mortality data. No website provides information regarding hospitalizations, even if the Global Burden of Disease report disability adjusted life years (DALYs) lost due to pollution, also divided by the year of life lost (YLL) to early mortality and those lost due to disability (YLD); this second measure of morbidity is based on several sources of data, including hospitalisation.

Technical and qualitative variables

A narrative synthesis of the website technical and qualitative variables is reported in Table 5 and results are presented according to the type of outcomes reported by the website.

Design and Interactivity. The analysis of the selected websites revealed wide variability in the degree of interactivity and complexity of the visual models adopted, with approaches ranging from the consultation of static data to the dynamic manipulation of complex datasets.

The websites have levels of interactivity that can be classified into three main categories:

1. High interactivity: platforms such as ‘State of Global Air’ and ‘Our World in Data’ offer users a wide range of configurable parameters, including dynamic selection of pollutants, metrics, geographic areas, and time intervals. In particular, ‘Our World in Data’ takes a chart-centred approach, allowing for advanced customization of individual charts and direct data downloads. The World Air Quality Index was also rated positively for its operational nature, allowing for zooming and immediate interaction with individual monitoring stations;

2. Medium interactivity: the EEA website and the Air Quality Life Index show an intermediate level, featuring zoom, pan, and country or station selection functions. The Air Quality Life Index, in particular, facilitates guided data exploration without reaching the complexity of professional analysis systems.

3. Low interactivity: the WHO portals and the Institute for Health Metrics and Evaluation (IHME) interface were found to be more oriented towards direct consultation and citation of official data than dynamic visual exploration, with filters limited to country or year.

During the analysis of platform interactivity, the need to evaluate usability for a diverse audience, not limited to experts in the sector, also became prominent. From this second analysis, two prevalent models emerged: the narrative and journalistic model, applied by sites such as AQLI, ‘Air Pollution and Health in Cities’, and ‘Our World in Data’, which use visual language similar to that of an interactive newspaper article, where the text guides the user in understanding the visualizations. These were rated as highly user-friendly. In contrast, the EEA’s ‘Air Pollution Atlas’ and the ‘State of Global Air’ platform used a technical and analytical model that features layered interfaces (GIS) or information-dense control panels that, while powerful for a technical user, are complex and less fluid for quick consultation or mobile devices.

To complete the analysis, a descriptive analysis was carried out on the chosen visual models (Table 5).

Readability. The readability assessment showed that all the websites analysed have a Flesch Reading Ease (FRE) score ranging from 35 to 50, indicating a difficult reading level. The Flesch-Kincaid Grade Level (FKGL) values fall between 11 and 16, corresponding to an educational level equivalent to college or university in the United States. The WHO website has the highest degree of complexity, with an FRE score of 26.2, which corresponds to a postgraduate reading level. For the World Air Quality Index, it was not possible to perform the readability assessment, because, apart from a brief introductory description of the map, no descriptive text was available for analysis; nevertheless, the maps can be considered self-explaining giving high readability to the website.

These findings indicate that seven out of eight examined websites feature substantial linguistic complexity and extensive use of technical terminology, making them more suitable for an expert audience than for the general public.

Accessibility. The assessment of website accessibility showed that the platforms with the best performance are those of the WHO, the IHME, and the World Air Quality Index project, each achieving accessibility levels above 85%. The highest score was obtained by the website of the EEA, which reaches an accessibility level of 100%.

Most of the other websites evaluated have accessibility levels ranging from 70% to 85%. A notable exception is the ‘Environment and Health Atlas for England and Wales’, which reports a substantially lower level of 57%.

Completeness/Comprehensiveness/Coverage/Scope and Transparency. The website of the EEA achieved the highest overall score (77.5), standing out for its extensive and detailed information on both exposure metrics and health outcomes. Data are clearly presented and cover an adequate temporal horizon; however, the availability of counterfactual scenarios is limited to the present and there are no projections for the future. Moreover, despite the breadth of information provided, maps and data sources are not immediately accessible.

The websites The Global Health Observatory, Air Pollution, Global Burden of Disease, State of Global Air, and Air Quality Life Index (from 2 to 6) obtained a soot score above 65, indicating generally high levels of transparency. In these cases, data on health outcomes and pollutants are clearly presented, with raw scores ranging from 7 to 9, and include information on multiple exposure types and causes of mortality. Nevertheless, the temporal coverage is generally very limited. Marked discrepancies are also observed in the definition of counterfactual scenarios, which are scarce and predominantly static for the websites of the World Health Organization, the Health Effects Institute (HEI), and the Institute at the University of Chicago (EPIC).

Finally, the websites of the Medical Research Council and Public Health England (number 7) and the World Air Quality Index project (number 8) provide a more limited set of information, as they focus on describing the association between air pollution and health outcomes without explicitly quantifying dose-response relationships. In particular, the World Air Quality Index project website presents synthetic air quality data linked to a general health index, without reporting specific mortality outcomes or counterfactual scenarios. Despite these limitations, the website demonstrates a good level of methodological transparency and clear traceability of data sources. By contrast, the UK website received the lowest score (56); although transparency is high, the causal link between exposures and health outcomes is not clearly articulated and the choice of the outcomes reported in the atlas is not explicitly justified.

Speed/Efficiency. The analysis of website speed and efficiency showed that the most responsive platforms were those of the HEI, the IHME, the GBD, and the EEA, all of which achieved performance scores above 70. The highest score was observed for the Small Area Health Statistics Unit (SAHSU) website, with a value of 88. However, this result should be interpreted with caution, as it is likely influenced by the limited amount of content and the low level of interactivity on the site, factors that contribute to faster page-loading times.

In contrast, the EPIC and the WHO websites have the lowest performance, both exhibiting scores below 40.

Discussion

Eight websites were analysed, most of which provide global-wide geographical coverage and include maps or infographics offering comprehensive information that relates air pollution and health outcomes. These platforms share the common aim of monitoring air pollution exposure levels and illustrating the associated health impacts and effects through interactive and graphical interfaces. Such tools allow users to personalize their experience, thereby facilitating a more effective understanding of the topic, particularly in the context of widespread public health challenges.  

The present analysis identified several examples of strong coherence between the objectives stated in the websites and the data presented. Notably, the platforms developed by the Institute for Health Metrics and Evaluation and the World Air Quality Index project demonstrated effective use of graphical representations tailored to their intended audiences. Differences in target users were reflected in the level of detail and complexity of the information provided, distinguishing resources designed for expert or academic audiences from those aimed at the general public. 

The Global Burden of Disease platform, for instance, offers highly detailed maps and a wide range of selectable indicators, highlighting its relevance for technically oriented users who do not require extensive explanatory guidance. On the other hand, the World Air Quality Index project targets a broader audience and therefore prioritizes simplified communication strategies, such as the use of emoticons and intuitive infographics, to facilitate immediate understanding among non-expert users. These findings prompt a broader reflection on the importance of effectively communicating complex messages, such as the health impacts of air pollution, while maintaining a scientific approach and making the topic accessible to all. Simplifying complex information is essential not only to support informed decision-making among policymakers, but also to enhance public understanding of environmental health risks and their associated impacts.

However, this analysis also identified some cases in which the alignment between stated aims and data presentation was not completely achieved. The website by the Medical Research Council and Public Health England, conceived as a resource for the public, researchers, and policymakers, was developed to support understanding of the geographic distribution of environmental agents and health conditions in England.²⁶ The website proposed several outcomes and several exposures without any explicit statement on the strength and certainty of the scientific evidence of association between them. This may be a stimulus for researchers, but it does not allow the non-technical users, both the public and policymakers, to draw any conclusion. It is worth reporting that, at its first appearance on the web, the UK Atlas caused several controversial reactions, with both alarm in the media and the general public and criticisms by the institutions.^27-30 This website exemplifies the inherent challenges of addressing multiple audiences with differing levels of expertise within a single digital tool. 

From the present analysis, it was easy to assess the level of involvement of technicians and the scientific community in the development of the website, since the scientific collaborations with single experts or academic institutions are clearly recognized when the publisher is not a scientific institution itself. On the contrary, it was impossible to assess whether the development of the website also involved the final users, through consultation or more active participation in a co-creative process. 

It is crucial to achieve a balance between scientific rigour and communicative clarity, to ensure that a relevant topic is accurately understood by its intended audience. Engaging citizens in a co-creative process may help in setting reasonable and relevant communication aims for a multi-target tool. Users’ involvement could also enhance content accessibility by promoting the use of clearer language and improving the presentation and interpretability of the maps and data, and support the development of more effective communication strategies without compromising scientific accuracy. 

Limitations of the study

One limitation should be acknowledged in relation to the present analysis. Difficulties were encountered in identifying standardised methods for data search and extraction within the scientific literature,¹⁶ which limited the ability to adopt a fully standardized extraction framework tailored to the specific needs of this analysis. In particular, no validated or universally accepted scales currently exist for assessing the interactivity, completeness, comprehensiveness, and transparency of ‘air and health atlases’ or comparable web-based public health communication tools. Given the heterogeneity of these platforms in terms of structure, objectives, target audience, and communication strategies, it is likely that a universally applicable evaluation scale may never be feasible. Rather, assessment tools necessarily need to be developed ad hoc according to the specific objectives, scope, and intended use of each review. Aware of this limitation, the working group was intentionally interdisciplinary, including experts in web design, infographics, and communication, as well as experts in environment, pollution, and epidemiology. This multidisciplinary approach was adopted to improve the robustness and interpretability of the evaluation process despite the absence of standardized frameworks. The group opted for including a large number of indicators, some of them well established and validated, some created ad hoc for this review, privileging the comprehensiveness and sacrificing the synthesis, accepting some overlapping of the domains being evaluated. 

Using generic search engines does not allow the researchers to set a reproducible method. An indirect indicator of the search sensitivity is the saturation that in this case was evident, with no website identified by only one search engine. Nevertheless, search engines use algorithm that put in evidence the most popular website and algorithms that match the search results to the interests of the user based on previous searches. These algorithms may limit the sensitivity of the searches, favouring the identification of the most popular sites. Including different engines should limit the risk; furthermore, engines that were never used before by the researchers were included, thus the results should not be biased by previous searches.

Another limitation of this study is that the use of artificial intelligence tools was not consider for web searching, some of which are specifically designed for this type of task. However, current search engine queries are increasingly accompanied by AI-generated results and summaries, reducing the need to perform direct searches on individual AI tools. Nevertheless, form a post hoc check, none of the most used AI tools – i.e., Gemini and Chat GPT – found further relevant web sites.  

This review was conducted with the primary scope of informing the Air and Health Atlas project, which is aimed at developing a national interactive atlas relating air quality data to health outcomes, designed to be understandable and accessible to both professionals and lay users. Consequently, the selection of websites was driven by predefined characteristics aligned with this aim, and the analyses focused on technical and qualitative variables considered potentially relevant for inclusion in the Atlas. The assessment of completeness and transparency, as well as the evaluation of included metrics, was therefore guided by this perspective. In particular, the fraction of attributable deaths to air pollution represents a central focus of the project, making the graphical representation of this parameter a key criterion in the project evaluation of the content completeness. Furthermore, although the assessment of future scenarios is essential for the intended functionality of the Atlas, such features were largely absent among the analysed platforms.

Conclusions

In conclusion, the review of existing websites presenting data on the health impacts of air pollution highlights considerable variability in content and presentation, depending on whether the target audience was composed of experts/academics or the general public. The findings underscore the importance of challenges of producing a multi-target tool able to communicate to experts, policymakers, and the general public, maintaining clarity and scientific rigour. At the same time, it appears difficult to export this method to other types of research, since the aims and functions of a portal may vary considerably at a subjective level, making any broad generalization only partially effective. Nevertheless, the Air and Health Atlas can be trustworthy and transparent only if it is accessible and clear for all three targets. The experiences reported in this review suggest that it is possible to convey the message also to non-experts. They also indicate that focusing on a few outcomes and exposures, providing numbers of the most certain effects on health, may help reach this goal. A schematic representation of key insights derived from this review is provided in Table 6.

Involving non-expert final users in defining the information needs and solutions will make the Atlas capable of effectively communicating to multiple audiences with varying levels of expertise.

Conflicts of interest: none declared.

Funding: project implemented with the technical and financial support of the Ministry of Health – PNC PREV-A-2022-12376.

References

1. Sierra-Vargas MP, Teran LM. Air pollution: impact and prevention. Respirology 2012;17(7):1031-38. doi: 10.1111/j.1440-1843.2012.02213.x
2. Schraufnagel DE, Balmes JR, De Matteis S, et al. Health Benefits of Air Pollution Reduction. Ann Am Thorac Soc 2019;16(12):1478-87. doi: 10.1513/AnnalsATS.201907-538CME
3. Cohen AJ, Brauer M, Burnett R, et al. Estimates and 25-year trends of the global burden of disease attributable to ambient air pollution: an analysis of data from the Global Burden of Diseases Study 2015. Lancet 2017;389(10082):1907-18. doi: 10.1016/S0140-6736(17)30505-6
4. Manisalidis I, Stavropoulou E, Stavropoulos A, Bezirtzoglou E. Environmental and Health Impacts of Air Pollution: A Review. Front Public Health 2020;8:14. doi: 10.3389/fpubh.2020.00014
5. European Environment Agency. How air pollution affects our health, Copenhagen: European Environment Agency; 2025. Available from: https://www.eea.europa.eu/en/topics/in-depth/air-pollution/eow-it-affects-our-health (last accessed: 02.02.2026).
6. Baker L, Wagner TH, Singer S, Bundorf MK. Use of the Internet and e-mail for health care information: results from a national survey. JAMA 2003;289(18):2400-6. doi: 10.1001/jama.289.18.2400
7. Karamitros GA, Kitsos NA, Sapountzis S. Systematic Review of Quality of Patient Information on Phalloplasty in the Internet. Aesthetic Plast Surg 2017;41(6):1426-34. doi: 10.1007/s00266-017-0937-5
8. McLeod SD. The quality of medical information on the Internet. A new public health concern. Arch Ophthalmol 1998;116(12):1663-65. doi: 10.1001/archopht.116.12.1663
9. Institute for Health Metrics and Evaluation. Global Burden of Disease (GBD) publications. Available from: https://www.healthdata.org/research-analysis/gbd-publications (last accessed: 02.02.2026).
10. World Health Organization. The Global Health Observatory. Explore a world of health data. Geneva: World Health Organization; 2026. Available from: https://www.who.int/data/gho (last accessed: 02.02.2026).
11. Fuller R, Landrigan PJ, Balakrishnan K, et al. Pollution and health: a progress update. Lancet Planet Health 2022;6(6):e535-47. doi: 10.1016/S2542-5196(22)00090-0
12. Morgan RL, Whaley P, Thayer KA, Schunemann HJ. Identifying the PECO: A framework for formulating good questions to explore the association of environmental and other exposures with health outcomes. Environ Int 2018;121(Pt 1):1027-31. doi: 10.1016/j.envint.2018.07.015
13. Ranzi A, Pasetto R, Cordioli M. I modelli per la valutazione dell’esposizione. Ecoscienza 2014;1(1):12-3.
14. Ademiluyi G, Rees CE, Sheard CE. Evaluating the reliability and validity of three tools to assess the quality of health information on the Internet. Patient Educ Couns 2003;50(2):151-55. doi: 10.1016/s0738-3991(02)00124-6
15. StatCounter Global Stats. Desktop & tablet console search engine market share Worldwide. StatCounter Global Stats 2017. Available from: https://gs.statcounter.com/search-engine-market-share/desktop-tabletconsole/worldwide#monthly-201611-201702 (last accessed: 10.12.2024).
16. Eysenbach G, Powell J, Kuss O, Sa ER. Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. JAMA 2002;287(20):2691-700. doi: 10.1001/jama.287.20.2691
17. Banna S, Hasan H, Meloche J. Opportunities for interactivity in public health websites: A content analysis approach. European and Mediterranean Conference on Information Systems 2009 (EMCIS2009); Izmir (Turkey), 13-14 July 2009.
18. Heer J, Shneiderman B. Interactive Dynamics for Visual Analysis. Available from: https://homes.cs.washington.edu/~jheer/files/idfva-draft.pdf (last accessed: 02.02.2026).
19. Jindal P, MacDermid JC. Assessing reading levels of health information: uses and limitations of flesch formula. Educ Health (Abingdon) 2017;30(1):84-88. doi: 10.4103/1357-6283.210517
20. International Web Association Italia. WCAG: Web Content Accessibility Guidelines. Venice (Italy); IWA Italy. Available from: https://wcag.it/ (last accessed: 02.02.2026).
21. Soot LC, Moneta GL, Edwards JM. Vascular surgery and the Internet: a poor source of patient-oriented information. J Vasc Surg 1999;30(1):84-91. doi: 10.1016/s0741-5214(99)70179-5
22. Davoli E, Bartolomei M, Ottolenghi M, et al. The co-creative process of the Air and Health Atlas project: stakeholder and user involvement in the definition of information and communication needs. Epidemiol Prev 2026;50(4-5) Suppl 1:45-58. doi: 10.19191/EP26.4-5.S1.A1027.091
23. Chrome Developers. Why speed matters: Chrome Developers. 2023. Available from: https://web.dev/learn/performance/why-speed-matters?utm_source=chatgpt.com&hl=it (last accessed: 02.02.2026).
24. Fellinger L, Fronimaki D. Enhancing user experience: Unveiling the impact of website speed optimisation on user engagement. J Digit Soc Media Mark 2024;11(4):324-38.
25. Google Developers. About PageSpeed Insights. Google Developers 2025. Available from: https://developers.google.com/speed/docs/insights/v5/about?hl=it (last accessed: 02.02.2026).
26. Hodgson S, Fecht D, Gulliver J, et al. Availability, access, analysis and dissemination of small-area data. Int J Epidemiol 2020;49 Suppl 1(Suppl 1):i4-14. doi: 10.1093/ije/dyz051
27. Science Media Centre. Expert reaction to the Environment and Health Atlas for England and Wales. London: Science Media Centre; 2026. Available from: https://www.sciencemediacentre.org/expert-reaction-to-the-environment-and-health-atlas-for-england-and-wales/ (last accessed: 02.02.2026).
28. Hawkes N. Atlas aims to show possible environmental effects on UK health. BMJ 2014;348:g2948. doi: 10.1136/bmj.g2948
29. Cancer Research UK. Mapping cancer risks – what can (and can’t) it tell us? Cancer Research UK 2014. Available from: https://news.cancerresearchuk.org/2014/04/25/mapping-cancer-risks/ (last accessed: 02.02.2026).
30. Lynch P. The Environment and Health Atlas for England and Wales. KnowledgeHub 2023. Available from: https://khub.net/web/p-lynch/user-blogs/-/blogs/the-environment-and-health-atlas-for-england-and-wales (last accessed: 02.02.2026).