Skip to main content
  • Brief communication
  • Open access
  • Published:

Oral administration of Lactobacillus paracasei NCC 2461 for the modulation of grass pollen allergic rhinitis: a randomized, placebo-controlled study during the pollen season

Abstract

Background

The efficacy of Lactobacillus paracasei NCC 2461 in modulating allergic rhinitis was previously evaluated in two exploratory clinical studies. Oral administration with NCC 2461 reduced specific subjective symptoms following nasal provocation tests with controlled grass pollen allergen concentrations. Our aim was to confirm the anti-allergic effect of NCC 2461 in grass pollen allergic subjects exposed to natural doses of allergens during the pollen season.

Findings

A double-blind, randomized, placebo-controlled, parallel study was conducted with 131 grass pollen allergic subjects from May to July 2012 in concomitance with the pollen season in Berlin. NCC 2461 or placebo was administered daily for an 8-week period to adult subjects with clinical history of allergic rhinitis to grass pollen, positive skin prick test and IgE to grass pollen. During the 8 weeks, symptoms and quality of life questionnaires were filled out, and plasma was collected for IgE analysis at screening and at the end of the intervention. All subjects were included within a 5-day interval, ensuring exposure to similar air pollen counts for each individual during the trial period. The results obtained show that symptoms increased with pollen loads, confirming a natural exposure to the allergen and presence of pollen-induced allergic rhinitis in the subjects. However, no significant differences were observed in allergic rhinitis symptoms scores, quality of life, or specific IgE levels between subjects receiving NCC 2461 as compared to placebo administration.

Conclusion

In contrast to previous findings, oral administration of NCC 2461 did not show a beneficial effect on allergic rhinitis in a field trial. The influence of study design, allergen exposure and intervention window on the efficacy of NCC 2461 in modulating respiratory allergy should be further evaluated.

Findings

Probiotics and allergic rhinitis

Allergic rhinitis to grass pollen affects around 20 % of the population in Europe, USA and Oceania and is characterized by nasal and ocular symptoms, such as nasal congestion or watery eyes, which significantly impact the quality of life of allergic subjects [13]. In addition to standard medications, such as anti-histamines and corticosteroids, aiming at the management of allergic rhinitis symptoms, nutritional interventions with probiotics have been explored in clinical trials [4, 5]. Oral intake of probiotics, in particular lactobacilli and bifidobacteria, has been shown to modulate allergic immune responses as well as to have a significant impact on symptoms reduction [611]. Administration of Lactobacillus paracasei NCC 2461 significantly decreased subjective nasal congestions in subjects with a medical history of allergic rhinitis to grass pollen, as evaluated following a nasal provocation test (NPT) in a proof-of-concept study performed outside of the pollen season [12]. In a second study, NCC 2461 improved NPT-induced nasal pruritus in grass pollen allergic subjects when compared to a different probiotic blend; the effect observed was comparable to the symptoms reduction obtained with an anti-allergic drug tested with the same NPT protocol [13]. In line with these results, we aimed to further demonstrate the efficacy of NCC 2461 in a clinical trial performed during the grass pollen season, in which subjects are exposed to natural doses of allergens.

Study design and outcomes

The study was designed as a randomized, double blind, placebo-controlled, parallel trial, and was performed between May and July 2012 at Charité Research Organisation GmBh in Berlin (ClinicalTrials.gov: NCT01653652). One hundred and thirty-one volunteers were enrolled on the basis of the following inclusion criteria: 18–65 years of age, body mass index in the range of 19–32 kg/m2, established allergic rhinitis to grass pollen for more than 2 years, positive skin prick test to grass pollen and grass pollen-specific IgE (≥0.35 kU/L). Exclusion criteria comprised pregnancy, infection or antibiotic intake at time of enrollment, asthma and other chronic disorders, and the use of systemic corticosteroids prior to randomization. Enrollment was completed within 5 days to ensure that all subjects were exposed to natural airborne pollen during the same period of the pollen season. The treatment consisted of daily oral intake of 5 g of placebo (maltodextrin) or a probiotic blend containing 5 × 109 CFU NCC 2461 over a period of 8 weeks. Placebo and probiotics were packed in powder form in single sachets for each daily dose. Study design and CONSORT flow diagram are outlined in Additional file 1: Figure S1.

The primary objective of the study was to evaluate the efficacy of NCC 2461 in mitigating grass pollen-induced allergic rhinitis symptoms by comparing daily total nasal symptom score (TNSS) over 8 weeks (from V1–V3) between the probiotic and the placebo group. TNSS as well as total ocular symptom score (TOSS, both described in Additional file 1: Questionnaires and plasma analysis) were recorded daily by the subjects with the help of an electronic patient reported outcome (ePRO) device. On a weekly basis, subjects were asked to fill a validated quality of life questionnaire specific for allergic rhinitis, the mini Rhinoconjuctivitis Quality of Life Questionnaire (miniRQLQ) [14]. Subjects were allowed to take any anti-allergic medication at their discretion, and the type and dose of each drug was recorded. Decrease in medication intake through probiotic intervention was a secondary objective of the study, and the outcome measure was the weekly rescue medication score questionnaire (Additional file 1: Questionnaires and plasma analysis). Of note, TNSS was adjusted for medication score at V1, to avoid possible biases on symptoms of the basal status of the subjects.

To evaluate the effect of probiotic administration on grass pollen-specific IgE, serum was collected at screening and at V3 and results compared between NCC 2461 and placebo group.

The primary statistical analysis was carried out on the change from baseline TNSS scores using a Linear Mixed Model, where treatment group, day of treatment and their interaction were considered as fixed effects and with subject number as a random effect to control for within subject variability. Secondary analyses were done in a similar fashion as the primary analysis. A summary of inclusion and exclusion criteria, primary and secondary outcomes, as well as methods can be found in the Additional file 1.

Efficacy of L. paracasei NCC 2461

Baseline demographics are shown in Additional file 1: Table S1. Sixty-eight subjects were assigned to the placebo group, sixty-three to the NCC 2461 group, and all one hundred and thirty-one subjects were included in the intention-to-treat (ITT) analysis. Of note, results of the per-protocol analysis did not differ from ITT. One subject receiving placebo withdrew from the study due to a bacterial infection unrelated to product intake.

Analysis of grass pollen concentrations in Berlin during the 2012 season indicated that at enrollment, beginning of May, pollen counts were still low (Fig. 1a). Further analysis highlighted a positive correlation between pollen counts and the primary outcome TNSS confirming the presence of grass pollen-induced allergic rhinitis manifestations in line with the clinical history of the study subjects. The maximum values for both pollen and symptom measures were observed 20 days after the start of the trial. Treatment with NCC 2461 however did not have an effect on symptoms, as no significant differences in TNSS were observed between probiotic and placebo at any time point (Fig. 1b). Individual symptoms scores were also comparable between groups (Additional file 1: Figure S2). In line with these results, NCC 2461 administration did not show an effect on quality of life over the 8-week period (Fig. 2), and grass pollen-specific IgE in plasma did not differ between placebo and probiotic group at both time points assessed (Additional file 1: Figure S3). Total and individual ocular symptoms were also comparable (data not shown). Subjects were allowed to take anti-allergic medication whenever they felt the need; the intake was not influenced by the probiotics, as no statistically significant differences in frequencies were observed when compared to placebo at any of the time points (Table 1).

Fig. 1
figure 1

TNSS is comparable between placebo and NCC 2461 administration. a The correlation between daily grass pollen counts during the trial period and daily TNSS values is represented; both parameters reached maximum values in the last week of May, 20 days after the start of the trial, and no significant differences were observed between the study groups. The solid and broken curves represent Loess local smoothing method (with 95 % level of confidence) applied to TNSS scores of placebo and NCC 2461 groups, respectively, in order to summarize daily symptom scores. The volatile line refers to daily pollen counts with corresponding levels (referring to the right vertical axis). Pollen data were provided by the Geowissenschaften Institut für Meteorologie of the Freie Universität in Berlin. b Mean TNSS score for each week of each subject was calculated; boxplots represent the weekly mean for the two study groups

Fig. 2
figure 2

NCC 2461 administration does not impact quality of life. Similar to TNSS, no statistically significant differences were observed between probiotic and placebo group at any time point. Boxplots for weekly mean miniRQLQ are shown

Table 1 Medication score comparison between NCC 2461 and placebo group

Probiotic administration was considered safe as no serious adverse event was recorded during the trial. Subjects receiving NCC 2461 reported fewer episodes of nasopharyngitis, however the frequency of minor adverse events was not a trial outcome and was therefore not considered in the statistical analysis.

Discussion

Prior to this study, the effect of NCC 2461 in modulating grass pollen allergic rhinitis was evaluated in two exploratory studies, outside the pollen season in nasal provocation test settings [12, 13]. Results of these trials suggested a role of NCC 2461 in immune regulation, as cytokine expression by peripheral blood mononuclear cells cultures [12] or in nasal secretions [13] was significantly different when compared to a placebo [12] or a different probiotic blend [13]. In addition, both studies showed that NCC 2461 administration might affect distinct subjective symptoms of allergic rhinitis, namely congestion [12] and pruritus [13]. In contrast, the results from the study presented in this manuscript indicate no effect of NCC 2461 in modulating TNSS, TOSS, nor any of the symptoms considered individually. The discrepancy in outcomes between the different studies using NCC 2461 might be explained by the difference in study design between the previous trials and the current one, and the different approaches taken to expose subjects to the allergen in order to induce allergic rhinitis symptoms.

Wassenberg et al. [12] as well as Perrin et al. [13] evaluated the effect of NCC 2461 administration on symptoms’ reduction following a NPT in which all subjects were exposed to the same increasing doses of grass pollen at each challenge; both trials were performed outside of the pollen season. In our current in-season trial, exposure of each subject to grass pollen occurred in a natural and obviously non-controlled manner, and allergen doses might have varied considerably over time as well as within the whole study population. In addition to pollen counts, the country in which the study was performed, as well as other environmental factors such as climate and air pollution could also influence the results of a trial conducted during the pollen season [15, 16].

The difference in outcome measures between studies should also be considered; in the previous trials subjective symptoms (pruritus and nasal congestion) were measured with visual analogue scales, and immunological parameters such as cytokine secretion and IgG4 were not measured in the study performed in Berlin. Also, the dose we used here (5 × 109 CFU/day) was slightly lower than in previous NCC 2461 studies due to the different format used; however, it was still in the range of common doses of probiotics tested in clinical trials [6, 9, 17, 18].

Another important variable to take into account is the use of anti-allergic medication by the subjects. Intake was recorded and compared between probiotic and placebo group through the rescue medication score questionnaire; no significant differences could be observed. On the other hand, there was no adjustment of the primary endpoint TNSS for covariates such as anti-allergic medication intake at any time point except for V1, to correct for basal level. For this reason we cannot exclude that the single consumption of anti-histamines or corticosteroids by the subjects had an influence on the reporting of their symptoms. In this case, a primary outcome comprising both symptoms and medication intake score results could have been a relevant approach [19]. Moreover, allergic rhinitis to other aeroallergens such as house dust mite or pet dander was not an exclusion criterion, as the majority of subjects suffering from allergic airway diseases are polysensitized; thus, reactions other than to grass pollen could have had an impact on TNSS and medication score as well.

Our objective was to evaluate the role of NCC 2461 in conditions as near as possible to real-life, namely with natural exposure to allergens and with medication intake at subject’s discretion. It can be speculated however that the effect of the intervention is more difficult to detect in these settings. Of note, in three clinical trials in which probiotic administration was associated to the controlled intake of anti-allergic drugs such as loratadine or levocetirizine, a significant impact on symptoms reduction or quality of life improvement on top of the medication was observed [6, 18, 20]. The trials described in Costa et al. and Lue et al. were performed in subjects with moderate-to-severe rhinitis according to ARIA classification; significant clinical differences might be better detectable in this population as compared to mild allergic rhinitis [21, 22]. In our trial, we did not differentiate subjects on the basis of ARIA classification.

Importantly, in contrast to our earlier NCC 2461 trials, in which probiotics were administered for 4 weeks prior to pollen challenge by NPT [12, 13], probiotic intake in the study presented here occurred at the same time as the exposure to the allergen. In fact in most of previously published studies in which probiotics had a positive impact on allergic rhinitis to pollen (grass, birch or Japanese cedar), the intervention started weeks before season start or nasal challenge [1113, 17, 2225]. On the other hand, in the only other reported trial that had a similar experimental design as the current one, i.e., parallel, 8-week long, with similar outcomes and most importantly run at the peak of the pollen season, we could show that administration of Bifidobacterium lactis NCC 2818 did significantly reduce TNSS in allergic subjects [9]. The mechanisms of action of probiotics, including NCC 2461, are not entirely elucidated and can vary according to the strain used; the same is true for the appropriate doses and intervention windows needed to observe a beneficial effect [21, 22, 2631]. It can be speculated that some probiotics have a higher impact on allergic responses by priming the immune system prior to allergen challenge; this could be the case of NCC 2461. Different strains might conversely be more effective in symptoms management when the allergy reaction is already ongoing. The immune-regulatory properties and mechanisms of action of specific probiotic strains are thus worth being carefully investigated; study design and intervention window of future efficacy trials should also be aligned accordingly.

Finally, we believe more evidence is needed in closer-to-real-life, field-trials settings to conclude on the most appropriate probiotic intervention. Taken together however, our NCC 2461 trials and other previously published observations suggest that starting probiotic intake before the pollen season as well as administration together with a controlled medication regimen may be the preferable strategy for the management of seasonal allergic manifestations.

Abbreviations

ARIA:

allergic rhinitis impact on asthma

CFU:

colony forming unit

ePRO:

electronic patient reported outcome

ITT:

intent-to-treat

NCC:

Nestlé culture collection

NPT:

nasal provocation test

RQLQ:

rhinoconjunctivitis quality of life questionnaire

TNSS:

total nasal symptom score

TOSS:

total ocular symptom score

References

  1. Bousquet PJ, Chinn S, Janson C, Kogevinas M, Burney P, Jarvis D. Geographical variation in the prevalence of positive skin tests to environmental aeroallergens in the European Community Respiratory Health Survey I. Allergy. 2007;62:301–9.

    Article  PubMed  Google Scholar 

  2. Greiner AN, Hellings PW, Rotiroti G, Scadding GK. Allergic rhinitis. Lancet. 2011;378:2112–22.

    Article  PubMed  Google Scholar 

  3. Meltzer EO, Blaiss MS, Derebery MJ, Mahr TA, Gordon BR, Sheth KK, et al. Burden of allergic rhinitis: results from the pediatric allergies in America survey. J Allergy Clin Immunol. 2009;124:S43–70.

    Article  PubMed  Google Scholar 

  4. Clark MJ, Million RP. Allergic rhinitis: market evolution. Nat Rev Drug Discov. 2009;8:271–2.

    Article  CAS  PubMed  Google Scholar 

  5. Kalliomaki M, Antoine JM, Herz U, Rijkers GT, Wells JM, Mercenier A. Guidance for substantiating the evidence for beneficial effects of probiotics: prevention and management of allergic diseases by probiotics. J Nutr. 2010;140:713S–21S.

    Article  CAS  PubMed  Google Scholar 

  6. Costa DJ, Marteau P, Amouyal M, Poulsen LK, Hamelmann E, Cazaubiel M, et al. Efficacy and safety of the probiotic Lactobacillus paracasei LP-33 in allergic rhinitis: a double-blind, randomized, placebo-controlled trial (GA2LEN Study). Eur J Clin Nutr. 2014;68:602–7.

    Article  CAS  PubMed  Google Scholar 

  7. Ishida Y, Nakamura F, Kanzato H, Sawada D, Hirata H, Nishimura A, et al. Clinical effects of Lactobacillus acidophilus strain L-92 on perennial allergic rhinitis: a double-blind, placebo-controlled study. J Dairy Sci. 2005;88:527–33.

    Article  CAS  PubMed  Google Scholar 

  8. Ivory K, Wilson AM, Sankaran P, Westwood M, McCarville J, Brockwell C, et al. Oral delivery of a probiotic induced changes at the nasal mucosa of seasonal allergic rhinitis subjects after local allergen challenge: a randomised clinical trial. PLoS One. 2013;8:e78650.

    Article  PubMed Central  PubMed  Google Scholar 

  9. Singh A, Hacini-Rachinel F, Gosoniu ML, Bourdeau T, Holvoet S, Doucet-Ladeveze R, et al. Immune-modulatory effect of probiotic Bifidobacterium lactis NCC2818 in individuals suffering from seasonal allergic rhinitis to grass pollen: an exploratory, randomized, placebo-controlled clinical trial. Eur J Clin Nutr. 2013;67:161–7.

    Article  CAS  PubMed  Google Scholar 

  10. Wang MF, Lin HC, Wang YY, Hsu CH. Treatment of perennial allergic rhinitis with lactic acid bacteria. Pediatr Allergy Immunol. 2004;15:152–8.

    Article  PubMed  Google Scholar 

  11. Xiao JZ, Kondo S, Yanagisawa N, Takahashi N, Odamaki T, Iwabuchi N, et al. Probiotics in the treatment of Japanese cedar pollinosis: a double-blind placebo-controlled trial. Clin Exp Allergy. 2006;36:1425–35.

    Article  CAS  PubMed  Google Scholar 

  12. Wassenberg J, Nutten S, Audran R, Barbier N, Aubert V, Moulin J, et al. Effect of Lactobacillus paracasei ST11 on a nasal provocation test with grass pollen in allergic rhinitis. Clin Exp Allergy. 2011;41:565–73.

    Article  CAS  PubMed  Google Scholar 

  13. Perrin Y, Nutten S, Audran R, Berger B, Bibiloni R, Wassenberg J, et al. Comparison of two oral probiotic preparations in a randomized crossover trial highlights a potentially beneficial effect of Lactobacillus paracasei NCC2461 in patients with allergic rhinitis. Clin Transl Allergy. 2014;4:1.

    Article  PubMed Central  PubMed  Google Scholar 

  14. Juniper EF, Thompson AK, Ferrie PJ, Roberts JN. Development and validation of the mini Rhinoconjunctivitis Quality of Life Questionnaire. Clin Exp Allergy. 2000;30:132–40.

    Article  CAS  PubMed  Google Scholar 

  15. Dolle S, Berg J, Rasche C, Worm M. Tolerability and clinical outcome of coseasonal treatment with Escherichia coli strain Nissle 1917 in grass pollen-allergic subjects. Int Arch Allergy Immunol. 2014;163:29–35.

    Article  PubMed  Google Scholar 

  16. Lin GC, Zacharek MA. Climate change and its impact on allergic rhinitis and other allergic respiratory diseases. Curr Opin Otolaryngol Head Neck Surg. 2012;20:188–93.

    Article  PubMed  Google Scholar 

  17. Ivory K, Chambers SJ, Pin C, Prieto E, Arques JL, Nicoletti C. Oral delivery of Lactobacillus casei Shirota modifies allergen-induced immune responses in allergic rhinitis. Clin Exp Allergy. 2008;38:1282–9.

    Article  CAS  PubMed  Google Scholar 

  18. Lin WY, Fu LS, Lin HK, Shen CY, Chen YJ. Evaluation of the effect of Lactobacillus paracasei (HF.A00232) in children (6–13 years old) with perennial allergic rhinitis: a 12-week, double-blind, randomized, placebo-controlled study. Pediatr Neonatol. 2014;55:181–8.

    Article  PubMed  Google Scholar 

  19. Clark J, Schall R. Assessment of combined symptom and medication scores for rhinoconjunctivitis immunotherapy clinical trials. Allergy. 2007;62:1023–8.

    Article  CAS  PubMed  Google Scholar 

  20. Lue KH, Sun HL, Lu KH, Ku MS, Sheu JN, Chan CH, et al. A trial of adding Lactobacillus johnsonii EM1 to levocetirizine for treatment of perennial allergic rhinitis in children aged 7–12 years. Int J Pediatr Otorhinolaryngol. 2012;76:994–1001.

    Article  PubMed  Google Scholar 

  21. Bousquet J, Schunemann HJ, Samolinski B, Demoly P, Baena-Cagnani CE, Bachert C, et al. Allergic Rhinitis and its Impact on Asthma (ARIA): achievements in 10 years and future needs. J Allergy Clin Immunol. 2012;130:1049–62.

    Article  CAS  PubMed  Google Scholar 

  22. Tamura M, Shikina T, Morihana T, Hayama M, Kajimoto O, Sakamoto A, et al. Effects of probiotics on allergic rhinitis induced by Japanese cedar pollen: randomized double-blind, placebo-controlled clinical trial. Int Arch Allergy Immunol. 2007;143:75–82.

    Article  PubMed  Google Scholar 

  23. Ishida Y, Nakamura F, Kanzato H, Sawada D, Yamamoto N, Kagata H, et al. Effect of milk fermented with Lactobacillus acidophilus strain L-92 on symptoms of Japanese cedar pollen allergy: a randomized placebo-controlled trial. Biosci Biotechnol Biochem. 2005;69:1652–60.

    Article  CAS  PubMed  Google Scholar 

  24. Ouwehand AC, Nermes M, Collado MC, Rautonen N, Salminen S, Isolauri E. Specific probiotics alleviate allergic rhinitis during the birch pollen season. World J Gastroenterol. 2009;15:3261–8.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  25. Yonekura S, Okamoto Y, Okawa T, Hisamitsu M, Chazono H, Kobayashi K, et al. Effects of daily intake of Lactobacillus paracasei strain KW3110 on Japanese cedar pollinosis. Allergy Asthma Proc. 2009;30:397–405.

    Article  PubMed  Google Scholar 

  26. Bertazzoni E, Donelli G, Midtvedt T, Nicoli J, Sanz Y. Probiotics and clinical effects: is the number what counts? J Chemother. 2013;25:193–212.

    Article  PubMed  Google Scholar 

  27. Bron PA, Tomita S, Mercenier A, Kleerebezem M. Cell surface-associated compounds of probiotic lactobacilli sustain the strain-specificity dogma. Curr Opin Microbiol. 2013;16:262–9.

    Article  CAS  PubMed  Google Scholar 

  28. Hardy H, Harris J, Lyon E, Beal J, Foey AD. Probiotics, prebiotics and immunomodulation of gut mucosal defences: homeostasis and immunopathology. Nutrients. 2013;5:1869–912.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  29. Ozdemir O. Various effects of different probiotic strains in allergic disorders: an update from laboratory and clinical data. Clin Exp Immunol. 2010;160:295–304.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  30. Szajewska H. Understanding the role of probiotics and prebiotics in preventing allergic disease: evidence and methodological issues. Immunotherapy. 2013;5:869–78.

    Article  CAS  PubMed  Google Scholar 

  31. West CE. Gut microbiota and allergic disease: new findings. Curr Opin Clin Nutr Metab Care. 2014;17:261–6.

    Article  PubMed  Google Scholar 

Download references

Authors’ contributions

CN participated in study coordination, interpreted data and wrote the manuscript. AS conceived the study, participated in its design and coordination, and interpreted data. CADC analyzed data and helped to draft the manuscript. AM and SN participated in conceiving the study and its design, and interpreted data. All authors read and approved the final manuscript.

Competing interests

CN, CADC, AM and SN are all employees of Nestec S.A., which funded the work. AS was employee of Nestec S.A. at the time of conduct of the study.

Ethics

The protocol was approved by the Ethical Committee of the Charité University Hospital in Berlin, Germany (Ethikkommission der Charité—Universitätsmedizin, Berlin) with the approval number EA1/008/12. All enrolled subjects provided informed consent before the start of the study.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Chiara Nembrini.

Additional information

Chiara Nembrini and Anurag Singh contributed equally to this work

Additional file

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nembrini, C., Singh, A., De Castro, C.A. et al. Oral administration of Lactobacillus paracasei NCC 2461 for the modulation of grass pollen allergic rhinitis: a randomized, placebo-controlled study during the pollen season. Clin Transl Allergy 5, 41 (2015). https://doi.org/10.1186/s13601-015-0085-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13601-015-0085-4

Keywords