Exploring how speech air flow may impact the spread of airborne diseases

I am participating on an American Association for the Advancement of Science (AAAS) 2022 meeting panel on “Transmission of Airborne Pathogens through Expiratory Activities” on Friday, February 18th from 6:00 to 6:45 AM Greenwich mean time. You can register for the meeting by clicking here. In advance of that meeting, the University of British Columbia asked me some Q&A questions exploring how speech air flow may impact the spread of airborne diseases.

The AAAS meeting itself is hosted by Prof. Bryan Gick of the University of British Columbia. It has individual talks by Dr. Sima Asadi on “Respiratory behavior and aerosol particles in airborne pathogen transmission”, Dr. Nicole M. Bouvier on “Talking about respiratory infectious disease transmission”, and myself on “Human airflow while breathing, speaking, and singing with and without masks”.

Dr. Sima Asadi’s talk focuses on the particles emitted during human speech, and the efficacy of masks in controlling their outward emission. For this work, Sima received the Zuhair A. Munir Award for the Best Doctoral Dissertation in Engineering from UC Davis in 2021. She is currently a postdoctoral associate in Chemical Engineering at MIT (Boston).

Dr. (Prof) Nicole M. Bouvier is an associate professor of Medicine and Infectious Diseases and Microbiology at the Icahn School of Medicine at Mount Sinai (New York). Nichole discusses how we understand the roots by which respiratory microorganisms, like viruses and bacteria, transmit between humans, which is fundamental in how we develop both medical and public health countermeasures to reduce or prevent their spread. However, much of what we think we know is based on evidence that is incomplete at best, and full of confusing terminology, as the current COVID-19 pandemic has made abundantly clear.

I myself am new to airborne transmission research, coming instead from the perspective that visual and aero-tactile speech help with speech perception, and so masks would naturally interfere with clear communication. They would do this by potentially muffling some speech sounds, but mostly by cutting off the perceiver form visual and even tactile speech signals.

However, since my natural interests involve speech air flow, I was ideally suited to move into research studying how these same air flows may be reduced or eliminated by face masks. I conduct this research with a Mechanical Engineering team at the University of Canterbury, and some of their results are featured in my individual presentation. Our most recent publication on Speech air flow with and without face masks was highlighted in previous posts on Maps of Speech, and in a YouTube video found here.

Speech air flow with and without face masks

It took a while due to the absolutely shocking amount of work required for the “Gait change in tongue movement” article, but Natalia Kabaliuk, Luke Longworth, Peiman Pishyar‑Dehkordi, Mark Jermy and I were able to get our article on “Speech air flow with and without face masks” accepted to Scientific Reports (Nature Publishing Group). The article is now out (though a pre-review version had been available since we submitted this article to Sci Rep). You can also watch my YouTube video describing many of the results.

Here is an example of a low-stiffness air-flow from a porous mask, which allows leaks from the tops, bottoms, and sides, and forward flow prevention, as taken from Figure 5 of the article.

Figure 5. Audio and Schlieren of speech through a porous face mask (Frame 621, 1st block, CORI Supermask). Image from 88 ms after the release burst for the [kh ] in “loch”. Note that the k’s puff is smoother and less well defined than the one in Fig. 2, but still has eddies that change air-density across the span of the puff. The red-dashed line in the audio waveform indicates the timing of the schlieren frame.

And here is an example of typical higher-stiffness flow from a less porous mask from Figure 8.

Figure 8. Audio and Schlieren of speech with a tightly fitting surgical mask (Frame 334, 1st block, Henry Schlein surgical mask [level 2]). Air slowly flows out above the eyes, floating out and upward continuously. The red-dashed line in the audio waveform indicates the timing of the schlieren frame.

Masks can be made to fit tighter, as in well-designed KN95/N95 masks and masks with metal strips at the nose to prevent upward-escaping air flow. However, for all the masks we studied, the tradeoff was not entirely avoided. And with that, here is our abstract:

Face masks slow exhaled air flow and sequester exhaled particles. There are many types of face masks on the market today, each having widely varying fits, filtering, and air redirection characteristics. While particle filtration and flow resistance from masks has been well studied, their effects on speech air flow has not. We built a schlieren system and recorded speech air flow with 14 different face masks, comparing it to mask-less speech. All of the face masks reduced air flow from speech, but some allowed air flow features to reach further than 40 cm from a speaker’s lips and nose within a few seconds, and all the face masks allowed some air to escape above the nose. Evidence from available literature shows that distancing and ventilation in higher-risk indoor environment provide more benefit than wearing a face mask. Our own research shows all the masks we tested provide some additional benefit of restricting air flow from a speaker. However, well-fitted mask specifically designed for the purpose of preventing the spread of disease reduce air flow the most. Future research will study the effects of face masks on speech communication in order to facilitate cost/benefit
analysis of mask usage in various environments.

Gait Change in Tongue Movement

Bryan Gick and I recently published an article on “Gait Change in Tongue Movement” in Scientific Reports (Nature Publishing Group). Below is the abstract, with images alongside. However, if you want an easy-to-follow walkthrough of the paper, I also published a YouTube video on the paper on my YouTube Channel for Maps of Speech.

During locomotion, humans switch gaits from walking to running, and horses from walking to trotting to cantering to galloping, as they increase their movement rate. It is unknown whether gait change leading to a wider movement rate range is limited to locomotive-type behaviours, or instead is a general property of any rate-varying motor system. The tongue during speech provides a motor system that can address this gap. In controlled speech experiments, using phrases containing complex tongue-movement sequences, we demonstrate distinct gaits in tongue movement at different speech rates. As speakers widen their tongue-front displacement range, they gain access to wider speech-rate ranges.

At the widest displacement ranges, speakers also produce categorically different patterns for their slowest and fastest speech. Speakers with the narrowest tongue-front displacement ranges show one stable speech-gait pattern, and speakers with widest ranges show two. Critical fluctuation analysis of tongue motion over the time-course of speech revealed these speakers used greater effort at the beginning of phrases—such end-state-comfort effects indicate speech planning.

Based on these findings, we expect that categorical motion solutions may emerge in any motor system, providing that system with access to wider movement-rate ranges.

Colorized Schlieren – with and without face masks

This is my first professional (as opposed to personal) youtube video, on my work channel for Maps of Speech. Today I’m making my debut with a scientific report on colorized schlieren images of speech air flow with and without masks. Please share widely, and encourage others to share widely. This is intended to be a worldwide release, and has been approved by the New Zealand Ministry of Business, Innovation, and Employment’s communications team, as well as our COVID-research team. In general, videos on this channel are more formal than my HoT videos, and will often be made in collaboration with whatever research team is working on the related projects. The details below this video link to three other unlisted videos that show the entirety of the schlieren videos referenced – without any audio commentary.

Frame of Colorized Schlieren data from a speaker saying “The beige hues on the waters of the loch impressed all”, no face mask.

Evidence for active control of tongue lateralization in Australian English /l/

Jia Ying, Jason A. Shaw, Christopher Carignan, Michael Proctor, myself, and Catherine T. Best just published Evidence for active control of tongue lateralization in Australian English /l/. Most research on /l/ articulation has looked at motion timing along the midline, or midsagittal plane. This study compares that information to motion on the sides of the tongue. It focuses on Australian English (AusE), using three-dimensional electromagnetic articulography (3D EMA).

Fig. 11. Temporal dynamics of tongue curvature in the coronal plane over the entire V-/l/ interval. The brackets indicate onset (red) and coda (blue) /l/ intervals. Each bracket extends from the /l/ onset to its peak. For onset /l/s, the peak occurs earlier (at about 200 ms) than coda /l/s (at about 450 ms). A time of zero indicates the vowel onset. The 800-interval window captures the entire V-/l/ articulation in every token.

The articulatory analyses show: 1) consistent with past work, the timing lag between mid-sagittal tongue tip and tongue body gestures differs for syllable onsets and codas, and for different vowels.

2) The lateral channel is formed by tilting the tongue to the left/right side of the oral cavity as opposed to curving the tongue within the coronal plane

3) the timing of lateral channel formation relative to the tongue body gesture is consistent across syllable positions and vowel contexts – even though temporal lag between tongue tip and tongue body gestures varies.

This last result suggests that lateral channel formation is actively controlled as opposed to resulting as a passive consequence of tongue stretching. These results are interpreted as evidence that the formation of the lateral channel is a primary articulatory goal of /l/ production in AusE.

Locating de-lateralization in the pathway of sound changes affecting coda /l/

Patrycja Strycharczuk, Jason Shaw, and I just published Locating de-lateralization in the pathway of sound changes affecting coda /l/, in which we analyze New Zealand English /l/ using Ultrasound and Articulometry. You can find the article here. Put in the simplest English terms, the article shows the process by which /l/-sounds in speech can change over time from a light /l/ (like the first /l/ in ‘lull’) to a darker /l/ (like the second /l/ in ‘lull’). This darkening is the result of the upper-back, or dorsum, of the tongue moving closer to the back of the throat. This motion in turn reduces lateralization, or the lowering of the sides of the tongue away from the upper teeth. This is followed, over time, by the tongue tip no longer connecting to the front of the hard palate – the /l/ becomes a back vowel or vocalizes.

Two subcategories identified in the distribution of TT raising for the Vl#C context. Red = vocalized.

If you want a more technical description, Here is the abstract:

‘Vocalization’ is a label commonly used to describe an ongoing change in progress affecting coda /l/ in multiple accents of English. The label is directly linked to the loss of consonantal constriction observed in this process, but it also implicitly signals a specific type of change affecting manner of articulation from consonant to vowel, which involves loss of tongue lateralization, the defining property of lateral sounds. In this study, we consider two potential diachronic pathways of change: an abrupt loss of lateralization which follows from the loss of apical constriction, versus slower gradual loss of lateralization that tracks the articulatory changes to the dorsal component of /l/. We present articulatory data from seven speakers of New Zealand English, acquired using a combination of midsagittal and lateral EMA, as well as midsagittal ultrasound. Different stages of sound change are reconstructed through synchronic variation between light, dark, and vocalized /l/, induced by systematic manipulation of the segmental and morphosyntactic environment, and complemented by comparison of different individual articulatory strategies. Our data show a systematic reduction in lateralization that is conditioned by increasing degrees of /l/-darkening and /l/-vocalization. This observation supports the idea of a gradual diachronic shift and the following pathway of change: /l/-darkening, driven by the dorsal gesture, precipitates some loss of lateralization, which is followed by loss of the apical gesture. This pathway indicates that loss of lateralization is an integral component in the changes in manner of articulation of /l/ from consonantal to vocalic.

The Impacts of a Community-Based Health Education and Nutritional Support Program on Birth Outcomes Among Migrant Workers in Maesot, Thailand: A Retrospective Review

Used by permission: Charis Project / Shade Tree Foundation.

Wayland Joseph Blue, myself, and Carrien Leith Blue have co-authored a paper on the benefits of community-based health education and nutritional support on the birth outcomes among migrant workers in Maesot, Thailand, recently published in the International Social Science Review.

We worked on this article together because it was important to share with the world how low-cost education and nutritional supplementation can massively improve birth weight and early childhood health. The Charis Project / Shade tree Foundation have been active on such tasks for many years now, and I have be honoured to get to be a part of documenting their great work. To quote the first author Wayland: “These approaches aren’t limited to developing countries. They are as relevant in inner-city communities in the United States as they are in rural Thailand and Myanmar.”

Used by permission: Charis Project / Shade Tree Foundation.

The acknowledgment section of the article didn’t make it through proofing, so I put that material here on my website. We wanted to thank Zohreh Bayatrizi for being an early reader of the article and providing great feedback, and for TinTin’s efforts in helping us meticulously work through all of the data we had – she spend many days enduring my rather thorough questions. These questions extended beyond what we put in the article, and helped Charis reorganize their records. I am also grateful that UC permitted me to spend some of my Sabbatical time on this task as it is outside of my usual research focus. In addition, Pi Gamma Mu has proven itself dedicated to making excellent social science research freely available. In an age of far too many important articles behind paywalls, I am glad that this article is not, and I would like to thank Pi Gamma Mu and all of their alumni for making this possible.

Abstract

Here is presented a retrospective review of the Charis Project’s Family Engagement Program (FEN) as it existed in 2014-2017. FEN was a program of women’s health education, nutrition supplements, and family visitation. The education program consisted of a 12-week course on nutrition, maternity, and sex education taught individually and in groups, focusing on pregnant Burmese migrant laborers, but including approximately 20 percent male participation. The nutrition supplements consisted of 5 kilograms of fresh vegetables and 12 eggs weekly to pregnant mothers, from course onset to about six months after childbirth depending on family needs. Family visitation took place during food deliveries, and focused on individual counselling and family stability. The program served 39 families from 2014-2017. FEN did not reduce neonatal mortality (due to a miscarriage and severe congenital birth defect), but resulted in all surviving infants being born normal weight and surviving to the end of 2019, representing a significant improvement over the 25.6 percent low birth weights reported for Kayin State, Myanmar.

Phonological contrast and phonetic variation: the case of velars in Iwaidja

Jason A. Shaw, Christopher Carignan, Tonya Agostini, Robert Mailhammer, Mark Harvey and I have recently had an article on Iwaidja accepted to Language. The article is now out, is an open publication, and can be accessed at project MUSE here.

Figure 3 from the article

It was a privilege to be a part of this project!. The article represents several years of work on the part of all of us – my part seeming to me to be the least of all. We acknowledge a host of granting agencies, lab supporters, and researchers, but most of all the Iwaidja speakers: Charlie Mangulda, David Cooper, James Cooper, Henry Guwiyul, Ilijili Lamilami, Isobel Lamilami, and Maggi Maburnbi for sharing the Iwaidja language and culture. (For those of you who don’t know, Iwaidja is an endangered Australian Language from NorthWest Australia.) The abstract itself summarizes the article succinctly.

Abstract

A field-based ultrasound and acoustic study of Iwaidja, an endangered Australian aboriginal language, investigated the phonetic identity of non-nasal velar consonants in intervocalic position, where past work had proposed a [+continuant] vs [-continuant] phonemic contrast. We analyzed the putative contrast within a continuous phonetic space, defined by both acoustic and articulatory parameters, and found gradient variation from more consonantal realizations, e.g. [ɰ], to more vocalic realizations, e.g. [a]. The distribution of realizations across lexical items and speakers did not support the proposed phonemic contrast. This case illustrates how lenition that is both phonetically gradient and variable across speakers and words can give the illusion of a contextually restricted phonemic contrast.

Native language influence on brass instrument performance

Matthias Heyne, myself, and Jalal Al-Tamimi recently published Native language influence on brass instrument performance: An application of generalized additive mixed models (GAMMs) to midsagittal ultrasound images of the tongue. The paper contains the bulk of the results form Matthias’ PhD Dissertation. The study is huge, with ultrasound tongue recordings of 10 New Zealand English (NZE) and 10 Tongan trombone players. There are 12,256 individual tongue contours of vowel tokens (7,834 for NZE, 4,422 for Tongan) and 7,428 individual tongue contours of sustained note production (3,715 for NZE, 3,713 for Tongan).

Figure 4 in the paper.

The results show that native language influences tongue position during Trombone note production. This includes tongue position and note variability. The results also support Dispersion Theory (Liljencrants and Lindblom 1972; Lindblom, 1986; Al-Tamimi and Ferragne, 2005) in that vowel production is more variable in Tongan, which has few vowels, then in NZE, which has many.

The results also show that note production at the back of the tongue maps to low-back vowel production (schwa and ‘lot’ for NZE, /o/ and /u/ for schwa). These two result sets support an analysis of local optimization with semi-independent tongue regions (Ganesh et al., 2010, Loeb, 2012).

The results do not, however, support the traditional brass pedagogy hypothesis that higher notes are played with a closer (higher) tongue position. However, Matthias is currently working with MRI data that *does* support the brass pedagogy hypothesis, and that we might not have seen this because of the ultrasound transducer stabilization system needed to keep the ultrasound probe aligned to the participant’s head.

Liljencrants, Johan, and Björn Lindblom. 1972. “Numerical Simulation of Vowel Quality Systems: The Role of Perceptual Contrast.” Language, 839–62.

Lindblom, Björn. 1963. Spectrographic study of vowel reduction. The Journal of the Acoustical Society of America 35(11): 1773–1781.

Al-Tamimi, J., and Ferragne, E. 2005. “Does vowel space size depend on language vowel inventories? Evidence from two Arabic dialects and French,” in Proceedings of the Ninth European Conference on Speech Communication and Technology, Lisbon, 2465–2468.

Ganesh, Gowrishankar, Masahiko Haruno, Mitsuo Kawato, and Etienne Burdet. 2010. “Motor Memory and Local Minimization of Error and Effort, Not Global Optimization, Determine Motor Behavior.” Journal of Neurophysiology 104 (1): 382–90.

Loeb, Gerald E. 2012. “Optimal Isn’t Good Enough.” Biological Cybernetics 106 (11–12): 757–65.

Tri-modal speech: Audio-visual-tactile integration in speech perception

Figure 3 in paper.

Myself, Doreen Hansmann, and Catherine Theys just published our article on “Tri-modal Speech: Audio-visual-tactile Integration in Speech Perception” in the Journal of the Acoustical Society of America. This paper was also presented as a poster at the American Speech-Language-Hearing Association (ASHA) Annual Convention in Orlando, Florida, November 21-22, 2019, winning a meritorious poster award.

TL-DR; People use auditory, visual, and tactile speech information to accurately identify syllables in noise. Auditory speech information is the most important, then visual information, and lastly aero-tactile information – but we can use them all at once.

Abstract

Speech perception is a multi-sensory experience. Visual information enhances (Sumby and Pollack, 1954) and interferes (McGurk and MacDonald, 1976) with speech perception. Similarly, tactile information, transmitted by puffs of air arriving at the skin and aligned with speech audio, alters (Gick and Derrick, 2009) auditory speech perception in noise. It has also been shown that aero-tactile information influences visual speech perception when an auditory signal is absent (Derrick, Bicevskis, and Gick, 2019a). However, researchers have not yet identified the combined influence of aero-tactile, visual, and auditory information on speech perception. The effects of matching and mismatching visual and tactile speech on two-way forced-choice auditory syllable-in-noise classification tasks were tested. The results showed that both visual and tactile information altered the signal-to-noise threshold for accurate identification of auditory signals. Similar to previous studies, the visual component has a strong influence on auditory syllable-in-noise identification, as evidenced by a 28.04 dB improvement in SNR between matching and mismatching visual stimulus presentations. In comparison, the tactile component had a small influence resulting in a 1.58 dB SNR match-mismatch range. The effects of both the audio and tactile information were shown to be additive.

Derrick, D., Bicevskis, K., and Gick, B. (2019a). “Visual-tactile speech perception and the autism quotient,” Frontiers in Communication – Language Sciences 3(61), 1–11, doi: http://dx.doi.org/10.3389/fcomm.2018.00061

Gick, B., and Derrick, D. (2009). “Aero-tactile integration in speech perception,” Nature 462, 502–504, doi: https://doi.org/10.1038/nature08572.

McGurk, H., and MacDonald, J. (1976). “Hearing lips and seeing voices,” Nature 264, 746–748, doi: http://dx.doi.org/https://doi.org/10.1038/264746a0