Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

One in five experiences hearing loss. The World Health Organization estimates that this number will increase to one in four in 2050. Luckily, effective hearing devices such as hearing aids and cochlear implants exist with advanced noise suppression and speaker enhancement algorithms that can significantly improve the quality of life of people suffering from hearing loss. State-of-the-art hearing devices, however, underperform in a so-called `cocktail party' scenario, when multiple persons are talking simultaneously. In such a situation, the hearing device does not know which speaker the user intends to attend to and thus which speaker to enhance and which other ones to suppress. Therefore, a new problem arises in cocktail party problems: determining which speaker a user is attending to, referred to as the auditory attention decoding (AAD) problem.The problem of selecting the attended speaker could be tackled using simple heuristics such as selecting the loudest speaker or the one in the user's look direction. However, a potentially better approach is decoding the auditory attention from where it originates, i.e., the brain. Using neurorecording techniques such as electroencephalography (EEG), it is possible to perform AAD, for example, by reconstructing the attended speech envelope from the EEG using a neural decoder (i.e., the stimulus reconstruction (SR) algorithm). Integrating AAD algorithms in a hearing device could then lead to a so-called `neuro-steered hearing device'. These traditional AAD algorithms are, however, not fast enough to adequately react to a switch in auditory attention, and are supervised and fixed over time, not adapting to non-stationarities in the EEG and audio data. Therefore, the general aim of this thesis is to develop novel signal processing algorithms for EEG-based AAD that allow fast, accurate, unsupervised, and time-adaptive decoding of the auditory attention.In the first part of the thesis, we compare different AAD algorithms, which allows us to identify the gaps in the current AAD literature that are partly addressed in this thesis. To be able to perform this comparative study, we develop a new performance metric - the minimal expected switch duration (MESD) - to evaluate AAD algorithms in the context of adaptive gain control for neuro-steered hearing devices. This performance metric resolves the traditional trade-off between AAD accuracy and time needed to make an AAD decision and returns a single-number metric that is interpretable within the application-context of AAD and allows easy (statistical) comparison between AAD algorithms. Using the MESD, we establish that the most robust currently available AAD algorithm is based on canonical correlation analysis, but that decoding the spatial focus of auditory attention from the EEG holds more promise towards fast and accurate AAD. Moreover, we observe that deep learning-based AAD algorithms are hard to replicate on different independent AAD datasets.In the second part, we address one of the main signal processing challenges in AAD: unsupervised and time-adaptive algorithms. We first develop an unsupervised version of the stimulus decoder that can be trained on a large batch of EEG and audio data without knowledge of ground-truth labels on the attention. The unsupervised stimulus decoder is iteratively retrained based on its own predicted labels, resulting in a self-leveraging effect that can be explained by interpreting the iterative updating procedure as a fixed-point iteration. This unsupervised but subject-specific stimulus decoder, starting from a random initial decoder, outperforms a supervised subject-independent decoder, and, using subject-independent information, even approximates the performance of a supervised subject-specific decoder. We also extend this unsupervised algorithm to an efficient recursive time-adaptive algorithm, when EEG and audio are continuously streaming in, and show that it has the potential to outperform a fixed supervised decoder in a practical use case of AAD.In the third part, we develop novel AAD algorithms that decode the spatial focus of auditory attention to provide faster and more accurate decoding. To this end, we use both a linear common spatial pattern (CSP) filtering approach and its nonlinear extension using Riemannian geometry-based classification (RGC). The CSP method achieves a much higher accuracy compared to the SR algorithm at a very fast decision rate. Furthermore, we show that the CSP method is the preferred choice over a similar convolutional neural network-based approach, and is also applicable on different directions of auditory attention, in a three-class problem with different angular domains, using only EEG channels close to the ears, and when generalizing to data from an unseen subject. Lastly, the RGC-based extension further improves the accuracy at slower decision rates, especially in the multiclass problem.To summarize, in this thesis we have developed crucial building blocks for a plug-and-play, time-adaptive, unsupervised, fast, and accurate AAD algorithm that could be integrated with a low-latency speaker separation and enhancement algorithm, and a wearable, miniaturized EEG system to eventually lead to a neuro-steered hearing device.

Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

Interest is rising towards EEG-based classification of the attentional engagement to speech regarding applications in the auditory and educational domain. Current hearing aid technologies do not have the knowledge whether the user is attentively listening to someone or not. However, this knowledge could allow the hearing aid to be turned into a passive working state when it is not ought to operate. Furthermore, performance of current unsupervised auditory attention decoding models (AAD) unraveling the attended speaker in a cocktail party scenario, could be ameliorated by selecting relevant training data, i.e. data from moments when the subject was listening actively to someone. Considering education, decoding the attention to speech of students in class could give insight about potential teaching malfunctions. However, research and data considering the decoding of attention to a single speaker are sparse. Studies on discrete sound stimuli are not representative considering the continuous nature of language. On top of that, classification into two distinct attention and no attention classes is currently not yet examined. In this thesis, attention decoding features are validated and classified using the Vanthornhout dataset and a new recorded dataset consisting of single speaker EEG data. The new recorded EEG dataset includes various distraction conditions and trials with alternating attention. We investigate linear envelope tracking models (LS and CCA) and spatial features (power spectrum and spectral entropy), which are classified with an LDA classifier. LS and CCA both show significant envelope tracking differences for the attention and no attention condition for both datasets. No significant difference is seen between the results of these two methods. None of the spatial features show a significant difference between the attention and no attention condition for both datasets. P-values for spectral entropy are however close to significance level. Spectral entropy shows more precise results than envelope tracking for the alternating attention conditions. LS, CCA and spectral entropy based classification perform significantly better than a random classifier for both datasets. No significant difference is seen between performance of these models. A combined model based on LS and spectral entropy together performs significantly better than when only using LS. It does not perform significantly better than when only using spectral entropy. We confirm the potential use of envelope tracking models in classification of attention to a single speaker. Additionally, spectral entropy is shown to be a valuable candidate as well, although it is currently not a golden standard in literature.

Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

Hearing-impaired people lack the ability to tune towards a speaker of interest in a multispeaker environment. For this purpose, neurosteered hearing devices are designed, wherein an auditory attention decoding (AAD) algorithm selects the attended speaker. Subjects could nevertheless be inattentive to any auditory stream, interfering with the setup. To this end, this work focuses on auditory attention measures. In addition, state-of-the-art attention detection requires subject-specific data, posing practical constraints, such that unsupervised implementations are pursued. A first objective is the selection of the auditory attention features themselves. A second objective is the conversion of these features into unsupervised ones. A third objective consists in the application of these features in an AAD framework. A dataset concerning auditory and visual attention is used, as well as a dataset concerning attention towards audio, mathematical exercises and texts for validating final performance. Regarding the first objective, least squares and canonical correlation analysis (CCA) prove discriminating between the attention cases. Furthermore, spectral entropy proves discriminating, whereas the band-power does not. In addition, a novel, Kullback-Leibler divergence (KLD) based feature attains a higher mean accuracy than the aforementioned features on low decision lengths. Combining the KLD and entropy features seems beneficial in both datasets. Finally, common spatial pattern attains the highest performance, yet suffers from a lack of interpretability. Regarding the second objective, results are focused on the least squares feature. CCA and principal component analysis (PCA) are applied to the EEG of both a source subject (not under study) and target subject (under study) in a domain adaptation manner. CCA proves to outperform the unadapted feature, whereas PCA does not. In addition, a novel discriminator-based approach is proposed. This method proves to be inferior to the unadapted feature, possibly due to the limited flexibility of said feature. Neither method does achieve correlation levels equal to the state-of-the-art iterative design, although the CCA method yields comparable differences in mean correlation. In fact, combining CCA and the iterative procedure seems to boost the mean difference in correlation significantly with respect to the individual methods. Regarding the third objective, auditory attention detection seems to allow for a reduction in the AAD training set size without compromising on performance.

Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

In de hedendaagse klinische audiologische praktijk maken we gebruik van gedragsmatige testen om spraakverstaanbaarheid te meten. Deze testen vergen de actieve participatie van de patiënt. Bij bepaalde populaties, zoals kleuters, kunnen we deze noodzakelijke coöperatie niet altijd garanderen. Als alternatief gebruiken we in de praktijk objectieve testen. Deze vergen geen participatie van de patiënt, maar onderzoeken enkel detectie van stimuli. Hiermee komen we niets te weten over het effectief spraakverstaan van de patiënt. Daarom wordt een objectieve maat gebaseerd op hersenresponsen gemeten aan de hand van elektro-encefalografie (EEG) of magneto-encefalografie (MEG), ontwikkeld. De hersenen blijken de spraakenveloppe te volgen, dit heet neurale enveloppe tracking. De spraakenveloppe kan uit het gemeten hersensignaal gereconstrueerd worden. De correlatie tussen deze afgeleide spraakenveloppe en de oorspronkelijke enveloppe wordt gebruikt als basis voor de objectieve maat van spraakverstaan. Deze maat werd al onderzocht en verbeterd voor volwassenen, maar nooit bij kinderen. Daarom onderzoekt deze masterproef de objectieve maat van spraakverstaan bij 5-/6-jarige kinderen. Eerst willen we nagaan of de objectieve maat ook bij kinderen een goede voorspeller is voor spraakverstaanbaarheid. Ook gaan we onderzoeken hoe de objectieve maat correleert met de gedragsmatige maat. Ten laatste zijn we geïnteresseerd in de relatie tussen het spraakverstaan en de taalontwikkeling bij deze jonge kinderen. Eerst, nemen we bij 14 normaalhorende kleuters een gedragsmatige spraakverstaantest in ruis af. Vervolgens, voeren we de EEG-meting met het aangepaste en kindvriendelijk protocol uit, gebruik makend van verhalen aangeboden in ruis. Deze data gaan we op twee manieren verwerken. Aan de hand van de decoder reconstrueren we de enveloppe uit het EEG-signaal. Het omgekeerde doen we aan de hand van TRF’s, we voorspelen hiermee het EEG-signaal aan de hand van de spraakenveloppe. Vervolgens wordt de Peabody Picture Vocabulary Test afgenomen om later de taalontwikkeling van de deelnemers te vergelijken met hun spraakverstaan. De resultaten van deze masterproef tonen aan dat ook bij kleuters de neurale enveloppe tracking toeneemt naarmate de SNR toeneemt. Dit zien we zowel in de backward- als de forward-verwerking. We vinden het significant effect terug in de delta- en thetaband en in de amplitude van de TRF-piek in de frontocentrale hersenregio. De gedragsmatige en objectieve maat vergelijken we op de 0 dB SNR-conditie. Deze data worden in deze proef niet genormaliseerd. De backward-verwerking brengt resultaten met zich mee die niet overeenkomen met de resultaten gevonden in de literatuur. Er is echter genoeg evidentie om te suggereren dat de gedragsmatige en objectieve maat van spraakverstaan gelinkt zijn met elkaar. We hebben geen verband kunnen aantonen tussen het spraakverstaan en de taalontwikkeling van de deelnemers. Om hierover conclusies te trekken, is er verder onderzoek nodig met kinderen met meer uiteenlopende taalontwikkelingsniveaus. Bijgevolg kunnen we besluiten dat de objectieve maat voor spraakverstaan een goede voorspeller is voor spraakverstaanbaarheid bij 5-/6-jarige kinderen. Het is aangeraden om in vervolgonderzoek de maat verder te analyseren bij slechthorende kleuters die een hoortoestel of cochleair implantaat dragen. Ook moet in verder onderzoek de maat verbeterd worden om deze in de klinische praktijk te kunnen implementeren.