Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

This master thesis has for objective to explore different techniques (architecture, pruning as an architecture search, knowledge distillation, quantization) to improve the inference time of convolutionnal neural networks performing image classification on an embedded device.

machine learning --- deep learning --- Ingénierie, informatique & technologie > Sciences informatiques

Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

A multi-modal neural network exploits information from different channels and in different terms (e.g., images, text, sounds, sensor measures) in the hope that the information carried by each mode is complementary, in order to improve the predictions the neural network. Nevertheless, in realistic situations, varying levels of perturbations can occur on the data of the modes, which may decrease the quality of the inference process. An additional difficulty is that these perturbations vary between the modes and on a per-sample basis. This work presents a solution to this problem. The three main contributions are described below. 
First, a novel attention module is designed, analysed and implemented. This attention module is constructed to help multi-modal networks handle modes with perturbations.

Secondly, two new regularizers are developed to improve the generalization of the robustness gain on more intensive failing modes (relative to the training set).

Lastly, a unified multi-modal attention module is presented, combining the main types of attention mechanisms in the deep learning literature with our module. We suggest that this unified module could be coupled with a prediction model to enable the latter face unexpected situations, and improve the extraction of the relevant information in the data.

Multimodal, Deep Learning, Attention, Robustness --- Ingénierie, informatique & technologie > Sciences informatiques

Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

Today, for Billy and many accounting fiduciaries, invoice information is usually encoded manually by the accountant or by a low-performance software, so a lot of time is wasted on encoding and not on advice. Indeed, During the year 2021, Billy's accountants spent 37% of their time on encoding. Consequently, the recognition of fields in a semi-structured document of variable layout (i.e. invoice) is a growing need for accountants, and especially for Billy, in which the number of new customers increasing every month. Nonetheless, the text and image pre-training strategies of the Transformer architecture model have proven to be efficient in the field of document understanding. Thus, several OCR tools were tested, and the Azure OCR tool, which gave the higher performances, was selected to extract text from image invoice of Billy's customers in order to create datasets. Indeed, this allows the elaboration of four datasets partially annotated, named BTT, BTT Star, BTT QV, and BTT QV Date, which were created from scanned purchase, sales documents, and their accounting encoding in the accounting Horus software. Then, the fine-tuning of the pre-trained multi-modal models LayoutLMv2_BASE, LayoutLMv2_LARGE, and LayoutXLM_BASE has been done. In contrast to the previous architectural models of the LayoutLM family, these models include, in addition to the text and layout information, information that can be provided by the document image. Thanks to spatial-aware self-attention mechanisms integrated in the Transformer architecture model, it is able to interpret relations through different bounding boxes.&#13;According to Billy's accountants, invoice information is recognized by Horus in 70\% of the cases. During the experiments conducted in this Master thesis, it was shown that on token classification tasks, higher results were obtained for the the different datasets in terms of F1-score: BTT (0.9420), BTT Star (0.9553), BTT QV ( 0.9413) and BTT QV Date (0.9472). In addition, similar state-of-the-art results were obtained using the open source CORD dataset which gives an F1-score of 0.9354. Moreover, the impact of a pre-trained model on a dataset composed only of English documents LayoutXLM_BASE) was studied in comparison with a pre-trained model on a multi-lingual dataset (LayoutLMv2_BASE) to classify tokens on documents mostly in French. The results show that the pre-trained model does not have a great impact on the final result for this type of task: (BTT (0.9323 -> 0.9338), BTT Star (0.9354 -> 0.9468), BTT QV (0.9229 <- 0.8955), and BTT QV Date (0.9411 <- 0.9328). To conclude, since the results of the four datasets were close to each other, the dataset BTT Star produced the best results. This dataset has the largest number of labels and is the most widely distributed over the documents, leading to the hypothesis that a more widely distributed set of labels provides better results.&#13;Finally, to concretize this work, a web application was developed in parallel in order to use this tool in everyday life for both the accountants and the customers.

Machine learning, Deep learning --- Ingénierie, informatique & technologie > Sciences informatiques

Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

Being able to automatically personalize football content according to each viewer's preferences is the future of the broadcasting world. One of the steps is to be able to classify the current game event in a football video stream. During the game, the main camera is filming a panoramic view of the football game events. This master's thesis introduces a three stages framework where global features such as the field, the players and the lines are extracted from the video stream of the main camera and processed to compute second stage features that are used inside of a decision mechanism to predict the current game event. Computer vision and machine learning techniques such as deep learning networks and semantic segmentation are used to extract the main features. From these features, some second stage features such as the extraction of the main circle or the mean position of the players inside of the field are computed and handed out to a decision tree that classifies the current game event. A playout application developed in this master's thesis allows the user to visualize some of the features computed by the algorithms and the game event predicted. The performances of the features extraction found in this work are impressive, especially for the line extraction problem for which we have a global accuracy of 95% at the pixel level. While the interpretation of game events remains challenging and unsolved, we managed, with our framework, to achieve an accuracy of more than 90% for the classification between attack, defense and middle game.

football --- deep learning --- computer vision --- machine learning --- Ingénierie, informatique & technologie > Ingénierie électrique & électronique

Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

Variable and feature selection have become the focus of much research, especially in bioinformatics where there are many applications. Machine learning is a powerful tool to select features, however not all machine learning algorithms are on an equal footing when it comes to feature selection. Indeed, many methods have been proposed to carry out feature selection with random forests, which makes them the current go-to model in bioinformatics.&#13;On the other hand, thanks to the so-called deep learning, neural networks have benefited a huge interest resurgence in the past few years. However neural networks are blackbox models and very few attempts have been made in order to analyse the underlying process. Indeed, quite a few articles can be found about feature extraction with neural networks (for which the underlying inputs-outputs process does not need to be understood), while very few tackle feature selection.&#13;&#13;In this document, we propose new algorithms in order to carry out feature selection with deep neural networks. To assess our results, we generate regression and classification problems which allow us to compare each algorithm on multiple fronts: performances, computation time and constraints. The results obtained are really promising since we manage to achieve our goal by surpassing (or equaling) random forests performances in every case (which was set to be our “state-of-the-art” comparison).&#13;&#13;Due to the promising results obtained on artificial datasets we also tackle the DREAM4 challenge. Due to the very small number of samples available in the datasets, this challenge is supposedly an ill-suited problem for neural networks. We were nevertheless able to achieve near state of the art results.&#13;&#13;Finally, extensions are given for most of our methods. Indeed, the algorithms discussed are very modulable and can be adapted regarding the problem faced. For example, we explain how one of our algorithm can be adapted in order to prune neural networks without losing accuracy.

Machine learning --- Deep learning --- Feature selection --- Artificial neural networks --- Ingénierie, informatique & technologie > Sciences informatiques