PRACTICAL DEEP LEARNING AGLORITHMS USING ESTIMATION THEORY

Abstract

Deep Learning Networks (DLN) is a relatively new artificial intelligence algorithm that gained popularity quickly due to its unprecedented performance. One of the key elements for this success is DL’s ability to extract a high-level of information from large amounts of raw data. This ability comes at the cost of high computational and memory requirements for the training process. Estimation algorithms such as the Extended Kalman Filter (EKF) and the Smooth Variable Structure Filter (SVSF) are used in literature to train small Neural Networks. However, they have failed to scale well with deep networks due to their excessive requirements for computation and memory size. In this thesis the concept of using EKF and SVSF for DLN training is revisited. A New family of filters that are efficient in memory and computational requirements are proposed and their performance is evaluated against the state-of-the-art algorithms. The new filters show competitive performance to existing algorithms and do not require fine tuning. These new findings change the scientific community’s perception that estimation theory methods such as EKF and SVSF are not practical for their application to large networks. A second contribution from this research is the application of DLN to Fault Detection and Diagnosis. The findings indicate that DL can analyze complex sound and vibration signals in testing of automotive starters to successfully detect and diagnose faults with 97.6% success rates. This proves that DLN can automate end-of-line testing of starters and replace operators who manually listen to sound signals to detect any deviation. Use of DLN in end-of-line testing could lead to significant economic benefits in manufacturing operations. In addition to starters, another application considered is the use of DLN in monitoring of the State-Of-Charge (SOC) of batteries in electric cars. The use of DLN for improving the SOC prediction accuracy is discussed.