Shaima Abdelmageed Wrist-based Phonocardiogram Diagnosis Leveraging Machine Learning ACTA WASAENSIA 416 ACADEMIC DISSERTATION To be presented, with the permission of the Board of the School of Technology and Innovations of the University of Vaasa, for public examination in Auditorium Kurtén (C203) on the 2nd of May, 2019, at noon. Reviewers Professor Tarik Taleb Aalto University School of Electrical Engineering Department of Communications and Networking P.O.BOX 11000 FI-00076 AALTO Finland Docent Teemu Myllylä University of Oulu Electronics and Communications Engineering P.O.BOX 8000 FI-90014 University of Oulu Finland III Julkaisija Julkaisupäivämäärä Vaasan yliopisto Huhtikuu 2019 Tekijä(t) Julkaisun tyyppi Shaima Tajalsir Abdelmageed Abdel Rahman Väitöskirja Orcid ID Julkaisusarjan nimi, osan numero Acta Wasaensia, 416 Yhteystiedot ISBN Vaasan yliopisto Teknologian ja innovaatiojohtamisen yksikkö PL 700 FI-65101 VAASA 978-952-476-850-4 (painettu) 978-952-476-851-1 (verkkojulkaisu) URN:ISBN:978-952-476-851-1 ISSN 0355-2667 (Acta Wasaensia 416, painettu) 2323-9123 (Acta Wasaensia 416, verkkoaineisto) Sivumäärä Kieli 134 englanti Julkaisun nimike Rannepohjaisen fonokardiogrammin analysointi koneoppimista hyödyntäminen Tiivistelmä Teknologian valtavan kehittymisen ja nopean elämänrytmin myötä välittömästi saatu tieto on noussut jokapäiväiseksi välttämättömyydeksi, erityisesti hätätapauksissa, joissa jokainen säästetty minuutti on tärkeää ihmishenkien pelastamiseksi. Mobiiliterveys, eli mHealth, on yleisesti valjastettu käyttöön nopeaksi diagnoosimenetelmäksi mobiililaitteiden avulla. Käyttö on kuitenkin ollut haastavaa korkean datan laatuvaatimuksen ja suurten tiedonkäsittelyvaatimuksien, nopean laskentatehon ja sekä suuren virrankulutuksen vuoksi. Tämän tutkimuksen tavoitteena oli diagnosoida sydänsairauksia fonokardiogrammianalyysin (PCG) perusteella käyttämällä koneoppimistekniikoita niin, että käytettävä laskentateho rajoitetaan vastaamaan mobiililaitteiden kapasiteettia. PCG-diagnoosi tehtiin käyttäen kahta tekniikkaa 1. Parametrinen estimointi käyttäen moniulotteista luokitusta, erityisesti signaalien erotteluanalyysin avulla. Tätä asiaa tutkittiin syvällisesti käyttäen erilaisia tilastotieteellisesti kuvailevia piirteitä. Piirteiden irrotus suoritettiin käyttäen Wavelet- muunnosta ja suodatinpankkia. 2. Keinotekoisia neuroverkkoja ja erityisesti hahmontunnistusta. Tässä menetelmässä käytetään myös PCG-signaalin hajoitusta ja Wavelet-muunnos -suodatinpankkia. Tulokset osoittivat, että PCG 19dB:n signaali-kohina-suhteella voi johtaa 97,33% onnistuneeseen diagnoosiin käytettäessä ensimmäistä tekniikkaa. Signaalin hajottaminen neljään alikaistaan suoritettiin käyttämällä toisen asteen suodatinpankkia. Jokainen alikaista kuvattiin käyttäen kahta piirrettä: signaalin keskiarvoa ja kovarianssia, näin saatiin yhteensä kahdeksan ominaisuutta kuvaamaan noin yhden minuutin näytettä PCG- signaalista. Lisäksi tutkittiin ja verrattiin eriasteisia suodattimia ja piirteitä. Toista tekniikkaa käyttäen diagnoosi johti 100% onnistuneeseen luokitteluun 83,3% luotettavuustasolla. Tuloksia käsitellään ja pohditaan, sekä tehdään niistä johtopäätöksiä. Lopuksi ehdotetaan ja suositellaan käytettyihin menetelmiin uusia parannuksia jatkotutkimuskohteiksi. Asiasanat Analyysi, luokittelu, tiedon laatu, taudinmääritys, eTerveys, suodatinpankki, mobiiliterveys, neuroverkot, fonokardiogrammi, signaalikohinasuhde, Wavelet- muunnos V Publisher Date of publication Vaasan yliopisto April 2019 Author(s) Type of publication Shaima Tajalsir Abdelmageed Abdel Rahman Doctoral thesis Orcid ID Name and number of series Acta Wasaensia, 416 Contact information ISBN University of Vaasa Faculty Department or subject P.O. Box 700 FI-65101 Vaasa Finland 978-952-476-850-4 (print) 978-952-476-851-1 (online) URN:ISBN:978-952-476-851-1 ISSN 0355-2667 (Acta Wasaensia 416, print) 2323-9123 (Acta Wasaensia 416, online) Number of pages Language 134 English Title of publication Wrist-based Phonocardiogram Diagnosis Leveraging Machine Learning Abstract With the tremendous growth of technology and the fast pace of life, the need for instant information has become an everyday necessity, more so in emergency cases when every minute counts towards saving lives. mHealth has been the adopted approach for quick diagnosis using mobile devices. However, it has been challenging due to the required high quality of data, high computation load, and high-power consumption. The aim of this research is to diagnose the heart condition based on phonocardiogram (PCG) analysis using Machine Learning techniques assuming limited processing power, in order to be encapsulated later in a mobile device. The diagnosis of PCG is performed using two techniques; 1. parametric estimation with multivariate classification, particularly discriminant function. Which will be explored at length using different number of descriptive features. The feature extraction will be performed using Wavelet Transform (Filter Bank). 2. Artificial Neural Networks, and specifically Pattern Recognition. This will also use decomposed version of PCG using Wavelet Transform (Filter Bank). The results showed 97.33% successful diagnosis using the first technique using PCG with a 19 dB Signal-to-Noise-Ratio. When the signal was decomposed into four sub- bands using a Filter Bank of the second order. Each sub-band was described using two features; the signal’s mean and covariance. Additionally, different Filter Bank orders and number of features are explored and compared. Using the second technique the diagnosis resulted in a 100% successful classification with 83.3% trust level. The results are assessed, and new improvements are recommended and discussed as part of future work. Keywords Analysis, classification, Data quality, diagnosis, eHealth, Filter Banks, mHealth, Neural Networks, PCG, SNR, Wavelet Transform VII ACKNOWLEDGEMENT “Desire is the key to motivation, but it's determination and commitment to an unrelenting pursuit of your goal - a commitment to excellence - that will enable you to attain the success you seek.” - Mario Andretti I started this journey in 2012, the same year that I finished my master’s degree and started working for Lumi Mobile (now known as Auga Solutions.). Despite my enthusiasm for this research that’s very dear to me, I had to prioritise work. Not just for the obvious reasons but also because I passionately liked my job. This continued for two years; leading the development team of one of the great products at the company with no progress on the research front! That’s when I had to stop and reprioritise! I, then, lowered my working hours and picked up research work at the university, all to come back full force to my beloved research… and it almost worked! Only to be hindered again by my promotion to become the product manager of that same product I liked so much. It was sad, because my success at work seemed to push my PhD dream farther away. Life became a race between work and school, and the tunnel seemed to have no end... Until I made it! I found the balance and made it all fit together. And what is an extra year or two when you get to do it all at once! This was not easy; it wasn’t always “peaches and cream” and although I claim to be a super multitasker; this was indeed a true test, especially in the last few months. None of it would’ve been possible if it wasn’t for the awesome people in my life… My very understanding boss, Marcus Wikars, who didn’t only allow me to lower my hours, but he kept me challenged and inspired. Thank you for your trust, Ace! My friend, Reino Virrankoski, who hired me as a researcher at the University of Vasa and gave me the option to do more. Kept me focused on the research and the area of eHealth with the project “Nordic Telemedicine Center”. Thank you for NTC, Reino! My brilliant mentor, Prof. Mohammed Elmusrati, who has been with me every step of the way. For every productive discussion, all the tips and the constant guidance, I, sincerely, thank you! My dear father, my loving mother, and my very supportive sister and brother, I thank you with all my heart. It’s only possible because you believe in me. Vasa, Oct. 1st, 2018 IX Contents ACKNOWLEDGEMENT ........................................................................... VII 1 INTRODUCTION ................................................................................ 1 1.1 Motivation ............................................................................... 2 1.2 Objectives and contributions ................................................... 3 1.3 Methods .................................................................................. 5 1.4 Why focus on heart condition? ................................................. 8 1.5 Thesis structure ...................................................................... 9 2 LITERATURE REVIEW ........................................................................ 11 2.1 About eHealth and mHealth .................................................. 11 2.2 History of eHealth/mHealth ................................................... 12 2.3 mHealth for emergency healthcare ........................................ 13 2.4 Problems with current emergency solutions .......................... 17 2.5 Studies About Heart Conditions ............................................ 18 3 ARTIFICIAL NEURAL NETWORKS ....................................................... 27 3.1 Network Architecture ............................................................ 27 3.1.1 Single-Layer Feedforward Networks ........................ 28 3.1.2 Multilayer Feedforward Networks ............................ 28 3.1.3 Recurrent Networks ................................................ 29 3.2 Learning Tasks ...................................................................... 30 3.2.1 Pattern Association ................................................. 30 3.2.2 Pattern Recognition ................................................ 30 3.3 More Neural Networks Applications in Tele-cardiology .......... 31 3.4 Deep Learning ....................................................................... 32 4 WAVELET TRANSFORMS ................................................................... 38 4.1 Filter Banks ........................................................................... 39 4.1.1 Two-Channel Filter Bank ......................................... 41 4.1.2 Tree-Structured Filter Bank ..................................... 42 4.2 Applications of Wavelet Transform in Tele-Cardiology ........... 43 4.3 Filter banks and Neural Networks .......................................... 44 5 MODELLING THE CARDIOVASCULAR SYSTEM ................................... 46 5.1 Electrocardiogram (ECG) Signal ............................................. 48 5.1.1 Mapping the ECG Signal to Electrical Circuits .......... 49 5.1.1.1 Vessel Resistance ............................................ 50 5.1.1.2 Vessel Compliance .......................................... 50 5.1.1.3 Blood Inertia ................................................... 51 5.1.1.4 Valves ............................................................. 51 5.1.1.5 The Windkessel Model ..................................... 52 5.1.1.6 Mossa’s engineering model ............................. 52 5.2 Phonocardiogram signal (PCG) .............................................. 55 5.2.1 Definitions and Descriptions ................................... 57 5.2.2 Characteristics of the Heart Acoustic Wave ............. 60 X 6 MODELLING HEART ACOUSTIC WAVE PROPAGATION ....................... 63 6.1 Analytical Approach: Attenuation .......................................... 63 6.1.1 Discussion .............................................................. 66 6.2 Stochastic Approach ............................................................. 68 6.2.1 Discussion (Realisation of Durand’s Research) ........ 71 6.3 Summary .............................................................................. 76 7 THE EXPERIMENT ............................................................................ 78 7.1 Classification of Heart Conditions ......................................... 83 7.1.1 Simplified Approach (Using two features) ............... 83 7.1.2 Advanced Approach (Using Wavelet Transform) ...... 89 7.1.3 Using Neural Network for Classification .................. 94 7.2 Summary .............................................................................. 99 8 CONCLUSIONS .............................................................................. 101 8.1 Future Work ........................................................................ 102 REFERENCES ....................................................................................... 106 XI Figures Figure 1. Decision making after receiving the signal ..................... 7 Figure 2. Experiment steps ........................................................... 8 Figure 3. Single-Layer Feedforward Network ............................... 28 Figure 4. Fully connected Multilayer feedforward network .......... 29 Figure 5. Simple Recurrent network (SRN) ................................... 29 Figure 6. Steepest Descent Problem (local minima) ..................... 34 Figure 7. Simple Comparison between Shallow and Deep Neural Networks ..................................................................... 35 Figure 8. M-channel filter bank (Mertins, 1999) .......................... 40 Figure 9. Two-channel filter bank ............................................... 41 Figure 10. Tree-structured filter banks equivalent system. ............ 42 Figure 11. Diagram of the human heart (Creative Commons) ........ 47 Figure 12. P, Q, R, S, and T waves of ECG (Creative Commons) ..... 49 Figure 13. Hydraulic analogue of the cardiovascular system ......... 53 Figure 14. Closed-loop lumped model of the cardiovascular system. ........................................................................ 54 Figure 15. Heart sound signal (Creative Commons). ..................... 59 Figure 16. Wiggers Diagram (Creative Commons). ........................ 61 Figure 17. Propagation Model of heart acoustic wave (Analytical) .. 63 Figure 18. Travel model of pulse wave sounds in blood vessels. ... 66 Figure 19. Durand’s basic model of the heart/thorax acoustic system ......................................................................... 69 Figure 20. Heart acoustic signal measured at the chest (input) ..... 73 Figure 21. Realisation of Durand’s Model (1) ................................ 74 Figure 22. Realisation of Durand’s Model (2) ................................ 74 Figure 23. Simulation model of the stochastic approach – simplified method ........................................................................ 75 Figure 25. Heart-wrist acoustic propagation model (analytical approach) .................................................................... 78 Figure 26. Original Signal: Healthy Heart Acoustic Signal .............. 79 Figure 27. Received signal at the wrist (heart-wrist propagation mode) .......................................................................... 80 Figure 29. Comparison of the success rate around the SNR threshold. .................................................................... 88 Figure 30. Results of Advanced Approach – 8 features (using Filter Banks) ......................................................................... 91 Figure 31. Classification success vs. the number of levels (High SNR) ............................................................................ 92 Figure 32. Classification success vs. the number of levels (Low SNR) ............................................................................ 92 Figure 33. Visual demonstration of the impact of the number of hypotheses. ................................................................. 93 Figure 34. Neural Pattern Recognition Network Diagram ............... 94 Figure 35. Neural Pattern Recognition performance plot ............... 95 Figure 37. PCG-based Diagnosis Using Machine Learning (three methods) ................................................................... 100 Figure 38. Data Fusion-based Diagnosis using Machine Learning 104 XII Tables Table 1. Shallow vs. Deep Neural Networks ................................. 33 Table 2. Sound velocity in different media .................................. 62 Table 3. Acoustic attenuation ( ) of human body tissues ............ 64 Table 4. Acoustic properties for human tissue between heart and wrist ............................................................................. 67 Table 5. Calculating P(d) for S1 and S2 frequencies in different tissues .......................................................................... 68 Table 6. Classes for The Diagnostic System ................................ 82 Table 7. False classifications using simplified approach (scenario 2) .................................................................................. 87 Table 8. Snippet of the Classification Result. .............................. 97 XIII Abbreviations ADC Analogue-Digital Converters ADTree Alternating Decision Tree AED Automated External Defibrillator AI Artificial Intelligence ANN Artificial Neural Networks API Application Programming Interface bps bits per second CDMA Code Division Multiple Access CS Compressed Sensing CWT Continuous Wavelet Transform CVD Cardio-Vascular Disease DWT Discrete Wavelet Transform ECG Electro-Cardio-Gram ED Emergency Department EF Ejection Fraction EGG Electro-Gastro-Gram eHR Electronic Health Record ELM Extreme Learning Machine EMD Empirical Mode Decomposition EMR Electronic Medical Record ER Emergency Room FDSS Fuzzy Decision Support System FFNN Feed Forward Neural Network FFNNBP Feed Forward Neural Network Back Propagation FFT Fast Fourier Transform FIR Finite Impulse Response FNN Fuzzy Neural Networks XIV GPS Global Positioning System GSM Global System for Mobile HR Heart Rate HRV Heart Rate Variability IBC Intra Body Communications ICT Information and Communications Technology IFFT Inverse Fast Fourier Transform IIR Infinite Impulse Response IOM Institute of Medicine IoT Internet of Things LMMSE Linear Minimum Mean Square Error LPC Linear Prediction Coefficients LQG Linear Quadratic Gaussian MAE Mitral Annular Excursion MCI Mass Causality Incident MEDTOC Medical Data Transmission Over Cellular Networks MEMS Micro-Electro Mechanical System MLP Multi-Layer Perceptron MM Mini-Max Estimator MMSE Minimum Mean Square Error MV Minimum Variance MVU Minimum Variance Unbiased OCC One-Class Classification PCA Principal Component Analysis PCG Phono-Cardio-Gram PC Personal Computer PDA Personal Digital Assistant PDF Probability Density Function PSD Power Spectral Density XV PSO Particle Swarm Optimisation PR Perfect Reconstruction RBFN Radial Basis Function Networks REST Representational State Transfer SMS Short Message Service SNR Signal to Noise Ratio SPL Sound Pressure Level STFT Short-Time Fourier Transform SVM Support Vector Machine TAR Temporal Association Rules TSH The Scarborough Hospital UMTS Universal Mobile Telecommunications System VCG Vector-Cardio-Gram VPC Ventricular Premature Contraction WES Wearable ECG Sensor XVI Formulas XVII 1 INTRODUCTION “Creativity, as has been said, consists largely of rearranging what we know in order to find out what we do not know. Hence, to think creatively, we must be able to look afresh at what we normally take for granted.” — George Kneller eHealth is a relatively new terminology that represents the integration of electronics and communication in health systems. Moreover, mHealth (or mobile- health) is a more specific term to represent the applications of mobile devices (e.g. mobile phone) as a tool for communication, data processing, and positioning of health systems. However, eHealth is more general than mHealth. eHealth has been one of the hottest topics of biotechnology research areas. It is multidisciplinary type of research that includes information and communication technology (ICT), medical science, electrical engineering, computer science, embedded systems, etc. The field has been growing slowly but surely over the past two decades following the technology closely. This research looks back at the history of eHealth with a critical eye, the focus is particularly put on emergency healthcare. The decision to focus on emergency healthcare comes from the situation of the world nowadays, catastrophes are not rare anymore! The criticism is observed to identify the drawbacks of eHealth solutions for emergency healthcare. This research will highlight limitations in current solutions that affect their practicality in catastrophic situations. Namely, the design assumptions that stand between small-scale experiments of current eHealth solutions and their wide-scale applicability. The research then challenges these limitations utilizing a case study of cardiovascular system diagnosis. Having worked with cardiovascular system in master’s degree research, this case study was a clear choice. Challenging the limitations that are forced by design assumptions increases the chances of practicality of any of the eHealth solutions, this means more reliable solutions for emergency healthcare during catastrophes. The first step is very simple; while designing eHealth solutions, assume worst case scenario. And that is exactly what this research does. The research develops a method to diagnose heart conditions using noisy, corrupted, low quality and low power heart acoustic signals. It utilises signal estimation techniques to equalize the channel and restore as much as possible of the information held in the received heart sound signal. It will, potentially, lower 2 Acta Wasaensia the demands on the quality of biomedical signals and the processing energy, which will open the door for more realistic designs for emergency mHealth solutions. Although the focus here is on the quality of the data, but there are more design assumptions that should be addressed, such as: energy consumption, size of equipment, and many more that could be the focus of future research. In the rest of this chapter, the motive behind this research is revealed along with the scientific contributions and the method adopted to achieve the objectives. The chapter ends with thesis structure details. 1.1 Motivation With the tremendous growth of technology and the fast pace of life, the need for instant information has become an everyday necessity. One of the affected life aspects is health; the world is obsessed with monitoring vital signs and sharing results instantly not just with their doctors but also with their relatives. The doors were opened in the early 90s for repurposing Information and Communication Technology (ICT) for health, but the progress has been very slow due to the nature of the field and the sensitivity of the data (see chapter 2 for more details). My journey with ICT for health (also known as eHealth) started 8 years ago when I started researching wireless monitoring systems and hoped to build one myself, one that I called “The Vital Transmitter”. My goal was to build this system for my diabetic father. Mostly for me, to know his blood sugar level at any given time. For practical reasons the focus had to be shifted from reading blood sugar to heart rate, but the concept remained intact. The vital transmitter was built as part of my master’s degree research in 2012. It served its purpose; feature phones (non- smart) that triggers an SMS when the heart rate goes above a pre-calculated threshold or drops below another. This came from an Android app that compared the received heart rate to personal thresholds. The heart rate is received via low energy Bluetooth from a heart rate sensor and the thresholds were calculated using an algorithm that factors in age, gender, activity level, and accounts for sleep values. The algorithm was approved by a licensed cardiologist in Prince Sultan Cardiac Centre, Riyadh, Saudi Arabia. (Abdelmageed, 2012). Researching in this area has elucidated that, previously most of the ICT applications and systems built for health sector (eHealth), particularly eHealth solutions for emergency healthcare, assume perfect conditions. The system usually expects to deal with high quality data, and they are mostly designed with large, if not, multiple central processing units, which in turn consumes large amount of energy. Such systems fail to meet expectations when Acta Wasaensia 3 put to practice, especially in catastrophic situations. When data quality drops due to network status. The question that motivated this research is very simple; in case of catastrophes, when conditions are far from perfect, would these systems function? This question enticed the investigation and assessing the possibilities to making sense of low- quality data with low processing energy. However, this thesis is more focused on processing low quality data. 1.2 Objectives and contributions Listening to the heart sound using stethoscope is probably the oldest and fastest method to check the heart functionality (Health essentials, 2014). As many heart problems affect the way the heart beats in some manner, and since the heart acoustic is a result of the heart beating; it should hold valuable information about the causing problem. However, the human ear might not be able to distinguish small differences in the heart sound that indicate a disease, due to its low frequency that falls out of the spectrum range of human hearing or due its small power. Therefore, heart diagnosis using the conventional sound listening is not effective in most heart diseases (Health essentials, 2014). Thus, traditionally more sophisticated diagnostics and investigation methods are needed to fully report the heart condition, such as Electrocardiogram (ECG) and Echocardiogram. Nevertheless, most of these advanced methods require large devices and large power and cannot be implemented in small equipment’s like wearable watches or mobile phones. Consequently, this thesis proposes the application of advanced machine learning technologies to extract useful diagnostic information from the heart sound data. In this process, the entire sound spectrum should be used, including the bands outside of the human hearing range, and this process of data collection and processing should be performed with limited processing energy and limited capabilities, the likes of which mobile devices are capable of performing. The success of this concept will open the door for tremendous changes to the game of heart monitoring and diagnosis using wearable small devices like watches. Its task is to process low quality signals to obtain health diagnosis with acceptable error margin. The focus is alarming diagnostics and the goal is to save precious time at emergencies in less than perfect conditions (such conditions that are usually experienced in catastrophes). During the first couple years of research the following questions have emerged: In health diagnostics, how low could the quality of the data get before it is declared useless? 4 Acta Wasaensia Could signal classification techniques help make sense of distorted/corrupted biomedical signal? How reliable are the diagnostics that are based on processed data? And while this research responds to these questions, it had also raised few more that would leave the door open for future work. This thesis documents the process of going through the history of mHealth, focusing on emergency healthcare, and why it lacks practicality in catastrophes. Moreover, it explains in detail the experiment that answer the research questions. The key tasks and contributions of this research are: • To study the reliability of the pre-diagnosis found in low quality acoustic signal of the heart after processing, using different techniques (i.e. simple and advanced Machine Learning techniques and Neural Networks) to identify the information held in the signal. To study eHealth history and, specifically, mHealth tele-cardiology solutions that were discussed in recent years (2006 – 2018). To, potentially, lower the demands on the quality of the data for medical diagnostics, after proving the concept using the heart acoustic signals. To allow performing offsite diagnostics (during catastrophes/ in accident locations) despite tough circumstances that could include bad connections, small processing units and low/limited energy (mobile devices) in the future. To start the wave of measuring vital signals from unpredictable spots, which opens the door for new generation of mHealth products. The contributions of this thesis have been published/accepted/drafted in/by the following journals; - Abdelmageed, S. and Elmusrati, M. (2018). Phonocardiogram Based Diagnosis using Machine Learning: Parametric Estimation with Multivariant Classification. Bioscience & Engineering: An International Journal, Oct 30, vol. 5, no. 1/2/3/4, pp. 1-6 Available from: http://dx.doi.org/10.5121/bioej.2018.5401. (Published) Acta Wasaensia 5 - Abdelmageed, S. and Elmusrati, M. Machine Learning and Wearable Devices for Phonocardiogram-Based Diagnosis. 6th International Conference on Bioinformatics and Bioscience, May 2019. (Accepted) The following journal has been drafted to be submitted this year to a suitable journal in the field; - Abdelmageed, S. and Elmusrati, M. Wrist-based Phonocadiogram Diagnosis Leveraging Machine Learning (Drafted) 1.3 Methods Critical review was essential to identify the problems in current mHealth solutions for emergency healthcare. It was important to assess a large number of applications dedicated to emergency healthcare, to understand the proposed systems and challenge the described characteristics. This was the adopted method that led to defining the research problems. To stress the focus on alarming diagnostics, cardiovascular system was chosen as a case study. It was also the clear choice as a continuation after the master’s degree research (Abdelmageed, 2012). The experiment is performed on MATLAB/SIMULINK. The research is studying the reliability of the health diagnosis that is based on processed biomedical signal. This was achieved in six steps: STEP 1. Heart sound signal, also known as Phono-Cardio-Gram (PCG) was used. A healthy heart sound signal was used as a reference, this signal is called (S) in this section. This was used to bypass the difficulties of obtaining human heart sound signals; as this study takes a pure engineering approach, no ethical approval was required. STEP 2. A model of the cardiovascular acoustic wave propagation system was deduced from existing literature. The model should represent the propagation system (signal channel) from the heart (chest area) to the wrist. The wrist was chosen to leave room for future work related to wearable wrist bands that could carry out the task of diagnosing. Not to mention, that it is reasonably distant from the heart/chest area. STEP 3. The reference sound signal with added noise have been applied to the model from STEP 2. The received signal was then compared with the original 6 Acta Wasaensia signal to measure the effect of propagation system. The resultant signal is given by the equation Where is the received signal measured from the wrist, is the original signal ( measured from the chest) after the effect of the acoustic propagation system model, and is the added noise. It should be mentioned here that the channel model can be very complicated. For the following reasons: 1. The heart acoustic wave propagates over and through different matters, such as bones, blood, flesh, etc. Each one has different wave speed, propagation constant, and other characteristics that has a distinctive effect on the propagation. 2. The band of interest can vary in frequency from a fraction of one Hertz (e.g., 0.1 Hz) to several tens (e.g., 100 Hz). Within this band the wavelength can vary from 3 meters to 3000 meters. Hence, the propagation model characteristics may change considerably between different frequencies. However, since the modelling process is not the main topic of this thesis work, the chosen model was selected after studying several models that were proposed in the literature. However, it became evident that there is room for further studies in this topic of modelling. Depending on the level of distortion in the received signal, STEP 4 of the experiment will be decided according to the following routes. A clarification of this decision is shown in Figure 1. If the received signal is highly distorted and has low to no correlation with the original signal STEP 4. The received signal will be weak and highly distorted version of the original signal. Hence, the received signal ( ) should be used to restore the original signal ( ). This is called equalization process, which is common in different applications of wireless communications to equalize the channel impacts on the signal. There are many well-known equalization techniques. However, most of the conventional equalization methods might not work easily in this case because of the large nonlinearity caused by the large changes in the wavelength. Therefore, feed-forward neural network with backpropagation was used to retrieve the information held in the original signal, the resultant signal is denoted ( ). The error margin given by Equation (2) is calculated and kept as small as possible Acta Wasaensia 7 This process is repeated for every sound signal, where the resultant ( ) represents a hypothesis of a heart condition. For example, the healthy heart signal ( ) represents ( ) that is “healthy heart”. These hypotheses are used as references throughout the experiment. The plan is to define at least five hypotheses. Else, if the received signal correlates with the original signal STEP 4. The received signal ( ) is declared as hypothesis H0 as is, without restoration. Hypothesis H0 represents health heart condition. Figure 1. Decision making after receiving the signal Four more hypotheses are defined using different transfer functions, all are based on the original signal ( ), all used transfer functions are unique. More details on this in chapter 7. STEP 5. A number of cases are generated under every hypothesis, using different noise characteristics. Each case is then classified using the following techniques: Machine Learning; Parametric Estimation using Multivariate classification. This is explored at length, with different number of signal descriptive features. 8 Acta Wasaensia Artificial Neural Networks; Pattern Recognition Neural using MATLAB Toolbox. STEP 6. Artificial Neural Networks are used again to automate the diagnosing process. The ANN is trained to recognise the defined hypotheses from STEP 4, and accordingly every ( ) is put into the network to find the most probable diagnosis. This was a summary of the methods used in this research to achieve the discussed objectives. In Figure 2, the experiment steps are summarised and visualised. In the figure, the number of hypotheses is kept undefined to leave room for future growth and increased accuracy. Figure 2. Experiment steps Where is the original heart sound signal, in this case recorded at the chest, and is the heart sound signal as received at the wrist. Noise has different sources which can be internal; like lungs and respiratory sound, digestive system and specifically stomach sound. Or external sounds from outside the body that fall in the same frequency band of interest. Moreover, the noise generated by the sound recording sensor itself and the electronic devices used in measurements. is the received heart sound signal after restoring missed information, and last represent the resultant hypotheses of heart diagnosis. 1.4 Why focus on heart condition? The focus on the heart condition was because the heart beat is the first vital sign to check in emergencies. Detecting a heart pulse indicates that the emergency victim Acta Wasaensia 9 is alive whether it is a catastrophic situation or not. The next step is to diagnose the heart condition to determine the need for further cardiology treatment. A paramedic, or the medical personnel attending to an emergency, spends the most of their time eliminating life threats. Once they know the heart condition, they can find the proper way to restore its functionality to as normal as possible for the patient in question. Clearing the airway is just as important, however, having oxygen in the body will not help if the hear is not beating. Heart condition could have subtle symptoms to the patient, which makes it harder to identify and that adds to the necessity of cardiac diagnostic systems or at least systems that takes us closer to diagnosis. Not to mention that, early detection of heart diseases could tremendously reduce the mortality rate. 1.5 Thesis structure This thesis consists of eight chapters, each of which documents a part of the research and serves as a step towards the conclusion. Chapter one is an introduction to this research stating background and motivations that have driven interest towards this topic. Followed by the novelty of this research presented in the objectives and contributions. The research method is also presented in this chapter and then the structure of the thesis to help the reader keep up. In the next chapter, three years’ worth of literature review is laid out. Starting with defining Information and Communications Technology (ICT) for health and the story behind of the term “eHealth” and “mHealth”. The chapter then goes on to give an overview of the history of eHealth. Then, moves a step closer to the topic of interest by discussing eHealth solutions for emergency healthcare and stating the problems with current solutions, the goal here is to build a background for the next chapter and help the reader follow the train of thoughts that led to this work. The chapter continues to investigate tele-cardiology solutions presented in the current decade (since 2010). Tracing published papers that aimed to advance tele- cardiology; remote monitoring, heart signal analysis (ECG and sound), heart disease detection, heart disease diagnosis, and the list goes on. The selected papers were chosen based on relevance and date of publication. Chapter three is dedicated to Artificial Neural Networks (ANN); what it is and how it works and more about what it is used for in this research. This will be revisited in the experiment. 10 Acta Wasaensia Chapter four discuss wavelet transform and then focuses on filter banks and how they are used to extract features of the signal. These techniques will be revisited in the experiment. Chapter five discusses modelling the cardiovascular system. Starting with the mathematical model of the electrocardiogram (ECG) to explain the cardiovascular system function and circulation, this chapter summarises the essential learnings to understand the cardiovascular system in preparation for this research. Followed by the mathematical model of the phonocardiogram (PCG) as the focus of this research. Chapter six is where the heart acoustic wave propagation system is modelled. Few models are presented, and each model is discussed using MATLAB/SIMULINK custom designs to prove practicality. In chapter seven, the experiment is detailed, where the proposal is validated. The results are discussed and analysed at the end of every approach, with total of three approaches. This thesis is concluded in chapter eight, where the results are summarised. The eighth and final chapter presents an overview of the future work and next possible steps to expand this research. Acta Wasaensia 11 2 LITERATURE REVIEW This chapter summarises the available literature in the field of eHealth and specifically in the field of mHealth. The focus then shifts to eHealth/mHealth emergency medical solutions; because it is more relevant to the topic of this research. Extra attention is lent to heart related studies; as the main case study. 2.1 About eHealth and mHealth In the world of academics, eHealth is just another word for Information Communication Technology (ICT) for health. The term “eHealth” is relatively new; it was barely used before 1999. The term has been used to refer to Internet Medicine as well as any form of computer use in medicine. And just like most of the buzzwords (e.g. ecommerce, ebusiness) it was industry leaders and marketing people who first introduced the term “eHealth” rather than academics. (Eysenbach, 2001) It is rather late to try avoiding this term in the academic papers with more than 170 scientific journals already having it in their titles. Some of these journals are dated back to 1999, as referenced in MEDLINE index of biomedical journal literature. With all that, the need to scholarly define the term “eHealth” became urgent, yet almost impossible due to the nature of the science behind it. One JMIR Editorial Board member said “Stamping a definition on something like eHealth is somewhat like stamping a definition on 'the Internet': It is defined how it is used - the definition cannot be pinned down, as it is a dynamic environment, constantly moving.” However, one of the editors, named Gunther Eysenbach, had attempted a broad definition that hopefully encompasses eHealth dynamism “eHealth is an emerging field in the intersection of medical informatics, public health and business, referring to health services and information delivered or enhanced through the Internet and related technologies. In a broader sense, the term characterizes not only a technical development, but also a state-of-mind, a way of thinking, an attitude, and a commitment for networked, global thinking, to improve health care locally, regionally, and worldwide by using information and communication technology.” The referenced dynamism is evident through the history and progression of eHealth. This chapter traces the history of eHealth through the years discussing 12 Acta Wasaensia the reasons behind the slow adoption, then it sheds more light on eHealth solutions for emergency healthcare and at the end of this chapter a statement of the problems in current solutions is presented. These problems, in fact, are generic to all eHealth solutions, however, the focus here is emergency healthcare in catastrophic situations. mHealth is a more recent term, it is a component of eHealth that has not been standardly defined yet. In 2013, Global Observatory for eHealth (GOe) defined mHealth (mobile health) as “medical and public health practice supported by mobile devices, such as mobile phones, patient monitoring devices, personal digital assistants (PDAs), and other wireless devices”. mHealth employs the mobile phone’s core utility; voice and short messaging service (SMS) in addition to the more complex functionalities and application. This includes but is not limited to; third and fourth generation mobile telecommunications (3G and 4G systems), global positioning system (GPS), and Bluetooth technology (World Health Organization, 2013). 2.2 History of eHealth/mHealth The first thought of eHealth was in 1960s when the idea of electronic health records (eHR) were discussed, but that idea was not considered seriously until 1991 in the United States. Many recommendations were given to implement this system, but the lack of standards required for full interoperability increased the anticipated complexity of such system and led to the delay. eHR discussion rekindled in 1999 when the United States Institute of Medicine (IOM) reported that prescription and medication errors could be prevented through computerized order systems. Since then, standards’ developments and technological advances have driven substantial progress in the eHR (Miller, 2006). The same idea was implemented in Canada in the early 80s (Canada & Canada, 2003). However, it was less advanced. They were known in Canada as, Electronic medical records (EMRs) that was defined as digitalizing medical records. The 90s witnessed slow start in eHealth adoption and development, although many countries showed interest and all the findings indicated growing momentum for eHealth uptake (Meireles et al., 2013). Acta Wasaensia 13 The millennium brought strong growth to the field with ICT slowly but steadily getting integrated into health systems and services worldwide. In 2008, WHO ran another survey to study the progress of eHealth in Europe and the findings showed strong political will for eHealth across the European region. It also showed solid progress in implementing foundation actions towards the adoption and actual implementation of enabling actions (Miller, 2008). Looking back at the history of eHealth it is fair to say that its adoption is very slow. Despite the progress achieved in the millennial, eHealth took 20 years to reach where it is right now. And here is why it took so long: (Miller, 2008) The fragmented funding and governance of healthcare services. Resistance of professions to changes in existing models of care. Lack of rigorous research evidence on the benefits that might drive change. Reluctance of politicians to be seen to be tampering with a politically- sensitive service. Concerns about the costs and complexities associated with eHealth implementation. Concerns about how it will affect practitioners and consumers. There is a determined relationship between the progresses of eHealth and the country income group. Countries in the high-end and upper-middle income groups are more advanced in their eHealth development than those in the lower-middle- and low-end-income group countries. This explain why there is steady progress in European Union since 2002 and great success in Denmark since 1996 and Australia. Similar success would be expected in UK and US, but they had different reasons behind the slow adoption there: bureaucracy in UK and privacy concerns in the United States. 2.3 mHealth for emergency healthcare “Time is the most valuable thing to me.” — A family physician. Saving time is one of the major reasons why eHealth, and specifically mHealth, is appreciated by physicians and medical teams in emergency departments (ED). When the queue of patients in the Emergency Room (ER) grows, it is customary 14 Acta Wasaensia for the medical team to quickly assess the cases to prioritise and treat them efficiently. This assessment process is called triage, which refers to quick assessment of a patient in the Emergency Room (ER) with a view to define urgency of care and priorities in management. Triage evaluation can be completed in an organized and systematic manner relying on immediate visual and auditory assessment of appearance, breathing and circulation (Jayashree & Singhi, 2011). Triage process has evolved with the emerging technology, until eTriage came to light in 2008. It was presented as a new electronic triage application at The Scarborough Hospital (TSH), Ontario, Canada. This led to cutting triage time in half at the hospital and giving its multicultural patients the chance for more control over their reassessment while in the waiting room. This was part of the “Enhancing Emergency Services” project that combines an electronic application at the initial stage of triage process, with ongoing patient-controlled reassessment using kiosks located in the waiting room (Jones, 2008) . Before that and since 1998 (Jones, 2008) the same hospital has been tracking their patients’ case electronically at the emergency department. Another take at the eTriage was presented at the 7th International Conference on emerging Networking Experiments and Technologies (CoNEXT) that was held in December 2011 in Tokyo, Japan. The students proposed an eTriage system to handle mass casualty incidents (MCIs) such as earthquakes. The proposed solution was a wireless communication service platform consisting of electronic triage tags that combine a small vital sensor with a wireless device. The tag is attached to the casualty and their vital signs are gathered via the sensor and transmitted wirelessly via the ad hoc networks constructed by the electronic tag. The ad hoc networks were also used for localisation and local map generation to ease finding patients/casualties while monitoring vital signs (Jentsch et al., 2013). In 2012, a team in Norway presented eTriage under the umbrella of BRIDGE project that was funded by European Union’s Seventh Framework Program (SINTEF ICT, 2012). BRIDGE eTriage assists in marking and monitoring victims and in creating real-time situation awareness. It aims to ease the trigger’s task and bridge the process from triage to hospital admission. The system consists of 1) Triage Bracelet that is augmented with microelectronic components and various sensors. 2) a small device called Triage Relay that is intended to gather data from the incident field and transmit them to the in-charge person in the ER. 3) Triage Tablet that is used to visualise the triage data with localisation and local map. 4) more clip-on sensors to the patient’s body Acta Wasaensia 15 are added on-demand, such as heart rate, breathing rate, blood pressure, etc. Saving time could actually start before arriving at the ER! It has been proven that the first assessment of the patient/emergency case is crucial to their survival and that was the motive behind smart ambulances. The work on ambulance intelligence has started in 1996, when group of researchers at the National Technical University of Athens have successfully demonstrated real-time transmission of ECG data from a moving ambulance vehicle using GSM data links (Pavlopoulos et al., 1996). Soon after, a group of researchers at the University of Maryland Hospital have developed a wireless ambulance telemedicine system for stroke victims. However, applications were limited to storing and forwarding obtained bio signals although, in many emergency applications, real-time bio signal monitoring is needed (Pavlopoulos et al., 1998). In 1998, the ambulance project started in Athens University. The team developed a portable emergency telemedicine device that supports real-time transmission of critical bio signals as well as still images of the patients. The system allows a specialised physician to review critical bio signals and images of the patient and thus perform remote diagnosis and in return, provide specialized prehospital care. The transmission was all done over GSM networks, and it assumed a data rate of 9600 bps, which was the maximum at the time of the project (Pavlopoulos et al., 1998). In 2001, a team of two introduced a cost-effective portable tele-trauma system that assists healthcare centres in providing prehospital trauma care. They developed a software architecture with intelligent modules such as transcoding, differentiation, and congestion control to significantly improve the system transmission efficiency. The team claimed their system can accommodate much higher frame rates than the ones reported in previously proposed systems. They avoided high cost by using standard the available 3G wireless cellular data services, specifically, code-division multiple-access (CDMA)]. This tele-trauma system provides simultaneous transfer of video, still medical images, and ECG (Chu & Ganz, 2004). In 2013, a team of three developed a multifunctional telemedicine system for prehospital emergency medical services (Thelen et al., 2013). The system, like the others, allowed paramedics on the emergency scene to get a physician consult on the patient’s case and send the patient’s bio-signals along with still images to the hospital unit over GSM cellular network. The system was designed with user participation to assure practicality, and although built by engineers, the involvement of emergency physicians was significant. Inside the ambulance 16 Acta Wasaensia they’ve put a video camera, printer, and transceiver. Paramedics can take a monitor, defibrillator, headset and a portable transceiver outside of the ambulance. On the hospital side, where the tele-consultation centre is, they have a desktop application for displaying the scene and its data, they have touch-control panel and of course a headset to carry conversations with the paramedics. Transmission Control Protocol/Internet Protocol (TCP/IP) network traffic is established between the ambulance and the centre, where the transceiver is the gateway. All devices inside the ambulance are connected to the transceiver over the ambulance Local Area Network (LAN), while devices outside of the ambulance are connected via Bluetooth. Then wearables came to the picture, ranging from micro sensors that are seamlessly integrated in textiles through consumer electronics, like sensors that are embedded in fashionable clothes and computerized watches to belt-worn personal computers (PCs) with display units. This technology made it possible to detect and alert unobtrusively. In 2003, an advanced care and alert portable tele- medical monitor (AMON) was developed. It is a wearable medical monitoring and alert system that targets high-risk cardiac and/or respiratory attacks. It includes continuous collection and evaluation of multiple vital signs and it is equipped with intelligent multi-parameter medical emergency detection that is connected over the cellular network to a medical centre. The system was integrated into a wrist- worn device. This wrist band was designed with low-power techniques, which made continuous long-term monitoring possible and less restricting to the patients’ mobility (Anliker et al., 2004). Following that, another solution for prehospital assessment was delivered. The system is a third-generation universal mobile telecommunications system (UMTS) based and it delivers biomedical information from an ambulance to a hospital. This system transmitted voice, real- time video, electrocardiogram signals, and medical scans in a realistic cellular multiuser simulation environment (Gallego et al., 2005). Recently, the need for emergency solutions have been on the rise due to the world population and the life pace that causes more accidents than it solves. This is evident in the number of researches focusing on building smart ambulance and improving on emergency response rates and the like. Another group developed a smart ambulance system, although their motive was to improve the emergency response rate in India their solution is applicable globally. They (Gupta et al., 2016) explored advancing ambulance systems over Internet of Things (IoT), by collecting location coordinates via Global Positioning System (GPS) and using Google Maps Application Programming Interface (API) to plot the locations of ambulance cars, which could also be done for hospitals, in a way, they built an Uber-like service for ambulances. Patient’s health data is collected via medical equipment available on Acta Wasaensia 17 the ambulance and is sent to the hospital. The communications between the Android app and the database server at the hospital uses Representational State Transfer Application Programming Interface (REST API). Their idea is not new, however, they focused mostly on the mobile application rather than stationary system on the ambulance car. In early 2016, Hooman Samani and Rongbo Zhu developed a robotic automated external defibrillator (AED) to address cardiac arrest emergencies (Samani & Zhu, 2016). To survive such cases of cardiac arrests, patients need supervised AED within minutes of the occurrence. The robot (Ambubot) gets dispatched to the victim of a cardiac arrest as soon as detected via a sensor attached to the patient or active call via the mobile app. Both means send the GPS location to the robot, along with more information about the patient. The system also informs the family and calls for an actual ambulance. Later in 2016, a team of three (Kumar et al., 2016) have focused their efforts to develop a system that monitors the condition of elderly people using micro-electro mechanical system (MEMS) connected wirelessly to heart beat, body temperature, and vibration sensors. The parameters that describe the condition of the person are sent to an Android app via Bluetooth protocol. When emergency hits, the system sends a message to the server via GSM that includes the GPS location of the user. The novelty of the system is that it searches for the nearest ambulance and sends it to the address of the user, it also sends an SMS to a predefined relative. One research is worth mentioning as it focused on handling emergencies during disasters (Khoumbati et al. 2010). They developed a scheme called Medical Data Transmission Over Cellular Networks (MEDTOC), they transfer patients’ vital signs from the ambulance to the hospital over UMTS. The novelty is that they are aggregating the data of multiple patients; using special packet format that orders the data. After reviewing these trials, projects, and studies, it is obvious that the research to utilise eHealth for emergency healthcare and prehospital care operations is not near done. However, some of these solid trials are still in progress and yet to encourage full adoption. 2.4 Problems with current emergency solutions It is evident in the history of eHealth for emergency healthcare, particularly, the projects and trials dedicated to this field that many systems fail wide-scale practicality. Because in the design stage, most of these systems made big 18 Acta Wasaensia assumptions that were never matched back to reality. And while, assumptions are quite important at early stages of design, it is customary in design tasks to generalize and/or address assumptions at the end of trials. One could use assumptions to put the system in context, define use-cases, and build test-cases. However, these assumptions might force limitations on applicability, which might lead to impracticality in worst cases. These systems have failed to address their assumptions and, unfortunately, will end up being a liability instead of being a reliable system during catastrophes. This is axiomatic in eHealth solutions for emergency healthcare. One crucial limitation in such systems is assuming perfect conditions. For instance, large processing units, full coverage of wireless service, high speed Internet connections, low to no noise. In short, it presumes perfect transmission and perfect reception, where the message is preserved throughout the process. Unfortunately, this is seldom the case in many developing countries and rarely is the case in developed countries. Having any of the perfect condition characteristics is a luxury when in catastrophes and chaotic environments. And it is during catastrophes that people rely on emergency healthcare systems the most. That is why this assumption is dangerous; the result is building impractical systems that fail at the very first test. This is the problem that this research is trying to address; in less than perfect conditions, how reliable can mHealth solutions for emergency healthcare be? 2.5 Studies About Heart Conditions Due to the importance of the heart condition, many researches were dedicated to build monitoring systems, find measuring methods and analysis techniques. The goal of these researches is to speed up the process of diagnosing the heart condition, the difference is mostly in the degree of dependence on the doctor or the medical personnel. This area of eHealth is known as “Tele-cardiology”. In this chapter, some of the work that was published in the current decade is traced; in such fast-growing field of research looking beyond this decade would not be of worth. The papers discussed here were selected based on relevance to this research and are presented chronologically. In 2010, Tovar et al. presented their work of diagnosing heart murmurs by analysing phono-cardio-graphic (PCG) signal by proposing joint time-frequency distribution and present it in time-frequency map. Although, the team used a simulated signal to build and test the method, they applied their method to real Acta Wasaensia 19 patients’ signal obtained from a hospital (Tovar et al., 2010). The three patients in question had different condition severity; one healthy and two with medium- severe and severe, respectively. The results were satisfactory. In the same year, Sufi et al. proposed a mobile phone tele-cardiology system. They used five sample points of the QRS Complex, specifically the centroid and four extreme points on the cardioid of the QRS Complex (Sufi et al., 2010) to identify cardiac abnormalities instead of the usual hundreds, this has led to a faster and more efficient mechanism of cardiovascular disease (CVD) detection from ECG signal. The mechanism causes less computational burden that known mechanisms at the time, which made it suitable for wireless mobile phone based tele-cardiology applications. Also, that year, Tang et al. developed a method to separate heart sound signal from noise utilizing joint cycle frequency–time–frequency domains. In practice, they decomposed the heart sound signal into small components by means of a Gaussian modulation model, these components were characterized by time delay, frequency, amplitude, time width, and phase. These components assemble in the joint domains (Tang et al., 2010), while the noise component disseminated and with that, they managed to separate the heart sound signal components from noise based on fuzzy detection. The noise was simulated as non-Gaussian, nonstationary, and coloured noise. In 2011, Ding et al. took over the task of lowering the power consumption required for sensing the heart signal (ECG), this was done by introducing a novel method, compressed sensing (CS), to wearable ECG sensor (WES). In practice, the team sampled the analogue signals at sub-Nyquist rate at the analogue-digital converters (ADCs). The task was to classify the compressed measurement into normal and abnormal state rather than an actual diagnosis of the heart arrhythmia, they used wavelet transform for anomaly detection. When a cardiac anomaly was detected (Ding et al., 2011), the signal is stored in a memory and is then transferred to a cardiologist for further diagnosis of cardiac arrhythmias using the reconstructed signals from the compressed measurements, this step is done off-line and out of the system. The results showed that the method reduced the power consumption with 34% (Ding et al., 2011). In the same year, Su et al. developed an ECG analysis algorithm based on wavelet transform. To diagnose the heart condition, the algorithm locates the position of Q, R, and S wave in an ECG signal (Su et al., 2011) QRS waves hold great information about the heart condition; such as identifying cardiac arrhythmias by counting the number of QRS in a minute. The team have declared this as a noise regardless, effective and efficient process. 20 Acta Wasaensia Also, in that year, Nasrabadi and Kani developed a low budget phone-based ECG acquisition, analysis, and visualising system. It consists of microcontroller that mimics Holter device to read the ECG signal; electrodes in the chest area. The signal is then transmitted to mobile phone via Bluetooth protocol for display and analysis, for that they built a J2ME app. The analysis is based on locating the position of the QRS waves (Nasrabadi & Kani, 2011). In 2012, Mandal et al. developed a system that acquires the heart sound signal from the chest area, the signal goes directly to the connected PC where they deploy discrete wavelet transform to remove internal noise and reconstruct the de-noised heart sound signal. The system then uses a novel algorithm “end point detection” to detect nature of heart sound components M1, T1 of S1 and A2, P2 of S2, their locations, durations, frequencies present, length of cardiac cycle (Mandal et al., 2012). In the same year, Kumar et al. proposed a method to de-noise ECG signals using a hybrid technique. In practice, they combined Empirical Mode Decomposition (EMD) with wavelet thresholding (Kumar et al., 2012). They used EMD to decompose the signal and soft wavelet thresholding to remove the noise from the decomposed signal. The de-noised signal is then reconstructed from the series of intrinsic mode functions (result of decomposing). And, in the same year, Uslu & Biglin used local discrete Fourier transform to extract ECG signal features that help diagnosing the heart condition (Uslu & Bilgin, 2012). The locality based DFT is, in fact, deploying DFT after partitioning the signal into smaller frames, each frame consists of some samples, and then obtain the sequence in question by shifting window structure iteratively (Uslu & Bilgin, 2012). In 2013, Meireles et al. investigated new technique for heart diagnosing based on spatial recording of electrical heart activity, known as Vector-cardiogram (VCG) (Meireles et al., 2013). The idea is presented as a portable solution that records VCG and use digital signal processing for diagnosing the heart condition, particularly, Myocardial Infarction. They also study the possibility of converting VCG to 12-lead ECG. They used classical Finite Impulse Response (FIR) and Infinite Impulse Response (IIR) for noise cancellation. In the same year, Ishanka et al. developed a software tool to detect cardiac anomalies using heart sound (Perera et al.,2013). The heart sound signal is recorded from four locations in the chest area using electronic stethoscope. The signal is then de-noised using wavelet and decomposition. The detection is done by deploying few different algorithms (Perera et al., 2013). Acta Wasaensia 21 Moreover, in that year, another team realised the necessity for ECG classification systems and proposed a method for classifying ECG arrhythmias (Sarma et al., 2013). The proposed system uses artificial neural networks. In practice, they used fast Fourier transform for pre-processing the signal followed by linear prediction coefficients (LPC) and principal component analysis (PCA) for extracting the signal features. The actual classification is done using multilayer perceptron (MLP) artificial neural networks. Another research group did similar work, in the same year (Patel & Joshi, 2013), they developed an artificial neural network-based system for heart disease classification. They focused on stroke stage classification using multilayer feed forward network with back propagation learning algorithm. In 2014, group of researchers (Sani et al., Dec 2014) aimed to reduce the number of heart attack victims by proposing a framework for remote monitoring of heart attack diagnosis system for ambulatory patient. The system continuously monitors the cardiac markers of the patient and generates an alarm (SMS or call) when it reaches a predefined threshold. In practice, they used biosensors to detect the markers from the blood. The detected values are transmitted wirelessly utilising mobile phones/PDAs (over cellular networks) to the hospital system for storage and analysis. In the same year, Jabbar et al. hoped to make a difference in India and reduce the adverse reactions caused by not diagnosing heart diseases early enough. They developed alternating decision tree (ADTree) for early diagnosis of heart disease (Jabbar et al., 2014), this approach was a new type of classification rule at the time. It is based on decision tree; a data mining technique usually used for decision support process and machine learning. The team used principal component analysis (PCA) to gather features of the disease. In that same year, Alsalama and her supervisors tried using radial basis function networks (RBFN) with Gaussian function to classify heart diseases (Alsalamah et al., 2014), which is a learning system that reuses training datasets to reduce false classifications. And in the same year, group of researchers (Thiyagaraja et al., 2014) took over the task of developing a smart phone application to detect, monitory and analyse the split (delay between its two components) in second heart sound (S2). The heartbeat is recorded using a stethoscope. The team used fast Fourier transform to convert the sound signal to frequency domain and detect the first and second heart sound (S1, S2). They then use discrete wavelet transform to extract S2 and continuous wavelet transform to detect the characteristics of the heart sound 22 Acta Wasaensia signal; Aortic (A2) and the Pulmonic (P2). Those characteristics are used to calculate the split in S2. The application offered continuous monitoring and low- cost detection tool. Another group of researchers in the same year built a diagnostic system for heart diseases using fuzzy classifications technique (Krishnaiah et al., 2014). They modified uncertain unstructured data into “fuzzified” structured data using minimum Euclidean distance fuzzy K-NN classifier embedded with Symbolic approach and then classified the data. A team in Indonesia in that same year, built extreme learning Machine (ELM) based neural networks to diagnose heart disease (Fathurachman et al., 2014). This system is meant to overcome the long process of training neural networks, it is thought to be fast and require simple tuning. In the same year, group of researchers focused on diagnosing rheumatic heart disease using a mobile phone connected to stethoscope for auscultation, hoping for a cost-effective detection with no need for expert training (Springer et al., 2014). They used signal quality estimation techniques to overcome the limitation of the device primitiveness. Particularly, they used support vector machine (SVM) classifier with Gaussian kernel as a binary classification algorithm (good and bad quality). For sound segmentation, they used modified hidden semi-Markov models. Cabral and Oliveira worked on their own heart disease classification tool in that same year (Cabral et al., 2014). At the time, machine learning techniques have proven to be important tools for diagnosing several diseases and they aimed to find patients who are prone to cardiac disease before they show symptoms. They analysed medical data for cardiac diseases using five methods based on one-class classification (OCC) paradigm; kernel principal component analysis, feature boundaries detection, support vector machine, support vector data description, and Gaussian process OCC. For optimisation, they used particle swarm (PSO). In 2015, Jabbar et al. gave another try to heart disease diagnostic systems, this time using computational intelligence technique (Jabbar et al., 2015). They used discretisation method and genetic search to remove redundant features and optimisation, it is an enhancement to Naïve Bayes classification. Naïve Bayes is linear and probabilistic classifier that is based on Bayes theorem; all features are independent, and presence or absence of a disease depends on a feature itself (Jabbar et al., 2015). Acta Wasaensia 23 In the same year, another group of researchers tried to help physicians avoid misdiagnosing heart patients with an intelligent system (Olaniyi et al., 2015). It is modelled on multilayer neural network trained with backpropagation and simulated on feedforward neural network. They normalised the data before inputting it to the system; dividing each sample of a feature by the corresponding highest sample value. Another team used extreme learning machine (ELM) algorithm in that same year to model the independent factors leading to the diagnosis of a heart disease (Ismaeel et al., 2015). It is a warning system for probable presence of heart disease based on collected information about the patient; age, sex, serum cholesterol, blood sugar and more. In 2016, a team combined Naïve Bayes classifier with temporal association rules for coronary heart disease diagnosis (Orphanou et al., 2016). The features of the heart signal were temporal association rules annotated with the possible recurrence patterns of those features. They relied on several temporal data mining methods to analyse the signal; periodic temporal association rules (periodic TARs). In the same year, Kalaiselvi diagnosed heart disease using average K-nearest neighbour algorithm of data mining in a solo research (Kalaiselvi, 2016). The proposed algorithm is used to predict the heart disease with reduced number of attributes that are relevant to the disease. Moreover, in the same year, a team attempted developing a real-time automatic assessment of cardiac function in echocardiography (Storve et al., 2016). The system focused on estimating mitral annular excursion (MAE) and tissue Doppler parameters on cardiac ultrasound recordings to assess the heart condition. In the same year, a team proposed genetic algorithm based fuzzy decision support system (FDSS) for predicting the risk level of a heart disease (Paul et al., 2016). They pre-process the dataset, then use different methods to select effective attributes, these attributes help generating the fuzzy rules using genetic algorithm, which are used to build the FDSS that predicts the heart disease. Also, in that year, Feshki and Shijani worked on improving the heart disease diagnosis by using machine learning; particle swarm optimisation (PSO) and feedforward neural networks backpropagation (FFNNBP) (Feshki & Shijani, 2016). In practice, they used feature ranking on effective factors of disease by PSO and FFNN backpropagation. 24 Acta Wasaensia In the same year, Shi et al. developed a wireless stethoscope for recording heart and lung sounds (Shi et al., 2016). The goal is to ease continuous monitoring using stethoscopes. In practice, they developed a new method for analysing acoustic properties of the heart and lung sounds. After digitising the sound signal, they extract cardiac action parameters for analysis, combining this with lung sounds they got good insight into the cardiac and respiratory function. The signal was transmitted wirelessly to a receiver module that digitally filters the data and normalise the amplitude scaling. On the receiver side, they performed analysis over the acoustic properties of S1 and S2. In 2017, an attempt was made to reduce the time required to diagnose a heart failure (Manikandan, 2017) . This heart attack prediction system used a dataset from UCI Machine Learning Database, pre-processed the dataset using Rapid Miner and then investigated several algorithms to build the classifier, such as; Naïve Bayer, Decision Trees, K-Nearest Neighbour and Random Forest. It concluded that Naïve Bayer is the most fitting, where it resulted in 81.25% accuracy for the prediction. The system used 14 features to predict the failure. In the same year, a team proposed a Support Vector Machine (SVM)-based heart rhythm classifier. The system uses features like; timing, morphology, and spectral characteristics of the ECG to perform multi-source features and SVM for Atrial Fibrillation (AF) (Liu et al., 2017) . Also, in 2017, a novel method was proposed for heart sound classification without segmentation using Convolutional Neural Network (CNN) (Zhang & Han, 2017), where the different positions of the heart cycles are intercepted from the heart sound signals during the training phase. The spectrograms of the intercepted heart cycles are then scaled to a fixed size and input into the designed CNN architecture to generate features of different start positions in the testing phase. This has reduced the importance of the sound segmentation for prediction, the method was proven to be competitive when evaluated using public datasets. In 2018, the even detection approach based on deep recurrent neural networks was used to detect the position of state-sequence in a segmented heart sound (Messner et al., 2018) . Using this method, the researcher managed to detect the position of the first and second heart sounds (S1, S2) in heart sound recordings without incorporating a priori information of the state duration, which was also applicable to recordings with cardiac arrhythmias and extendable to detect extra heart sounds (S3, S4). Towards middle of 2018, a team attempted to automate the detection of abnormality in the heart sounds (Karaca et al., 2018). The processed Acta Wasaensia 25 phonocardiogram signals were classified to normal and abnormal using K-nearest neighbour method, with high accuracy that reached 98.2%. If there is anything in common between these researchers aside from their area of interest, it is the advancement of tele-cardiology as a result of their work. However, the level of advancement differs from one to another. Majority of these papers focused on heart disease diagnosis, whether by analysing ECG signal (electrical reflection of the heart) or the heart sound signal. Analysing ECG signal has been the core of tele-cardiology as the simplest form of representation as opposed to heart sound signal, which is far from being an easy interpretation. In fact, heart sounds interpretation is very subjective to the cardiologist’s experience and hearing ability (Health essentials, 2014). The researchers focused on reducing the time required for diagnosing heart conditions; by reducing the number of attributes to characterise a disease. Or optimising the extraction process of the rules around that disease. Or removing irrelevant attributes from the process. However, few researchers payed attention to the need to address noisy data, although noisy signal, whether ECG or sound, would hinder the diagnostic process massively. Noise that is caused by stethoscope friction against the skin and/or lungs’ respiration function is one thing but there is also the noise added once the signal is transmitted wirelessly. By now (see chapter 2) it has been established that perfect conditions are very rare, especially during catastrophic epidemics. And although none of the presented papers considered the practicality of their solutions during such circumstances, majority have presented mechanisms and algorithm optimisation rather than full systems compared to few who attempted developing an end-to-end solution. Furthermore, when a full system is developed, perfect conditions are assumed throughout the processes except for the case when a new device is proposed; researchers seem to avoid assumptions to maintain credibility of newly designed devices. However, the focus is usually put on removing noise more so than any other factor. The missing piece of the puzzle is clearly in addressing the conditions of the whole process; signal acquiring, signal analysis, signal transmission, and signal diagnosis. Each of these steps is a factor in the data quality (completeness, precision, validity, accuracy, consistency, timeliness, reasonableness, conformity, and integrity). They also factor in the power consumption (amount of energy consumed in every process), power consumption should be addressed in case of AC adapter sources and more so when it is battery-based. Not to mention, Speed and quality of the Internet connection used to transfer data between stations of the system. 26 Acta Wasaensia This research attempts to break the limitations rather than ignore them, by finding the sweet spot of low-quality data. Instead of assuming perfect conditions and highest quality, this research simulates low-quality signal and proposes methods and techniques to compensate the impact of the chaotic conditions. Acta Wasaensia 27 3 ARTIFICIAL NEURAL NETWORKS “Artificial Neural Networks (ANN) in the most general form is a machine that is designed to model the way in which the brain performs a particular task or function of interest” (Haykin, 1999) Modelling the heart acoustic propagation system is not an easy task, due to the complexity of the heart acoustic signal and the nature of the cardiovascular system. Moreover, understanding the impact of the organs in the human body on the sound and how the quality of the sound could affect the interpretation of the resultant sound signal, is all adding to this complexity. Therefore, the ICT applications in this field have been quite modest and mostly used machine learning and neural networks to approach the level of the advanced human brain that can be taught to interpret these complex signals. This is why it is important to study Artificial Neural Networks, and specifically their applications in heart diagnostics and heart health in general; neural networks will be used in this research to recognise patterns found in the heart acoustic signal to be able to classify each sound signal into a hypothetical disease. Hence, it is important to understand what is classification? And how could neural networks be used to classify a signal? And what are its applications? Classification is simply grouping things (in this case signals) based on similarity in their features and characteristics. Classifying objects is a survival instinct, animals must distinguish between threats, food, and potential mates. The brain often learns by association, quickly finding features that resembles known experiences and that is how it adds new objects to existing classes. Consequently, a neural network has to do the same job using neurons, their connections and arrangement. Next, the discussion goes into architecture of neural networks then will continue with the learning tasks focusing mostly on classification and pattern recognition. 3.1 Network Architecture The learning algorithm used to train a neural network is strongly connected to the structure of which the network that connects the neurons is like. There are different network structures that could be adopted, and the following is a summary of the classifications of these architectures (Haykin, 2009). 28 Acta Wasaensia 3.1.1 Single-Layer Feedforward Networks In this architecture, the neurons are structured to form layers. This network is rigorously a feedforward (acyclic) type. For instance, an input layer of source nodes projecting onto an output layer of neurons as computation nodes, but not vice versa. The term “single-layer” refers to the output layer of the neurons (computation nodes), input layers of source nodes are not counted since no computation is carried out there. Figure 3 shows an example of single-layer feedforward network with four nodes on input and output layers. Figure 3. Single-Layer Feedforward Network 3.1.2 Multilayer Feedforward Networks This architecture has hidden layers. The corresponding computation nodes (neurons) are called hidden neurons/units. The purpose of hidden neurons is to usefully interfere between external input and the network output. When a hidden layer or more is added to the network, it can extract higher-order statistics. Such network is considered fully connected, because each node in every layer in the network is connected to every node in every adjacent forward layer. If any synaptic connection is missing, the network is considered partially connected. Figure 4 shows an example of this type of architecture. The middle layer is hidden neurons. Acta Wasaensia 29 Figure 4. Fully connected Multilayer feedforward network Therefore, Multi-layer Feedforward Neural Networks are very suitable for complex classification such as heart signals, where the hidden layers compensate for the intermediate interconnected layers of the human brain and that made this architecture commonly used in pattern recognition (Fine, 1999) (Haykin, 2009). 3.1.3 Recurrent Networks This architecture has at least one feedback loop as opposed to only forward feedback in the previous architectures, which might have hidden layer or not. The feedback loop could be called self-feedback when the neuron feeds back into its own input. The feedback loops have a direct impact on the learning capability of the network, in addition, it implicates unit-delay elements that may lead to a nonlinear dynamical behaviour when the network contains nonlinear units. Figure 5 shows an example of this type of network. Figure 5. Simple Recurrent network (SRN) RNNs were developed in the 80s, and later many variations were developed to serve different purposes. Among which is Long Short-Term Memory (LSTM) that was proposed in the late 90s by Hochreiter and Schmidhuber. In 2018, a recurrent 30 Acta Wasaensia network was used to detect the position of the first heart sound (S1), also known as systole, and the second heart sound (S2), also known as diastole without using any a priori information about the state duration (Messner et al., 2018). 3.2 Learning Tasks The choice of a learning algorithm from the above is swayed by the learning task that the neural network is built to perform. There are at least six learning tasks that could be the purpose of neural networks, in this research the focus is put on pattern recognition (Haykin, 2009) . 3.2.1 Pattern Association This task expects the neural network to learn by association, more like the human memory. This could be auto-association, where a neural network is required to store set of patterns by repetitively showing them to the network using unsupervised learning. Or hetero-association, where an arbitrary set of patterns is paired with another arbitrary output patterns using supervised learning. 3.2.2 Pattern Recognition This task expects the neural network to assign a predefined classification or categorisation to a received pattern. This is done by enduring a training session, where sets of inputs along with their categorisation is presented to the network repetitively. Pattern recognition requires removing unwanted data/information from the input as much as possible, to reduce the error margin. This is known as denoising the signal, which is done by filtering. The filtering algorithm or method depends on the application, for instance for heart acoustic signals this could be, and most commonly is, Kalman Filter (Welch & Bishop, 2006) (Salleh et al., 2012). In rare cases when the signal-to-noise ratio is acceptable, this step is skipped. Next, the signal should be processed to identify unique features that distinguish one signal from another; this process is known as feature extraction. In the case of heart acoustic signals, this is about deriving the features of the disease. There are several methods and algorithms to extract features from signals; for example, Filter Banks and application of wavelet transform (Liung & Hartimo, 2002) (Tovar et al., 2010) , more about this topic in next chapter. Acta Wasaensia 31 The final stage of the pattern recognition is the classification, where the inputs are matched to particular category/class. The feedforward neural network is used widely in classification of signals, especially biological signals. There are many applications of this, for example, using deep feedforward neural networks, also known as convolutional neural networks, to classify heart sounds (Patel & Joshi, 2013) (Zhang & Han, 2017). 3.3 More Neural Networks Applications in Tele- cardiology The following applications show the impact of neural networks on tele-cardiology, and more specifically. And more so, the importance of feedforward neural networks in the field of classifying heart conditions. This section sheds light on relevant applications in the past three years. In 2016, feedforward neural network was used to predict the heart rate of a cyclist based on cycling cadence (Mutijarsa et al., 2016). The goal was to overcome the limitation of wearable sensors that do not read at regular intervals (e.g. 1 second, 2 seconds) by predicting the heart rate and complete the missing data. This should allow the cyclist to control the intensity of cycling and help them avoid risks of overtraining and heart attack. The feedforward neural network is used to model the mathematical relationship between the heart rate and cycling cadence. It expected the heart rate, the cadence on the second as input and gave the predictive value of the heart rate on the next second as output. They used large dataset for training and only 1% of the dataset size for testing, the experiment resulted in close prediction to measured heart rate values, with mean absolute error is 2.43 and 3.02 of training and test data, respectively. In 2017, two-layered perceptron neural network was used to analyse time- frequency cardio-rhythm-o-gram signal parameters using real-time heart rate value (Melnik et al., 2017). In another research in the same year Electrocardiogram (ECG) was used to detect several heart abnormalities, the accuracy of the ECG was improved using an artificial neural network with self-learning algorithm (Rastgar-Jazi et al., 2017). They used Main Lead II for extracting the features of the abnormalities in the ECG signals, which is known for feature extraction from ECG. Learning algorithms were not discussed in this research as they are out of scope, however, this does not reduce the relevance of this research. 32 Acta Wasaensia In 2018, a team analysed the spectral and statistical features of the Heart Rate Variability (HRV) signal to diagnose Obstructive Sleep Apnea (OSA) (Ali & Hossen, 2018) . HRV is a relatively new way to track well-being; it is a simple measurement of the variation in time between heartbeats (Campos, 2017). They used multiple artificial neural networks, including; single perceptron network, feedforward network with back-propagation, and the probabilistic neural network. The highest performance was achieved by feedforward network with back- propagation using wavelet-based frequency domain features with specificity, sensitivity, and accuracy of 90%, 100%, and 96.7%, respectively (Ali & Hossen, 2018). Another research in the same year, worked on classifying the fetal heart rate using convolutional neural network (Li et al., 2018). In their work, they divided the fetal heart rate into three classes; normal, suspicious, and abnormal. Then, they obtained records for each category from the hospital, and segmented each record into ten d-window segments and used convolutional neural network to process the data in parallel. And at the end, they used the voting method to determine the class of the record. Additionally, and to conduct a comparative study, they repeated the experiment using basic statistics feature extraction method and input the features into Support Vector Machine and Multi-layer perceptron to classify. Ultimately, the results had higher accuracy when using convolutional neural networks (Li et al., 2018). 3.4 Deep Learning Deep learning is new area of Machine Learning that was introduced as a step towards Artificial Intelligence (AI), it uses data to learn what was only thought to be possible for humans. This includes, perception; content recognition, prediction, and classification. It goes beyond simple learning algorithms to understand natural language and written documents, which makes it capable of making new discoveries. It is thought to be world changing science, especially for healthcare. It is not a separate science from Artificial Neural Networks; Deep Learning is a set of techniques that were discovered as an improvement to Neural Networks’ learning techniques. Table 1 below shows a simple comparison between Shallow and Deep Neural Networks (EDUCBA, 2018), although it is oversimplified, it gives a clear distinction between the two. Nonetheless, both are a class of machine learning algorithms where the artificial neuron forms the basic computational unit and networks are used to describe the interconnectivity among each other and can have units in multiple layers for feature transformation and extraction. Acta Wasaensia 33 Table 1. Shallow vs. Deep Neural Networks However, there is a known problem in deep neural networks that is Steepest Descent, also known as vanishing gradient problem, which was limiting the depth of Neural Network severely. Neural Networks are trained using backpropagation gradient descent, that relies on updating the weights of each layer as a function of the derivative of the previous layer. However, the update signal was lost as the 34 Acta Wasaensia depth is increased and that is a problem, which causes two issues; local minima and saddle point. Local Minima can be explained as follows, during the iterative optimisation algorithm to minimise the loss function (i.e. how far is the performance of the network from being perfect) the Network finds a local minima and it stops optimising, while in fact the optimal performance (real minima of the loss function) has not been reached yet. Local minima problem is visualised in Figure 6. Saddle Points is a similar concept with local maxima in the other direction of the local minima. The network also has a learning rate that dictates the size of the step taken from one iteration to another when seeking optimal performance. Figure 6. Steepest Descent Problem (local minima) To mitigate this, Neural Networks were limited to smaller number of hidden layers and in some cases, they were preferred to have no loops; they were either feedforward or recurrent; because the more hidden layers you add to the network the worse the steepest descent problem becomes. However, in 2006 unsupervised pre-training before starting the gradient descent was suggested as a new mitigation method (Hinton et al., 2006). Stochastic Gradient Descent and Batch Gradient Descent have also been used to mitigate this problem. The basic equation to update gradient descent (optimise the network in every iteration), is Where is the weight vector, which is used to subtract the gradient of the loss function with respect to the weights multiplied by the learning rate ( ), since the steepest descent is the opposite to the gradient. The gradient is a vector that gives the direction of the loss function’s steepest ascent. is the number of the iteration, and is the learning of the iteration. Now to improve the vanishing Acta Wasaensia 35 gradient problem using the techniques discussed above, Equation 3 is revised in Equation 4; This means that the gradient of the loss function is taken at every step, which differs from the actual loss function that is summation of loss of every learning iteration. This gives one-iteration-loss, while Equation 3 gives all-iterations-loss which in turn may lead to a local minima or a saddle point. The depth of the Neural Network is measured with the number of hidden layers; at the beginning two or more hidden layers counted as deep network but this number had been increasing over the years, Figure 7 shows a simple deep neural network in comparison to a shallow neural network, this was adopted from (Nielsen, 2015). In 2014, GoogLeNet was release, which had 22 layers and 1024 weights, it had won Image-Net Large Scale Visual Recognition Competition (ILSVRC’14) that year (Szegedy et al., 2014). Figure 7. Simple Comparison between Shallow and Deep Neural Networks Apple uses deep learning for face recognition in its iPhone X as a biometric authentication method (YML, 2017). The phone is equipped with “neural engine” that is two processing cores to handle machine learning algorithms, such as face recognition and augmented reality apps. This could open the door for massive improvements to mHealth. It has been used largely in the medical field; to form diagnosis using images or sounds. When using images, it is referred to as “computer vision”, which is used by Microsoft’s InnerEye Initiative that started in 2010 (Microsoft, 2010). Currently working on image diagnostic tools for tumour detection, this went on to be used for automatic segmentation of aggressive brain tumours and overall simplification of otherwise delicate surgeries. 36 Acta Wasaensia It is doing an even better job at suggesting treatment ideas and options leaning from previous patients and finding what worked best on similar conditions. This was the result of the collaboration between the Oncology department at the Memorial Sloan Kettering with IBM Watson; the result an intelligence- augmenting tool that helps doctors deal with unique cancer cases (Memorial Sloan Kettering Cancer Center, 2018). Deep Learning has opened the door for insightful medical research and tremendous chances for improvements and simplification of difficult decision- making processes in the healthcare world. However, the core source of Deep Learning power is data; the bigger and relevant the data, the better the learning. Therefore, crowdsourced medical data collection is now important. It is simply, pooling data from mobile devices to aggregate health data; this is what Apple’s HealthKit, Samsung Health, FitBit, and the likes are doing. In fact, Apple is aiming to use the data collected through their devices to find treatment for Parkinson’s disease and Asperger’s syndrome. They have built interactive apps that patients can use to assess their conditions over time, of course the data collected through the apps are anonymised and fed into the data pool for future research into a possible treatment. Not only that, they actually have multiple apps; for autism, seizures, concussion, melanoma, and more (Apple, 2017). They have recently introduced a new model of Apple Watch that is equipped with three sensors to perform electrocardiogram that results an ECG signal of the heart. Traditionally, electrocardiogram is performed using electrodes on the skin; about six on the chest and in some cases one on every limb (Cables & Sensors, 2018). Therefore, Apple Watch is a practical step in the same direction of this research; performing diagnostics on the go. It is also equipped with an accelerometer and gyroscope to detect falls. The watch is expected to monitor the heart rate continuously and use a software application to “diagnose” the user’s condition based on this data that may result in contacting the emergency