Shaima Abdelmageed Wrist-based Phonocardiogram Diagnosis Leveraging Machine Learning aaa ACTA WASAENSIA 416 ACADEMIC DISSERTATION To be presented, with the permission of the Board of the School of Technology and Innovations of the University of Vaasa, for public examination in Auditorium Kurtén (C203) on the 2nd of May, 2019, at noon. Reviewers Professor Tarik Taleb Aalto University School of Electrical Engineering Department of Communications and Networking P.O.BOX 11000 FI-00076 AALTO Finland Docent Teemu Myllylä University of Oulu Electronics and Communications Engineering P.O.BOX 8000 FI-90014 University of Oulu Finland III Julkaisija Julkaisupäivämäärä Vaasan yliopisto Huhtikuu 2019 Tekijä(t) Julkaisun tyyppi Shaima Tajalsir Abdelmageed Abdel Rahman Väitöskirja Orcid ID Julkaisusarjan nimi, osan numero Acta Wasaensia, 416 Yhteystiedot ISBN Vaasan yliopisto Teknologian ja innovaatiojohtamisen yksikkö PL 700 FI-65101 VAASA 978-952-476-850-4 (painettu) 978-952-476-851-1 (verkkojulkaisu) URN:ISBN:978-952-476-851-1 ISSN 0355-2667 (Acta Wasaensia 416, painettu) 2323-9123 (Acta Wasaensia 416, verkkoaineisto) Sivumäärä Kieli 134 englanti Julkaisun nimike Rannepohjaisen fonokardiogrammin analysointi koneoppimista hyödyntäminen Tiivistelmä Teknologian valtavan kehittymisen ja nopean elämänrytmin myötä välittömästi saatu tieto on noussut jokapäiväiseksi välttämättömyydeksi, erityisesti hätätapauksissa, joissa jokainen säästetty minuutti on tärkeää ihmishenkien pelastamiseksi. Mobiiliterveys, eli mHealth, on yleisesti valjastettu käyttöön nopeaksi diagnoosimenetelmäksi mobiililaitteiden avulla. Käyttö on kuitenkin ollut haastavaa korkean datan laatuvaatimuksen ja suurten tiedonkäsittelyvaatimuksien, nopean laskentatehon ja sekä suuren virrankulutuksen vuoksi. Tämän tutkimuksen tavoitteena oli diagnosoida sydänsairauksia fonokardiogrammianalyysin (PCG) perusteella käyttämällä koneoppimistekniikoita niin, että käytettävä laskentateho rajoitetaan vastaamaan mobiililaitteiden kapasiteettia. PCG-diagnoosi tehtiin käyttäen kahta tekniikkaa 1. Parametrinen estimointi käyttäen moniulotteista luokitusta, erityisesti signaalien erotteluanalyysin avulla. Tätä asiaa tutkittiin syvällisesti käyttäen erilaisia tilastotieteellisesti kuvailevia piirteitä. Piirteiden irrotus suoritettiin käyttäen Wavelet- muunnosta ja suodatinpankkia. 2. Keinotekoisia neuroverkkoja ja erityisesti hahmontunnistusta. Tässä menetelmässä käytetään myös PCG-signaalin hajoitusta ja Wavelet-muunnos -suodatinpankkia. Tulokset osoittivat, että PCG 19dB:n signaali-kohina-suhteella voi johtaa 97,33% onnistuneeseen diagnoosiin käytettäessä ensimmäistä tekniikkaa. Signaalin hajottaminen neljään alikaistaan suoritettiin käyttämällä toisen asteen suodatinpankkia. Jokainen alikaista kuvattiin käyttäen kahta piirrettä: signaalin keskiarvoa ja kovarianssia, näin saatiin yhteensä kahdeksan ominaisuutta kuvaamaan noin yhden minuutin näytettä PCG- signaalista. Lisäksi tutkittiin ja verrattiin eriasteisia suodattimia ja piirteitä. Toista tekniikkaa käyttäen diagnoosi johti 100% onnistuneeseen luokitteluun 83,3% luotettavuustasolla. Tuloksia käsitellään ja pohditaan, sekä tehdään niistä johtopäätöksiä. Lopuksi ehdotetaan ja suositellaan käytettyihin menetelmiin uusia parannuksia jatkotutkimuskohteiksi. Asiasanat Analyysi, luokittelu, tiedon laatu, taudinmääritys, eTerveys, suodatinpankki, mobiiliterveys, neuroverkot, fonokardiogrammi, signaalikohinasuhde, Wavelet- muunnos V Publisher Date of publication Vaasan yliopisto April 2019 Author(s) Type of publication Shaima Tajalsir Abdelmageed Abdel Rahman Doctoral thesis Orcid ID Name and number of series Acta Wasaensia, 416 Contact information ISBN University of Vaasa Faculty Department or subject P.O. Box 700 FI-65101 Vaasa Finland 978-952-476-850-4 (print) 978-952-476-851-1 (online) URN:ISBN:978-952-476-851-1 ISSN 0355-2667 (Acta Wasaensia 416, print) 2323-9123 (Acta Wasaensia 416, online) Number of pages Language 134 English Title of publication Wrist-based Phonocardiogram Diagnosis Leveraging Machine Learning Abstract With the tremendous growth of technology and the fast pace of life, the need for instant information has become an everyday necessity, more so in emergency cases when every minute counts towards saving lives. mHealth has been the adopted approach for quick diagnosis using mobile devices. However, it has been challenging due to the required high quality of data, high computation load, and high-power consumption. The aim of this research is to diagnose the heart condition based on phonocardiogram (PCG) analysis using Machine Learning techniques assuming limited processing power, in order to be encapsulated later in a mobile device. The diagnosis of PCG is performed using two techniques; 1. parametric estimation with multivariate classification, particularly discriminant function. Which will be explored at length using different number of descriptive features. The feature extraction will be performed using Wavelet Transform (Filter Bank). 2. Artificial Neural Networks, and specifically Pattern Recognition. This will also use decomposed version of PCG using Wavelet Transform (Filter Bank). The results showed 97.33% successful diagnosis using the first technique using PCG with a 19 dB Signal-to-Noise-Ratio. When the signal was decomposed into four sub- bands using a Filter Bank of the second order. Each sub-band was described using two features; the signal’s mean and covariance. Additionally, different Filter Bank orders and number of features are explored and compared. Using the second technique the diagnosis resulted in a 100% successful classification with 83.3% trust level. The results are assessed, and new improvements are recommended and discussed as part of future work. Keywords Analysis, classification, Data quality, diagnosis, eHealth, Filter Banks, mHealth, Neural Networks, PCG, SNR, Wavelet Transform VII ACKNOWLEDGEMENT “Desire is the key to motivation, but it's determination and commitment to an unrelenting pursuit of your goal - a commitment to excellence - that will enable you to attain the success you seek.” - Mario Andretti I started this journey in 2012, the same year that I finished my master’s degree and started working for Lumi Mobile (now known as Auga Solutions.). Despite my enthusiasm for this research that’s very dear to me, I had to prioritise work. Not just for the obvious reasons but also because I passionately liked my job. This continued for two years; leading the development team of one of the great products at the company with no progress on the research front! That’s when I had to stop and reprioritise! I, then, lowered my working hours and picked up research work at the university, all to come back full force to my beloved research… and it almost worked! Only to be hindered again by my promotion to become the product manager of that same product I liked so much. It was sad, because my success at work seemed to push my PhD dream farther away. Life became a race between work and school, and the tunnel seemed to have no end... Until I made it! I found the balance and made it all fit together. And what is an extra year or two when you get to do it all at once! This was not easy; it wasn’t always “peaches and cream” and although I claim to be a super multitasker; this was indeed a true test, especially in the last few months. None of it would’ve been possible if it wasn’t for the awesome people in my life… My very understanding boss, Marcus Wikars, who didn’t only allow me to lower my hours, but he kept me challenged and inspired. Thank you for your trust, Ace! My friend, Reino Virrankoski, who hired me as a researcher at the University of Vasa and gave me the option to do more. Kept me focused on the research and the area of eHealth with the project “Nordic Telemedicine Center”. Thank you for NTC, Reino! My brilliant mentor, Prof. Mohammed Elmusrati, who has been with me every step of the way. For every productive discussion, all the tips and the constant guidance, I, sincerely, thank you! My dear father, my loving mother, and my very supportive sister and brother, I thank you with all my heart. It’s only possible because you believe in me. Vasa, Oct. 1st, 2018 IX Contents ACKNOWLEDGEMENT ........................................................................... VII 1 INTRODUCTION ................................................................................ 1 1.1 Motivation ............................................................................... 2 1.2 Objectives and contributions ................................................... 3 1.3 Methods .................................................................................. 5 1.4 Why focus on heart condition? ................................................. 8 1.5 Thesis structure ...................................................................... 9 2 LITERATURE REVIEW ........................................................................ 11 2.1 About eHealth and mHealth .................................................. 11 2.2 History of eHealth/mHealth ................................................... 12 2.3 mHealth for emergency healthcare ........................................ 13 2.4 Problems with current emergency solutions .......................... 17 2.5 Studies About Heart Conditions ............................................ 18 3 ARTIFICIAL NEURAL NETWORKS ....................................................... 27 3.1 Network Architecture ............................................................ 27 3.1.1 Single-Layer Feedforward Networks ........................ 28 3.1.2 Multilayer Feedforward Networks ............................ 28 3.1.3 Recurrent Networks ................................................ 29 3.2 Learning Tasks ...................................................................... 30 3.2.1 Pattern Association ................................................. 30 3.2.2 Pattern Recognition ................................................ 30 3.3 More Neural Networks Applications in Tele-cardiology .......... 31 3.4 Deep Learning ....................................................................... 32 4 WAVELET TRANSFORMS ................................................................... 38 4.1 Filter Banks ........................................................................... 39 4.1.1 Two-Channel Filter Bank ......................................... 41 4.1.2 Tree-Structured Filter Bank ..................................... 42 4.2 Applications of Wavelet Transform in Tele-Cardiology ........... 43 4.3 Filter banks and Neural Networks .......................................... 44 5 MODELLING THE CARDIOVASCULAR SYSTEM ................................... 46 5.1 Electrocardiogram (ECG) Signal ............................................. 48 5.1.1 Mapping the ECG Signal to Electrical Circuits .......... 49 5.1.1.1 Vessel Resistance ............................................ 50 5.1.1.2 Vessel Compliance .......................................... 50 5.1.1.3 Blood Inertia ................................................... 51 5.1.1.4 Valves ............................................................. 51 5.1.1.5 The Windkessel Model ..................................... 52 5.1.1.6 Mossa’s engineering model ............................. 52 5.2 Phonocardiogram signal (PCG) .............................................. 55 5.2.1 Definitions and Descriptions ................................... 57 5.2.2 Characteristics of the Heart Acoustic Wave ............. 60 X 6 MODELLING HEART ACOUSTIC WAVE PROPAGATION ....................... 63 6.1 Analytical Approach: Attenuation .......................................... 63 6.1.1 Discussion .............................................................. 66 6.2 Stochastic Approach ............................................................. 68 6.2.1 Discussion (Realisation of Durand’s Research) ........ 71 6.3 Summary .............................................................................. 76 7 THE EXPERIMENT ............................................................................ 78 7.1 Classification of Heart Conditions ......................................... 83 7.1.1 Simplified Approach (Using two features) ............... 83 7.1.2 Advanced Approach (Using Wavelet Transform) ...... 89 7.1.3 Using Neural Network for Classification .................. 94 7.2 Summary .............................................................................. 99 8 CONCLUSIONS .............................................................................. 101 8.1 Future Work ........................................................................ 102 REFERENCES ....................................................................................... 106 XI Figures Figure 1. Decision making after receiving the signal ..................... 7 Figure 2. Experiment steps ........................................................... 8 Figure 3. Single-Layer Feedforward Network ............................... 28 Figure 4. Fully connected Multilayer feedforward network .......... 29 Figure 5. Simple Recurrent network (SRN) ................................... 29 Figure 6. Steepest Descent Problem (local minima) ..................... 34 Figure 7. Simple Comparison between Shallow and Deep Neural Networks ..................................................................... 35 Figure 8. M-channel filter bank (Mertins, 1999) .......................... 40 Figure 9. Two-channel filter bank ............................................... 41 Figure 10. Tree-structured filter banks equivalent system. ............ 42 Figure 11. Diagram of the human heart (Creative Commons) ........ 47 Figure 12. P, Q, R, S, and T waves of ECG (Creative Commons) ..... 49 Figure 13. Hydraulic analogue of the cardiovascular system ......... 53 Figure 14. Closed-loop lumped model of the cardiovascular system. ........................................................................ 54 Figure 15. Heart sound signal (Creative Commons). ..................... 59 Figure 16. Wiggers Diagram (Creative Commons). ........................ 61 Figure 17. Propagation Model of heart acoustic wave (Analytical) .. 63 Figure 18. Travel model of pulse wave sounds in blood vessels. ... 66 Figure 19. Durand’s basic model of the heart/thorax acoustic system ......................................................................... 69 Figure 20. Heart acoustic signal measured at the chest (input) ..... 73 Figure 21. Realisation of Durand’s Model (1) ................................ 74 Figure 22. Realisation of Durand’s Model (2) ................................ 74 Figure 23. Simulation model of the stochastic approach – simplified method ........................................................................ 75 Figure 25. Heart-wrist acoustic propagation model (analytical approach) .................................................................... 78 Figure 26. Original Signal: Healthy Heart Acoustic Signal .............. 79 Figure 27. Received signal at the wrist (heart-wrist propagation mode) .......................................................................... 80 Figure 29. Comparison of the success rate around the SNR threshold. .................................................................... 88 Figure 30. Results of Advanced Approach – 8 features (using Filter Banks) ......................................................................... 91 Figure 31. Classification success vs. the number of levels (High SNR) ............................................................................ 92 Figure 32. Classification success vs. the number of levels (Low SNR) ............................................................................ 92 Figure 33. Visual demonstration of the impact of the number of hypotheses. ................................................................. 93 Figure 34. Neural Pattern Recognition Network Diagram ............... 94 Figure 35. Neural Pattern Recognition performance plot ............... 95 Figure 37. PCG-based Diagnosis Using Machine Learning (three methods) ................................................................... 100 Figure 38. Data Fusion-based Diagnosis using Machine Learning 104 XII Tables Table 1. Shallow vs. Deep Neural Networks ................................. 33 Table 2. Sound velocity in different media .................................. 62 Table 3. Acoustic attenuation (5) of human body tissues ............ 64 Table 4. Acoustic properties for human tissue between heart and wrist ............................................................................. 67 Table 5. Calculating P(d) for S1 and S2 frequencies in different tissues .......................................................................... 68 Table 6. Classes for The Diagnostic System ................................ 82 Table 7. False classifications using simplified approach (scenario 2) .................................................................................. 87 Table 8. Snippet of the Classification Result. .............................. 97 XIII Abbreviations ADC Analogue-Digital Converters ADTree Alternating Decision Tree AED Automated External Defibrillator AI Artificial Intelligence ANN Artificial Neural Networks API Application Programming Interface bps bits per second CDMA Code Division Multiple Access CS Compressed Sensing CWT Continuous Wavelet Transform CVD Cardio-Vascular Disease DWT Discrete Wavelet Transform ECG Electro-Cardio-Gram ED Emergency Department EF Ejection Fraction EGG Electro-Gastro-Gram eHR Electronic Health Record ELM Extreme Learning Machine EMD Empirical Mode Decomposition EMR Electronic Medical Record ER Emergency Room FDSS Fuzzy Decision Support System FFNN Feed Forward Neural Network FFNNBP Feed Forward Neural Network Back Propagation FFT Fast Fourier Transform FIR Finite Impulse Response FNN Fuzzy Neural Networks XIV GPS Global Positioning System GSM Global System for Mobile HR Heart Rate HRV Heart Rate Variability IBC Intra Body Communications ICT Information and Communications Technology IFFT Inverse Fast Fourier Transform IIR Infinite Impulse Response IOM Institute of Medicine IoT Internet of Things LMMSE Linear Minimum Mean Square Error LPC Linear Prediction Coefficients LQG Linear Quadratic Gaussian MAE Mitral Annular Excursion MCI Mass Causality Incident MEDTOC Medical Data Transmission Over Cellular Networks MEMS Micro-Electro Mechanical System MLP Multi-Layer Perceptron MM Mini-Max Estimator MMSE Minimum Mean Square Error MV Minimum Variance MVU Minimum Variance Unbiased OCC One-Class Classification PCA Principal Component Analysis PCG Phono-Cardio-Gram PC Personal Computer PDA Personal Digital Assistant PDF Probability Density Function PSD Power Spectral Density XV PSO Particle Swarm Optimisation PR Perfect Reconstruction RBFN Radial Basis Function Networks REST Representational State Transfer SMS Short Message Service SNR Signal to Noise Ratio SPL Sound Pressure Level STFT Short-Time Fourier Transform SVM Support Vector Machine TAR Temporal Association Rules TSH The Scarborough Hospital UMTS Universal Mobile Telecommunications System VCG Vector-Cardio-Gram VPC Ventricular Premature Contraction WES Wearable ECG Sensor XVI Formulas 394 * Î 5 Ê !0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000> 3:4  Î ÕÒ/Õ000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000> 3;4 ( F (/52G  D —Ø(ٗ~ 0000000000000000000000000000000000000000000000000000000000000000000;< 3<4 "# Î ;. - Ú( F (/52G  —Ø(ÙÛ000000000000000000000000000000000000000000000;= 3=4 ØØ, ÙÙ Î ~CŽ J Ø%Ù42 Þ / Ž ß % ƒ§ /§ 0000000000000000000000000000000000000000000000000;A 3>4 –Ø!/ Ù, H 00000000000000000000000000000000000000000000000000000000000000000000000000000000000<8 3?4 }Ø+Ù Î ~ Ü}Ø+ÙØ+Ù Ê }Ø/+ÙØ/+ÙÝ000000000000000000000000000000000000000000000<9 3@4 ÓØ+Ù Î ~ Ü}Ø+Ù}Ø+Ù Ê ~Ø+Ù~Ø+ÙÝØ+Ù0000000000000000000000000000000000000000<9 3A4 Ø+Ù Î }Ø+Ù}Ø+Ù Ê ~Ø+Ù~Ø+Ù Î <+/š00000000000000000000000000000000000000000<9 3984 Ø+Ù Î }Ø/+Ù}Ø+Ù Ê ~Ø/+Ù~Ø+Ù Î :000000000000000000000000000000000000000000<9 3994 ~ ~Ø+ÙÜ}Ø+Ù Ê }Ø/+ÙÝ00000000000000000000000000000000000000000000000000000000000000000<: 39:4 Ø+Ù Î }Ø+Ù~Ø+Ù0000000000000000000000000000000000000000000000000000000000000000000000<: 39;4  Î 0 L  ’ Î 0 000000000000000000000000000000000000000000000000000000000000=8 39<4  Î  0% L   Î ’0%00000000000000000000000000000000000000000000000=9 39=4 Î 0% L  Î ’ 0%0000000000000000000000000000000000000000000000=9 39>4  Î : Ï 2 L  Î : Ï 200000000000000=9 39?4 Ø+Ù Î }/¥Â£0000000000000000000000000000000000000000000000000000000000000000000000000000000000>< 39@4 "&&B<: "~} ކأنÀ ß00000000000000000000000000000000000000000000000000000000000000000000000000>< 39A4 5 Î 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000>< 3:84 5 Î <: "~}ØÙ- 7† K A-@7†0000000000000000000000000000000000000000000000000000000000000>< 3:94 Ø+, Ù Î }/Ž“£0‚-000000000000000000000000000000000000000000000000000000000000000000000000000>< 3::4  ØÙ Î }/¤‘000000000000000000000000000000000000000000000000000000000000000000000000000000000000>= 3:;4  ‘‡ØÙ Î /;> Ê <: "ØÙ000000000000000000000000000000000000000000000000000000000000000000>= 3:<4  Ø, Ù Î Š0†Ø‘,“ىؓÙ֓ 0000000000000000000000000000000000000000000000000000000000000000000000000000>= 3:=4 Ø, Ù Î /¤‘0000000000000000000000000000000000000000000000000000000000000000000000000000000000000>= 3:>4 ›Ø!, Ù Î D (Ø Ù)Ø! Ê / 0<Ù/•¦“—ˆ/~—…} 000000000000000000000000000000000000>A 3:?4 ›Ø!, Ù Î D (Ø Ù*Ø! Ê / 0<Ù/•¦“—ˆ/~—…} 000000000000000000000000000000000000?8 3:@4 ¡¢Ø!, Ù Î ~‹ D ›2Ø!, Ù›Ø!, ً›…~ 0000000000000000000000000000000000000000000000000000?8 3:A4 ¡¡Ø!, Ù Î ~‹ D 1›Ø!, Ù1‹›…~ 00000000000000000000000000000000000000000000000000000000000?8 3;84 ¢¢Ø!, Ù Î ~‹ D 1›Ø!, Ù1‹›…~ 000000000000000000000000000000000000000000000000000000000000?8 3;94  Î D (Ø Ùˆ/~—…} 000000000000000000000000000000000000000000000000000000000000000000000000000000?8 3;:4 Ø!, Ù Î ¡¢Ø!, Ù Î ¡¢Ø!, Ù0¡¡Ø!, Ù000000000000000000000000000000000000000000?8 3;;4 6¡¢ Ø!, Ù Î 1ŒÇÈؘ,“Ù1 Á ŒÇÇؘ,“ÙŒÈÈؘ,“Ù00000000000000000000000000000000000000000000000000000000000000000000?8 3;<4 ž¢Ø!, Ù Î ¡¢Ø!, ÙØ ŒÇÇؘ,“ÙŒÇÇؘ,“ÙƒŒÄÄؘ,“ÙÙ0000000000000000000000000000000000000000000000?9 3;=4 Ø!, Ù Î ŒÇÇؘ,“ÙƒŒÄÄؘ,“ÙŒÇÇØÅ,ÃÙ Î ŒÆÆؘ,“Ù ŒÆÆؘ,“Ù/ŒÄÄؘ,“Ù00000000000000000000000000000000000000000?9 XVII 3;>4 6žŸ Ø!, Ù Î 6¡¢ Ø!, ÙØ ŒÇÇؘ,“ÙŒÇÇؘ,“ÙƒŒÄÄؘ,“ÙÙ0000000000000000000000000000000000000000000000000?9 3;?4 (Ø!Ù Î :->/Ø;/ "$ Þ¦˜‰/~ßÙ: Ð ! Ð /;000000000000000000000000000000000000000?: 3;@4 ” Î Ô7¢/7”Ô Ê 18¢/8”10000000000000000000000000000000000000000000000000000000000000000@; 3;A4 ” Î :->Ô7¢/7”Ô Ê 18¢/8”100000000000000000000000000000000000000000000000000000000000000000@< 3<84  Î ) Ì +2 œC˜00000000000000000000000000000000000000000000000000000000000000000000000000000000000000@< 3<94 7Ø Ù Î Š™ ’›2#!Ø;Ù000000000000000000000000000000000000000000000000000000000000000000000@= 3<:4 8 Î Š™ ’›2#!Ø;Ù Ê 000000000000000000000000000000000000000000000000000000000000000000@= 3<;4 Î 82#!Ø!, ;Ù Ê 700000000000000000000000000000000000000000000000000000000000000000000000@= 3<<4 8 Î Š™ ’›2#!Ø;Ù Ê ;000000000000000000000000000000000000000000000000000000000000000000@> 3<=4  Î /:-> "Ø%Ø$ÙÙ/:->Ø)/ Ù92!'#$Ø$Ù2Ø)/ Ù000000000000000000000000A8 3<>4  Î /:->Ø)/ Ù92!'#$Ø$Ù2Ø)/ Ù00000000000000000000000000000000000000000000000000A8 3 1 INTRODUCTION “Creativity, as has been said, consists largely of rearranging what we know in order to find out what we do not know. Hence, to think creatively, we must be able to look afresh at what we normally take for granted.” — George Kneller eHealth is a relatively new terminology that represents the integration of electronics and communication in health systems. Moreover, mHealth (or mobile- health) is a more specific term to represent the applications of mobile devices (e.g. mobile phone) as a tool for communication, data processing, and positioning of health systems. However, eHealth is more general than mHealth. eHealth has been one of the hottest topics of biotechnology research areas. It is multidisciplinary type of research that includes information and communication technology (ICT), medical science, electrical engineering, computer science, embedded systems, etc. The field has been growing slowly but surely over the past two decades following the technology closely. This research looks back at the history of eHealth with a critical eye, the focus is particularly put on emergency healthcare. The decision to focus on emergency healthcare comes from the situation of the world nowadays, catastrophes are not rare anymore! The criticism is observed to identify the drawbacks of eHealth solutions for emergency healthcare. This research will highlight limitations in current solutions that affect their practicality in catastrophic situations. Namely, the design assumptions that stand between small-scale experiments of current eHealth solutions and their wide-scale applicability. The research then challenges these limitations utilizing a case study of cardiovascular system diagnosis. Having worked with cardiovascular system in master’s degree research, this case study was a clear choice. Challenging the limitations that are forced by design assumptions increases the chances of practicality of any of the eHealth solutions, this means more reliable solutions for emergency healthcare during catastrophes. The first step is very simple; while designing eHealth solutions, assume worst case scenario. And that is exactly what this research does. The research develops a method to diagnose heart conditions using noisy, corrupted, low quality and low power heart acoustic signals. It utilises signal estimation techniques to equalize the channel and restore as much as possible of the information held in the received heart sound signal. It will, potentially, lower 2 Acta Wasaensia the demands on the quality of biomedical signals and the processing energy, which will open the door for more realistic designs for emergency mHealth solutions. Although the focus here is on the quality of the data, but there are more design assumptions that should be addressed, such as: energy consumption, size of equipment, and many more that could be the focus of future research. In the rest of this chapter, the motive behind this research is revealed along with the scientific contributions and the method adopted to achieve the objectives. The chapter ends with thesis structure details. 1.1 Motivation With the tremendous growth of technology and the fast pace of life, the need for instant information has become an everyday necessity. One of the affected life aspects is health; the world is obsessed with monitoring vital signs and sharing results instantly not just with their doctors but also with their relatives. The doors were opened in the early 90s for repurposing Information and Communication Technology (ICT) for health, but the progress has been very slow due to the nature of the field and the sensitivity of the data (see chapter 2 for more details). My journey with ICT for health (also known as eHealth) started 8 years ago when I started researching wireless monitoring systems and hoped to build one myself, one that I called “The Vital Transmitter”. My goal was to build this system for my diabetic father. Mostly for me, to know his blood sugar level at any given time. For practical reasons the focus had to be shifted from reading blood sugar to heart rate, but the concept remained intact. The vital transmitter was built as part of my master’s degree research in 2012. It served its purpose; feature phones (non- smart) that triggers an SMS when the heart rate goes above a pre-calculated threshold or drops below another. This came from an Android app that compared the received heart rate to personal thresholds. The heart rate is received via low energy Bluetooth from a heart rate sensor and the thresholds were calculated using an algorithm that factors in age, gender, activity level, and accounts for sleep values. The algorithm was approved by a licensed cardiologist in Prince Sultan Cardiac Centre, Riyadh, Saudi Arabia. (Abdelmageed, 2012). Researching in this area has elucidated that, previously most of the ICT applications and systems built for health sector (eHealth), particularly eHealth solutions for emergency healthcare, assume perfect conditions. The system usually expects to deal with high quality data, and they are mostly designed with large, if not, multiple central processing units, which in turn consumes large amount of energy. Such systems fail to meet expectations when Acta Wasaensia 3 put to practice, especially in catastrophic situations. When data quality drops due to network status. The question that motivated this research is very simple; in case of catastrophes, when conditions are far from perfect, would these systems function? This question enticed the investigation and assessing the possibilities to making sense of low- quality data with low processing energy. However, this thesis is more focused on processing low quality data. 1.2 Objectives and contributions Listening to the heart sound using stethoscope is probably the oldest and fastest method to check the heart functionality (Health essentials, 2014). As many heart problems affect the way the heart beats in some manner, and since the heart acoustic is a result of the heart beating; it should hold valuable information about the causing problem. However, the human ear might not be able to distinguish small differences in the heart sound that indicate a disease, due to its low frequency that falls out of the spectrum range of human hearing or due its small power. Therefore, heart diagnosis using the conventional sound listening is not effective in most heart diseases (Health essentials, 2014). Thus, traditionally more sophisticated diagnostics and investigation methods are needed to fully report the heart condition, such as Electrocardiogram (ECG) and Echocardiogram. Nevertheless, most of these advanced methods require large devices and large power and cannot be implemented in small equipment’s like wearable watches or mobile phones. Consequently, this thesis proposes the application of advanced machine learning technologies to extract useful diagnostic information from the heart sound data. In this process, the entire sound spectrum should be used, including the bands outside of the human hearing range, and this process of data collection and processing should be performed with limited processing energy and limited capabilities, the likes of which mobile devices are capable of performing. The success of this concept will open the door for tremendous changes to the game of heart monitoring and diagnosis using wearable small devices like watches. Its task is to process low quality signals to obtain health diagnosis with acceptable error margin. The focus is alarming diagnostics and the goal is to save precious time at emergencies in less than perfect conditions (such conditions that are usually experienced in catastrophes). During the first couple years of research the following questions have emerged:  In health diagnostics, how low could the quality of the data get before it is declared useless? 4 Acta Wasaensia  Could signal classification techniques help make sense of distorted/corrupted biomedical signal?  How reliable are the diagnostics that are based on processed data? And while this research responds to these questions, it had also raised few more that would leave the door open for future work. This thesis documents the process of going through the history of mHealth, focusing on emergency healthcare, and why it lacks practicality in catastrophes. Moreover, it explains in detail the experiment that answer the research questions. The key tasks and contributions of this research are: • To study the reliability of the pre-diagnosis found in low quality acoustic signal of the heart after processing, using different techniques (i.e. simple and advanced Machine Learning techniques and Neural Networks) to identify the information held in the signal.  To study eHealth history and, specifically, mHealth tele-cardiology solutions that were discussed in recent years (2006 – 2018).  To, potentially, lower the demands on the quality of the data for medical diagnostics, after proving the concept using the heart acoustic signals.  To allow performing offsite diagnostics (during catastrophes/ in accident locations) despite tough circumstances that could include bad connections, small processing units and low/limited energy (mobile devices) in the future.  To start the wave of measuring vital signals from unpredictable spots, which opens the door for new generation of mHealth products. The contributions of this thesis have been published/accepted/drafted in/by the following journals; - Abdelmageed, S. and Elmusrati, M. (2018). Phonocardiogram Based Diagnosis using Machine Learning: Parametric Estimation with Multivariant Classification. Bioscience & Engineering: An International Journal, Oct 30, vol. 5, no. 1/2/3/4, pp. 1-6 Available from: http://dx.doi.org/10.5121/bioej.2018.5401. (Published) Acta Wasaensia 5 - Abdelmageed, S. and Elmusrati, M. Machine Learning and Wearable Devices for Phonocardiogram-Based Diagnosis. 6th International Conference on Bioinformatics and Bioscience, May 2019. (Accepted) The following journal has been drafted to be submitted this year to a suitable journal in the field; - Abdelmageed, S. and Elmusrati, M. Wrist-based Phonocadiogram Diagnosis Leveraging Machine Learning (Drafted) 1.3 Methods Critical review was essential to identify the problems in current mHealth solutions for emergency healthcare. It was important to assess a large number of applications dedicated to emergency healthcare, to understand the proposed systems and challenge the described characteristics. This was the adopted method that led to defining the research problems. To stress the focus on alarming diagnostics, cardiovascular system was chosen as a case study. It was also the clear choice as a continuation after the master’s degree research (Abdelmageed, 2012). The experiment is performed on MATLAB/SIMULINK. The research is studying the reliability of the health diagnosis that is based on processed biomedical signal. This was achieved in six steps: STEP 1. Heart sound signal, also known as Phono-Cardio-Gram (PCG) was used. A healthy heart sound signal was used as a reference, this signal is called (S) in this section. This was used to bypass the difficulties of obtaining human heart sound signals; as this study takes a pure engineering approach, no ethical approval was required. STEP 2. A model of the cardiovascular acoustic wave propagation system was deduced from existing literature. The model should represent the propagation system (signal channel) from the heart (chest area) to the wrist. The wrist was chosen to leave room for future work related to wearable wrist bands that could carry out the task of diagnosing. Not to mention, that it is reasonably distant from the heart/chest area. STEP 3. The reference sound signal with added noise have been applied to the model from STEP 2. The received signal was then compared with the original 6 Acta Wasaensia signal to measure the effect of propagation system. The resultant signal is given by the equation 394 * Î 5 Ê ! Where v is the received signal measured from the wrist, xZ is the original signal (Z measured from the chest) after the effect of the acoustic propagation system model, and l is the added noise. It should be mentioned here that the channel model can be very complicated. For the following reasons: 1. The heart acoustic wave propagates over and through different matters, such as bones, blood, flesh, etc. Each one has different wave speed, propagation constant, and other characteristics that has a distinctive effect on the propagation. 2. The band of interest can vary in frequency from a fraction of one Hertz (e.g., 0.1 Hz) to several tens (e.g., 100 Hz). Within this band the wavelength can vary from 3 meters to 3000 meters. Hence, the propagation model characteristics may change considerably between different frequencies. However, since the modelling process is not the main topic of this thesis work, the chosen model was selected after studying several models that were proposed in the literature. However, it became evident that there is room for further studies in this topic of modelling. Depending on the level of distortion in the received signal, STEP 4 of the experiment will be decided according to the following routes. A clarification of this decision is shown in Figure 1. If the received signal is highly distorted and has low to no correlation with the original signal STEP 4. The received signal will be weak and highly distorted version of the original signal. Hence, the received signal (v) should be used to restore the original signal (Z ). This is called equalization process, which is common in different applications of wireless communications to equalize the channel impacts on the signal. There are many well-known equalization techniques. However, most of the conventional equalization methods might not work easily in this case because of the large nonlinearity caused by the large changes in the wavelength. Therefore, feed-forward neural network with backpropagation was used to retrieve the information held in the original signal, the resultant signal is denoted (Z×). The error margin given by Equation (2) is calculated and kept as small as possible 3:4  Î ÕÒ Ë Õ Acta Wasaensia 7 This process is repeated for every sound signal, where the resultant (Z×) represents a hypothesis of a heart condition. For example, the healthy heart signal (Z× ) represents (R}) that is “healthy heart”. These hypotheses are used as references throughout the experiment. The plan is to define at least five hypotheses. Else, if the received signal correlates with the original signal STEP 4. The received signal (v ) is declared as hypothesis H0 as is, without restoration. Hypothesis H0 represents health heart condition. Figure 1. Decision making after receiving the signal Four more hypotheses are defined using different transfer functions, all are based on the original signal (Z), all used transfer functions are unique. More details on this in chapter 7. STEP 5. A number of cases are generated under every hypothesis, using different noise characteristics. Each case is then classified using the following techniques:  Machine Learning; Parametric Estimation using Multivariate classification. This is explored at length, with different number of signal descriptive features. 8 Acta Wasaensia  Artificial Neural Networks; Pattern Recognition Neural using MATLAB Toolbox. STEP 6. Artificial Neural Networks are used again to automate the diagnosing process. The ANN is trained to recognise the defined hypotheses from STEP 4, and accordingly every (Z×) is put into the network to find the most probable diagnosis. This was a summary of the methods used in this research to achieve the discussed objectives. In Figure 2, the experiment steps are summarised and visualised. In the figure, the number of hypotheses is kept undefined to leave room for future growth and increased accuracy. Figure 2. Experiment steps Where Z is the original heart sound signal, in this case recorded at the chest, and v is the heart sound signal as received at the wrist. Noise has different sources which can be internal; like lungs and respiratory sound, digestive system and specifically stomach sound. Or external sounds from outside the body that fall in the same frequency band of interest. Moreover, the noise generated by the sound recording sensor itself and the electronic devices used in measurements. Z× is the received heart sound signal after restoring missed information, and last ÚR~, R, R€, . ,R¶Û represent the resultant hypotheses of heart diagnosis. 1.4 Why focus on heart condition? The focus on the heart condition was because the heart beat is the first vital sign to check in emergencies. Detecting a heart pulse indicates that the emergency victim Acta Wasaensia 9 is alive whether it is a catastrophic situation or not. The next step is to diagnose the heart condition to determine the need for further cardiology treatment. A paramedic, or the medical personnel attending to an emergency, spends the most of their time eliminating life threats. Once they know the heart condition, they can find the proper way to restore its functionality to as normal as possible for the patient in question. Clearing the airway is just as important, however, having oxygen in the body will not help if the hear is not beating. Heart condition could have subtle symptoms to the patient, which makes it harder to identify and that adds to the necessity of cardiac diagnostic systems or at least systems that takes us closer to diagnosis. Not to mention that, early detection of heart diseases could tremendously reduce the mortality rate. 1.5 Thesis structure This thesis consists of eight chapters, each of which documents a part of the research and serves as a step towards the conclusion. Chapter one is an introduction to this research stating background and motivations that have driven interest towards this topic. Followed by the novelty of this research presented in the objectives and contributions. The research method is also presented in this chapter and then the structure of the thesis to help the reader keep up. In the next chapter, three years’ worth of literature review is laid out. Starting with defining Information and Communications Technology (ICT) for health and the story behind of the term “eHealth” and “mHealth”. The chapter then goes on to give an overview of the history of eHealth. Then, moves a step closer to the topic of interest by discussing eHealth solutions for emergency healthcare and stating the problems with current solutions, the goal here is to build a background for the next chapter and help the reader follow the train of thoughts that led to this work. The chapter continues to investigate tele-cardiology solutions presented in the current decade (since 2010). Tracing published papers that aimed to advance tele- cardiology; remote monitoring, heart signal analysis (ECG and sound), heart disease detection, heart disease diagnosis, and the list goes on. The selected papers were chosen based on relevance and date of publication. Chapter three is dedicated to Artificial Neural Networks (ANN); what it is and how it works and more about what it is used for in this research. This will be revisited in the experiment. 10 Acta Wasaensia Chapter four discuss wavelet transform and then focuses on filter banks and how they are used to extract features of the signal. These techniques will be revisited in the experiment. Chapter five discusses modelling the cardiovascular system. Starting with the mathematical model of the electrocardiogram (ECG) to explain the cardiovascular system function and circulation, this chapter summarises the essential learnings to understand the cardiovascular system in preparation for this research. Followed by the mathematical model of the phonocardiogram (PCG) as the focus of this research. Chapter six is where the heart acoustic wave propagation system is modelled. Few models are presented, and each model is discussed using MATLAB/SIMULINK custom designs to prove practicality. In chapter seven, the experiment is detailed, where the proposal is validated. The results are discussed and analysed at the end of every approach, with total of three approaches. This thesis is concluded in chapter eight, where the results are summarised. The eighth and final chapter presents an overview of the future work and next possible steps to expand this research. Acta Wasaensia 11 2 LITERATURE REVIEW This chapter summarises the available literature in the field of eHealth and specifically in the field of mHealth. The focus then shifts to eHealth/mHealth emergency medical solutions; because it is more relevant to the topic of this research. Extra attention is lent to heart related studies; as the main case study. 2.1 About eHealth and mHealth In the world of academics, eHealth is just another word for Information Communication Technology (ICT) for health. The term “eHealth” is relatively new; it was barely used before 1999. The term has been used to refer to Internet Medicine as well as any form of computer use in medicine. And just like most of the buzzwords (e.g. ecommerce, ebusiness) it was industry leaders and marketing people who first introduced the term “eHealth” rather than academics. (Eysenbach, 2001) It is rather late to try avoiding this term in the academic papers with more than 170 scientific journals already having it in their titles. Some of these journals are dated back to 1999, as referenced in MEDLINE index of biomedical journal literature. With all that, the need to scholarly define the term “eHealth” became urgent, yet almost impossible due to the nature of the science behind it. One JMIR Editorial Board member said “Stamping a definition on something like eHealth is somewhat like stamping a definition on 'the Internet': It is defined how it is used - the definition cannot be pinned down, as it is a dynamic environment, constantly moving.” However, one of the editors, named Gunther Eysenbach, had attempted a broad definition that hopefully encompasses eHealth dynamism “eHealth is an emerging field in the intersection of medical informatics, public health and business, referring to health services and information delivered or enhanced through the Internet and related technologies. In a broader sense, the term characterizes not only a technical development, but also a state-of-mind, a way of thinking, an attitude, and a commitment for networked, global thinking, to improve health care locally, regionally, and worldwide by using information and communication technology.” The referenced dynamism is evident through the history and progression of eHealth. This chapter traces the history of eHealth through the years discussing 12 Acta Wasaensia the reasons behind the slow adoption, then it sheds more light on eHealth solutions for emergency healthcare and at the end of this chapter a statement of the problems in current solutions is presented. These problems, in fact, are generic to all eHealth solutions, however, the focus here is emergency healthcare in catastrophic situations. mHealth is a more recent term, it is a component of eHealth that has not been standardly defined yet. In 2013, Global Observatory for eHealth (GOe) defined mHealth (mobile health) as “medical and public health practice supported by mobile devices, such as mobile phones, patient monitoring devices, personal digital assistants (PDAs), and other wireless devices”. mHealth employs the mobile phone’s core utility; voice and short messaging service (SMS) in addition to the more complex functionalities and application. This includes but is not limited to; third and fourth generation mobile telecommunications (3G and 4G systems), global positioning system (GPS), and Bluetooth technology (World Health Organization, 2013). 2.2 History of eHealth/mHealth The first thought of eHealth was in 1960s when the idea of electronic health records (eHR) were discussed, but that idea was not considered seriously until 1991 in the United States. Many recommendations were given to implement this system, but the lack of standards required for full interoperability increased the anticipated complexity of such system and led to the delay. eHR discussion rekindled in 1999 when the United States Institute of Medicine (IOM) reported that prescription and medication errors could be prevented through computerized order systems. Since then, standards’ developments and technological advances have driven substantial progress in the eHR (Miller, 2006). The same idea was implemented in Canada in the early 80s (Canada & Canada, 2003). However, it was less advanced. They were known in Canada as, Electronic medical records (EMRs) that was defined as digitalizing medical records. The 90s witnessed slow start in eHealth adoption and development, although many countries showed interest and all the findings indicated growing momentum for eHealth uptake (Meireles et al., 2013). Acta Wasaensia 13 The millennium brought strong growth to the field with ICT slowly but steadily getting integrated into health systems and services worldwide. In 2008, WHO ran another survey to study the progress of eHealth in Europe and the findings showed strong political will for eHealth across the European region. It also showed solid progress in implementing foundation actions towards the adoption and actual implementation of enabling actions (Miller, 2008). Looking back at the history of eHealth it is fair to say that its adoption is very slow. Despite the progress achieved in the millennial, eHealth took 20 years to reach where it is right now. And here is why it took so long: (Miller, 2008)  The fragmented funding and governance of healthcare services.  Resistance of professions to changes in existing models of care.  Lack of rigorous research evidence on the benefits that might drive change.  Reluctance of politicians to be seen to be tampering with a politically- sensitive service.  Concerns about the costs and complexities associated with eHealth implementation.  Concerns about how it will affect practitioners and consumers.  There is a determined relationship between the progresses of eHealth and the country income group. Countries in the high-end and upper-middle income groups are more advanced in their eHealth development than those in the lower-middle- and low-end-income group countries. This explain why there is steady progress in European Union since 2002 and great success in Denmark since 1996 and Australia. Similar success would be expected in UK and US, but they had different reasons behind the slow adoption there: bureaucracy in UK and privacy concerns in the United States. 2.3 mHealth for emergency healthcare “Time is the most valuable thing to me.” — A family physician. Saving time is one of the major reasons why eHealth, and specifically mHealth, is appreciated by physicians and medical teams in emergency departments (ED). When the queue of patients in the Emergency Room (ER) grows, it is customary 14 Acta Wasaensia for the medical team to quickly assess the cases to prioritise and treat them efficiently. This assessment process is called triage, which refers to quick assessment of a patient in the Emergency Room (ER) with a view to define urgency of care and priorities in management. Triage evaluation can be completed in an organized and systematic manner relying on immediate visual and auditory assessment of appearance, breathing and circulation (Jayashree & Singhi, 2011). Triage process has evolved with the emerging technology, until eTriage came to light in 2008. It was presented as a new electronic triage application at The Scarborough Hospital (TSH), Ontario, Canada. This led to cutting triage time in half at the hospital and giving its multicultural patients the chance for more control over their reassessment while in the waiting room. This was part of the “Enhancing Emergency Services” project that combines an electronic application at the initial stage of triage process, with ongoing patient-controlled reassessment using kiosks located in the waiting room (Jones, 2008) . Before that and since 1998 (Jones, 2008) the same hospital has been tracking their patients’ case electronically at the emergency department. Another take at the eTriage was presented at the 7th International Conference on emerging Networking Experiments and Technologies (CoNEXT) that was held in December 2011 in Tokyo, Japan. The students proposed an eTriage system to handle mass casualty incidents (MCIs) such as earthquakes. The proposed solution was a wireless communication service platform consisting of electronic triage tags that combine a small vital sensor with a wireless device. The tag is attached to the casualty and their vital signs are gathered via the sensor and transmitted wirelessly via the ad hoc networks constructed by the electronic tag. The ad hoc networks were also used for localisation and local map generation to ease finding patients/casualties while monitoring vital signs (Jentsch et al., 2013). In 2012, a team in Norway presented eTriage under the umbrella of BRIDGE project that was funded by European Union’s Seventh Framework Program (SINTEF ICT, 2012). BRIDGE eTriage assists in marking and monitoring victims and in creating real-time situation awareness. It aims to ease the trigger’s task and bridge the process from triage to hospital admission. The system consists of 1) Triage Bracelet that is augmented with microelectronic components and various sensors. 2) a small device called Triage Relay that is intended to gather data from the incident field and transmit them to the in-charge person in the ER. 3) Triage Tablet that is used to visualise the triage data with localisation and local map. 4) more clip-on sensors to the patient’s body Acta Wasaensia 15 are added on-demand, such as heart rate, breathing rate, blood pressure, etc. Saving time could actually start before arriving at the ER! It has been proven that the first assessment of the patient/emergency case is crucial to their survival and that was the motive behind smart ambulances. The work on ambulance intelligence has started in 1996, when group of researchers at the National Technical University of Athens have successfully demonstrated real-time transmission of ECG data from a moving ambulance vehicle using GSM data links (Pavlopoulos et al., 1996). Soon after, a group of researchers at the University of Maryland Hospital have developed a wireless ambulance telemedicine system for stroke victims. However, applications were limited to storing and forwarding obtained bio signals although, in many emergency applications, real-time bio signal monitoring is needed (Pavlopoulos et al., 1998). In 1998, the ambulance project started in Athens University. The team developed a portable emergency telemedicine device that supports real-time transmission of critical bio signals as well as still images of the patients. The system allows a specialised physician to review critical bio signals and images of the patient and thus perform remote diagnosis and in return, provide specialized prehospital care. The transmission was all done over GSM networks, and it assumed a data rate of 9600 bps, which was the maximum at the time of the project (Pavlopoulos et al., 1998). In 2001, a team of two introduced a cost-effective portable tele-trauma system that assists healthcare centres in providing prehospital trauma care. They developed a software architecture with intelligent modules such as transcoding, differentiation, and congestion control to significantly improve the system transmission efficiency. The team claimed their system can accommodate much higher frame rates than the ones reported in previously proposed systems. They avoided high cost by using standard the available 3G wireless cellular data services, specifically, code-division multiple-access (CDMA)]. This tele-trauma system provides simultaneous transfer of video, still medical images, and ECG (Chu & Ganz, 2004). In 2013, a team of three developed a multifunctional telemedicine system for prehospital emergency medical services (Thelen et al., 2013). The system, like the others, allowed paramedics on the emergency scene to get a physician consult on the patient’s case and send the patient’s bio-signals along with still images to the hospital unit over GSM cellular network. The system was designed with user participation to assure practicality, and although built by engineers, the involvement of emergency physicians was significant. Inside the ambulance 16 Acta Wasaensia they’ve put a video camera, printer, and transceiver. Paramedics can take a monitor, defibrillator, headset and a portable transceiver outside of the ambulance. On the hospital side, where the tele-consultation centre is, they have a desktop application for displaying the scene and its data, they have touch-control panel and of course a headset to carry conversations with the paramedics. Transmission Control Protocol/Internet Protocol (TCP/IP) network traffic is established between the ambulance and the centre, where the transceiver is the gateway. All devices inside the ambulance are connected to the transceiver over the ambulance Local Area Network (LAN), while devices outside of the ambulance are connected via Bluetooth. Then wearables came to the picture, ranging from micro sensors that are seamlessly integrated in textiles through consumer electronics, like sensors that are embedded in fashionable clothes and computerized watches to belt-worn personal computers (PCs) with display units. This technology made it possible to detect and alert unobtrusively. In 2003, an advanced care and alert portable tele- medical monitor (AMON) was developed. It is a wearable medical monitoring and alert system that targets high-risk cardiac and/or respiratory attacks. It includes continuous collection and evaluation of multiple vital signs and it is equipped with intelligent multi-parameter medical emergency detection that is connected over the cellular network to a medical centre. The system was integrated into a wrist- worn device. This wrist band was designed with low-power techniques, which made continuous long-term monitoring possible and less restricting to the patients’ mobility (Anliker et al., 2004). Following that, another solution for prehospital assessment was delivered. The system is a third-generation universal mobile telecommunications system (UMTS) based and it delivers biomedical information from an ambulance to a hospital. This system transmitted voice, real- time video, electrocardiogram signals, and medical scans in a realistic cellular multiuser simulation environment (Gallego et al., 2005). Recently, the need for emergency solutions have been on the rise due to the world population and the life pace that causes more accidents than it solves. This is evident in the number of researches focusing on building smart ambulance and improving on emergency response rates and the like. Another group developed a smart ambulance system, although their motive was to improve the emergency response rate in India their solution is applicable globally. They (Gupta et al., 2016) explored advancing ambulance systems over Internet of Things (IoT), by collecting location coordinates via Global Positioning System (GPS) and using Google Maps Application Programming Interface (API) to plot the locations of ambulance cars, which could also be done for hospitals, in a way, they built an Uber-like service for ambulances. Patient’s health data is collected via medical equipment available on Acta Wasaensia 17 the ambulance and is sent to the hospital. The communications between the Android app and the database server at the hospital uses Representational State Transfer Application Programming Interface (REST API). Their idea is not new, however, they focused mostly on the mobile application rather than stationary system on the ambulance car. In early 2016, Hooman Samani and Rongbo Zhu developed a robotic automated external defibrillator (AED) to address cardiac arrest emergencies (Samani & Zhu, 2016). To survive such cases of cardiac arrests, patients need supervised AED within minutes of the occurrence. The robot (Ambubot) gets dispatched to the victim of a cardiac arrest as soon as detected via a sensor attached to the patient or active call via the mobile app. Both means send the GPS location to the robot, along with more information about the patient. The system also informs the family and calls for an actual ambulance. Later in 2016, a team of three (Kumar et al., 2016) have focused their efforts to develop a system that monitors the condition of elderly people using micro-electro mechanical system (MEMS) connected wirelessly to heart beat, body temperature, and vibration sensors. The parameters that describe the condition of the person are sent to an Android app via Bluetooth protocol. When emergency hits, the system sends a message to the server via GSM that includes the GPS location of the user. The novelty of the system is that it searches for the nearest ambulance and sends it to the address of the user, it also sends an SMS to a predefined relative. One research is worth mentioning as it focused on handling emergencies during disasters (Khoumbati et al. 2010). They developed a scheme called Medical Data Transmission Over Cellular Networks (MEDTOC), they transfer patients’ vital signs from the ambulance to the hospital over UMTS. The novelty is that they are aggregating the data of multiple patients; using special packet format that orders the data. After reviewing these trials, projects, and studies, it is obvious that the research to utilise eHealth for emergency healthcare and prehospital care operations is not near done. However, some of these solid trials are still in progress and yet to encourage full adoption. 2.4 Problems with current emergency solutions It is evident in the history of eHealth for emergency healthcare, particularly, the projects and trials dedicated to this field that many systems fail wide-scale practicality. Because in the design stage, most of these systems made big 18 Acta Wasaensia assumptions that were never matched back to reality. And while, assumptions are quite important at early stages of design, it is customary in design tasks to generalize and/or address assumptions at the end of trials. One could use assumptions to put the system in context, define use-cases, and build test-cases. However, these assumptions might force limitations on applicability, which might lead to impracticality in worst cases. These systems have failed to address their assumptions and, unfortunately, will end up being a liability instead of being a reliable system during catastrophes. This is axiomatic in eHealth solutions for emergency healthcare. One crucial limitation in such systems is assuming perfect conditions. For instance, large processing units, full coverage of wireless service, high speed Internet connections, low to no noise. In short, it presumes perfect transmission and perfect reception, where the message is preserved throughout the process. Unfortunately, this is seldom the case in many developing countries and rarely is the case in developed countries. Having any of the perfect condition characteristics is a luxury when in catastrophes and chaotic environments. And it is during catastrophes that people rely on emergency healthcare systems the most. That is why this assumption is dangerous; the result is building impractical systems that fail at the very first test. This is the problem that this research is trying to address; in less than perfect conditions, how reliable can mHealth solutions for emergency healthcare be? 2.5 Studies About Heart Conditions Due to the importance of the heart condition, many researches were dedicated to build monitoring systems, find measuring methods and analysis techniques. The goal of these researches is to speed up the process of diagnosing the heart condition, the difference is mostly in the degree of dependence on the doctor or the medical personnel. This area of eHealth is known as “Tele-cardiology”. In this chapter, some of the work that was published in the current decade is traced; in such fast-growing field of research looking beyond this decade would not be of worth. The papers discussed here were selected based on relevance to this research and are presented chronologically. In 2010, Tovar et al. presented their work of diagnosing heart murmurs by analysing phono-cardio-graphic (PCG) signal by proposing joint time-frequency distribution and present it in time-frequency map. Although, the team used a simulated signal to build and test the method, they applied their method to real Acta Wasaensia 19 patients’ signal obtained from a hospital (Tovar et al., 2010). The three patients in question had different condition severity; one healthy and two with medium- severe and severe, respectively. The results were satisfactory. In the same year, Sufi et al. proposed a mobile phone tele-cardiology system. They used five sample points of the QRS Complex, specifically the centroid and four extreme points on the cardioid of the QRS Complex (Sufi et al., 2010) to identify cardiac abnormalities instead of the usual hundreds, this has led to a faster and more efficient mechanism of cardiovascular disease (CVD) detection from ECG signal. The mechanism causes less computational burden that known mechanisms at the time, which made it suitable for wireless mobile phone based tele-cardiology applications. Also, that year, Tang et al. developed a method to separate heart sound signal from noise utilizing joint cycle frequency–time–frequency domains. In practice, they decomposed the heart sound signal into small components by means of a Gaussian modulation model, these components were characterized by time delay, frequency, amplitude, time width, and phase. These components assemble in the joint domains (Tang et al., 2010), while the noise component disseminated and with that, they managed to separate the heart sound signal components from noise based on fuzzy detection. The noise was simulated as non-Gaussian, nonstationary, and coloured noise. In 2011, Ding et al. took over the task of lowering the power consumption required for sensing the heart signal (ECG), this was done by introducing a novel method, compressed sensing (CS), to wearable ECG sensor (WES). In practice, the team sampled the analogue signals at sub-Nyquist rate at the analogue-digital converters (ADCs). The task was to classify the compressed measurement into normal and abnormal state rather than an actual diagnosis of the heart arrhythmia, they used wavelet transform for anomaly detection. When a cardiac anomaly was detected (Ding et al., 2011), the signal is stored in a memory and is then transferred to a cardiologist for further diagnosis of cardiac arrhythmias using the reconstructed signals from the compressed measurements, this step is done off-line and out of the system. The results showed that the method reduced the power consumption with 34% (Ding et al., 2011). In the same year, Su et al. developed an ECG analysis algorithm based on wavelet transform. To diagnose the heart condition, the algorithm locates the position of Q, R, and S wave in an ECG signal (Su et al., 2011) QRS waves hold great information about the heart condition; such as identifying cardiac arrhythmias by counting the number of QRS in a minute. The team have declared this as a noise regardless, effective and efficient process. 20 Acta Wasaensia Also, in that year, Nasrabadi and Kani developed a low budget phone-based ECG acquisition, analysis, and visualising system. It consists of microcontroller that mimics Holter device to read the ECG signal; electrodes in the chest area. The signal is then transmitted to mobile phone via Bluetooth protocol for display and analysis, for that they built a J2ME app. The analysis is based on locating the position of the QRS waves (Nasrabadi & Kani, 2011). In 2012, Mandal et al. developed a system that acquires the heart sound signal from the chest area, the signal goes directly to the connected PC where they deploy discrete wavelet transform to remove internal noise and reconstruct the de-noised heart sound signal. The system then uses a novel algorithm “end point detection” to detect nature of heart sound components M1, T1 of S1 and A2, P2 of S2, their locations, durations, frequencies present, length of cardiac cycle (Mandal et al., 2012). In the same year, Kumar et al. proposed a method to de-noise ECG signals using a hybrid technique. In practice, they combined Empirical Mode Decomposition (EMD) with wavelet thresholding (Kumar et al., 2012). They used EMD to decompose the signal and soft wavelet thresholding to remove the noise from the decomposed signal. The de-noised signal is then reconstructed from the series of intrinsic mode functions (result of decomposing). And, in the same year, Uslu & Biglin used local discrete Fourier transform to extract ECG signal features that help diagnosing the heart condition (Uslu & Bilgin, 2012). The locality based DFT is, in fact, deploying DFT after partitioning the signal into smaller frames, each frame consists of some samples, and then obtain the sequence in question by shifting window structure iteratively (Uslu & Bilgin, 2012). In 2013, Meireles et al. investigated new technique for heart diagnosing based on spatial recording of electrical heart activity, known as Vector-cardiogram (VCG) (Meireles et al., 2013). The idea is presented as a portable solution that records VCG and use digital signal processing for diagnosing the heart condition, particularly, Myocardial Infarction. They also study the possibility of converting VCG to 12-lead ECG. They used classical Finite Impulse Response (FIR) and Infinite Impulse Response (IIR) for noise cancellation. In the same year, Ishanka et al. developed a software tool to detect cardiac anomalies using heart sound (Perera et al.,2013). The heart sound signal is recorded from four locations in the chest area using electronic stethoscope. The signal is then de-noised using wavelet and decomposition. The detection is done by deploying few different algorithms (Perera et al., 2013). Acta Wasaensia 21 Moreover, in that year, another team realised the necessity for ECG classification systems and proposed a method for classifying ECG arrhythmias (Sarma et al., 2013). The proposed system uses artificial neural networks. In practice, they used fast Fourier transform for pre-processing the signal followed by linear prediction coefficients (LPC) and principal component analysis (PCA) for extracting the signal features. The actual classification is done using multilayer perceptron (MLP) artificial neural networks. Another research group did similar work, in the same year (Patel & Joshi, 2013), they developed an artificial neural network-based system for heart disease classification. They focused on stroke stage classification using multilayer feed forward network with back propagation learning algorithm. In 2014, group of researchers (Sani et al., Dec 2014) aimed to reduce the number of heart attack victims by proposing a framework for remote monitoring of heart attack diagnosis system for ambulatory patient. The system continuously monitors the cardiac markers of the patient and generates an alarm (SMS or call) when it reaches a predefined threshold. In practice, they used biosensors to detect the markers from the blood. The detected values are transmitted wirelessly utilising mobile phones/PDAs (over cellular networks) to the hospital system for storage and analysis. In the same year, Jabbar et al. hoped to make a difference in India and reduce the adverse reactions caused by not diagnosing heart diseases early enough. They developed alternating decision tree (ADTree) for early diagnosis of heart disease (Jabbar et al., 2014), this approach was a new type of classification rule at the time. It is based on decision tree; a data mining technique usually used for decision support process and machine learning. The team used principal component analysis (PCA) to gather features of the disease. In that same year, Alsalama and her supervisors tried using radial basis function networks (RBFN) with Gaussian function to classify heart diseases (Alsalamah et al., 2014), which is a learning system that reuses training datasets to reduce false classifications. And in the same year, group of researchers (Thiyagaraja et al., 2014) took over the task of developing a smart phone application to detect, monitory and analyse the split (delay between its two components) in second heart sound (S2). The heartbeat is recorded using a stethoscope. The team used fast Fourier transform to convert the sound signal to frequency domain and detect the first and second heart sound (S1, S2). They then use discrete wavelet transform to extract S2 and continuous wavelet transform to detect the characteristics of the heart sound 22 Acta Wasaensia signal; Aortic (A2) and the Pulmonic (P2). Those characteristics are used to calculate the split in S2. The application offered continuous monitoring and low- cost detection tool. Another group of researchers in the same year built a diagnostic system for heart diseases using fuzzy classifications technique (Krishnaiah et al., 2014). They modified uncertain unstructured data into “fuzzified” structured data using minimum Euclidean distance fuzzy K-NN classifier embedded with Symbolic approach and then classified the data. A team in Indonesia in that same year, built extreme learning Machine (ELM) based neural networks to diagnose heart disease (Fathurachman et al., 2014). This system is meant to overcome the long process of training neural networks, it is thought to be fast and require simple tuning. In the same year, group of researchers focused on diagnosing rheumatic heart disease using a mobile phone connected to stethoscope for auscultation, hoping for a cost-effective detection with no need for expert training (Springer et al., 2014). They used signal quality estimation techniques to overcome the limitation of the device primitiveness. Particularly, they used support vector machine (SVM) classifier with Gaussian kernel as a binary classification algorithm (good and bad quality). For sound segmentation, they used modified hidden semi-Markov models. Cabral and Oliveira worked on their own heart disease classification tool in that same year (Cabral et al., 2014). At the time, machine learning techniques have proven to be important tools for diagnosing several diseases and they aimed to find patients who are prone to cardiac disease before they show symptoms. They analysed medical data for cardiac diseases using five methods based on one-class classification (OCC) paradigm; kernel principal component analysis, feature boundaries detection, support vector machine, support vector data description, and Gaussian process OCC. For optimisation, they used particle swarm (PSO). In 2015, Jabbar et al. gave another try to heart disease diagnostic systems, this time using computational intelligence technique (Jabbar et al., 2015). They used discretisation method and genetic search to remove redundant features and optimisation, it is an enhancement to Naïve Bayes classification. Naïve Bayes is linear and probabilistic classifier that is based on Bayes theorem; all features are independent, and presence or absence of a disease depends on a feature itself (Jabbar et al., 2015). Acta Wasaensia 23 In the same year, another group of researchers tried to help physicians avoid misdiagnosing heart patients with an intelligent system (Olaniyi et al., 2015). It is modelled on multilayer neural network trained with backpropagation and simulated on feedforward neural network. They normalised the data before inputting it to the system; dividing each sample of a feature by the corresponding highest sample value. Another team used extreme learning machine (ELM) algorithm in that same year to model the independent factors leading to the diagnosis of a heart disease (Ismaeel et al., 2015). It is a warning system for probable presence of heart disease based on collected information about the patient; age, sex, serum cholesterol, blood sugar and more. In 2016, a team combined Naïve Bayes classifier with temporal association rules for coronary heart disease diagnosis (Orphanou et al., 2016). The features of the heart signal were temporal association rules annotated with the possible recurrence patterns of those features. They relied on several temporal data mining methods to analyse the signal; periodic temporal association rules (periodic TARs). In the same year, Kalaiselvi diagnosed heart disease using average K-nearest neighbour algorithm of data mining in a solo research (Kalaiselvi, 2016). The proposed algorithm is used to predict the heart disease with reduced number of attributes that are relevant to the disease. Moreover, in the same year, a team attempted developing a real-time automatic assessment of cardiac function in echocardiography (Storve et al., 2016). The system focused on estimating mitral annular excursion (MAE) and tissue Doppler parameters on cardiac ultrasound recordings to assess the heart condition. In the same year, a team proposed genetic algorithm based fuzzy decision support system (FDSS) for predicting the risk level of a heart disease (Paul et al., 2016). They pre-process the dataset, then use different methods to select effective attributes, these attributes help generating the fuzzy rules using genetic algorithm, which are used to build the FDSS that predicts the heart disease. Also, in that year, Feshki and Shijani worked on improving the heart disease diagnosis by using machine learning; particle swarm optimisation (PSO) and feedforward neural networks backpropagation (FFNNBP) (Feshki & Shijani, 2016). In practice, they used feature ranking on effective factors of disease by PSO and FFNN backpropagation. 24 Acta Wasaensia In the same year, Shi et al. developed a wireless stethoscope for recording heart and lung sounds (Shi et al., 2016). The goal is to ease continuous monitoring using stethoscopes. In practice, they developed a new method for analysing acoustic properties of the heart and lung sounds. After digitising the sound signal, they extract cardiac action parameters for analysis, combining this with lung sounds they got good insight into the cardiac and respiratory function. The signal was transmitted wirelessly to a receiver module that digitally filters the data and normalise the amplitude scaling. On the receiver side, they performed analysis over the acoustic properties of S1 and S2. In 2017, an attempt was made to reduce the time required to diagnose a heart failure (Manikandan, 2017) . This heart attack prediction system used a dataset from UCI Machine Learning Database, pre-processed the dataset using Rapid Miner and then investigated several algorithms to build the classifier, such as; Naïve Bayer, Decision Trees, K-Nearest Neighbour and Random Forest. It concluded that Naïve Bayer is the most fitting, where it resulted in 81.25% accuracy for the prediction. The system used 14 features to predict the failure. In the same year, a team proposed a Support Vector Machine (SVM)-based heart rhythm classifier. The system uses features like; timing, morphology, and spectral characteristics of the ECG to perform multi-source features and SVM for Atrial Fibrillation (AF) (Liu et al., 2017) . Also, in 2017, a novel method was proposed for heart sound classification without segmentation using Convolutional Neural Network (CNN) (Zhang & Han, 2017), where the different positions of the heart cycles are intercepted from the heart sound signals during the training phase. The spectrograms of the intercepted heart cycles are then scaled to a fixed size and input into the designed CNN architecture to generate features of different start positions in the testing phase. This has reduced the importance of the sound segmentation for prediction, the method was proven to be competitive when evaluated using public datasets. In 2018, the even detection approach based on deep recurrent neural networks was used to detect the position of state-sequence in a segmented heart sound (Messner et al., 2018) . Using this method, the researcher managed to detect the position of the first and second heart sounds (S1, S2) in heart sound recordings without incorporating a priori information of the state duration, which was also applicable to recordings with cardiac arrhythmias and extendable to detect extra heart sounds (S3, S4). Towards middle of 2018, a team attempted to automate the detection of abnormality in the heart sounds (Karaca et al., 2018). The processed Acta Wasaensia 25 phonocardiogram signals were classified to normal and abnormal using K-nearest neighbour method, with high accuracy that reached 98.2%. If there is anything in common between these researchers aside from their area of interest, it is the advancement of tele-cardiology as a result of their work. However, the level of advancement differs from one to another. Majority of these papers focused on heart disease diagnosis, whether by analysing ECG signal (electrical reflection of the heart) or the heart sound signal. Analysing ECG signal has been the core of tele-cardiology as the simplest form of representation as opposed to heart sound signal, which is far from being an easy interpretation. In fact, heart sounds interpretation is very subjective to the cardiologist’s experience and hearing ability (Health essentials, 2014). The researchers focused on reducing the time required for diagnosing heart conditions; by reducing the number of attributes to characterise a disease. Or optimising the extraction process of the rules around that disease. Or removing irrelevant attributes from the process. However, few researchers payed attention to the need to address noisy data, although noisy signal, whether ECG or sound, would hinder the diagnostic process massively. Noise that is caused by stethoscope friction against the skin and/or lungs’ respiration function is one thing but there is also the noise added once the signal is transmitted wirelessly. By now (see chapter 2) it has been established that perfect conditions are very rare, especially during catastrophic epidemics. And although none of the presented papers considered the practicality of their solutions during such circumstances, majority have presented mechanisms and algorithm optimisation rather than full systems compared to few who attempted developing an end-to-end solution. Furthermore, when a full system is developed, perfect conditions are assumed throughout the processes except for the case when a new device is proposed; researchers seem to avoid assumptions to maintain credibility of newly designed devices. However, the focus is usually put on removing noise more so than any other factor. The missing piece of the puzzle is clearly in addressing the conditions of the whole process; signal acquiring, signal analysis, signal transmission, and signal diagnosis. Each of these steps is a factor in the data quality (completeness, precision, validity, accuracy, consistency, timeliness, reasonableness, conformity, and integrity). They also factor in the power consumption (amount of energy consumed in every process), power consumption should be addressed in case of AC adapter sources and more so when it is battery-based. Not to mention, Speed and quality of the Internet connection used to transfer data between stations of the system. 26 Acta Wasaensia This research attempts to break the limitations rather than ignore them, by finding the sweet spot of low-quality data. Instead of assuming perfect conditions and highest quality, this research simulates low-quality signal and proposes methods and techniques to compensate the impact of the chaotic conditions. Acta Wasaensia 27 3 ARTIFICIAL NEURAL NETWORKS “Artificial Neural Networks (ANN) in the most general form is a machine that is designed to model the way in which the brain performs a particular task or function of interest” (Haykin, 1999) Modelling the heart acoustic propagation system is not an easy task, due to the complexity of the heart acoustic signal and the nature of the cardiovascular system. Moreover, understanding the impact of the organs in the human body on the sound and how the quality of the sound could affect the interpretation of the resultant sound signal, is all adding to this complexity. Therefore, the ICT applications in this field have been quite modest and mostly used machine learning and neural networks to approach the level of the advanced human brain that can be taught to interpret these complex signals. This is why it is important to study Artificial Neural Networks, and specifically their applications in heart diagnostics and heart health in general; neural networks will be used in this research to recognise patterns found in the heart acoustic signal to be able to classify each sound signal into a hypothetical disease. Hence, it is important to understand what is classification? And how could neural networks be used to classify a signal? And what are its applications? Classification is simply grouping things (in this case signals) based on similarity in their features and characteristics. Classifying objects is a survival instinct, animals must distinguish between threats, food, and potential mates. The brain often learns by association, quickly finding features that resembles known experiences and that is how it adds new objects to existing classes. Consequently, a neural network has to do the same job using neurons, their connections and arrangement. Next, the discussion goes into architecture of neural networks then will continue with the learning tasks focusing mostly on classification and pattern recognition. 3.1 Network Architecture The learning algorithm used to train a neural network is strongly connected to the structure of which the network that connects the neurons is like. There are different network structures that could be adopted, and the following is a summary of the classifications of these architectures (Haykin, 2009). 28 Acta Wasaensia 3.1.1 Single-Layer Feedforward Networks In this architecture, the neurons are structured to form layers. This network is rigorously a feedforward (acyclic) type. For instance, an input layer of source nodes projecting onto an output layer of neurons as computation nodes, but not vice versa. The term “single-layer” refers to the output layer of the neurons (computation nodes), input layers of source nodes are not counted since no computation is carried out there. Figure 3 shows an example of single-layer feedforward network with four nodes on input and output layers. Figure 3. Single-Layer Feedforward Network 3.1.2 Multilayer Feedforward Networks This architecture has hidden layers. The corresponding computation nodes (neurons) are called hidden neurons/units. The purpose of hidden neurons is to usefully interfere between external input and the network output. When a hidden layer or more is added to the network, it can extract higher-order statistics. Such network is considered fully connected, because each node in every layer in the network is connected to every node in every adjacent forward layer. If any synaptic connection is missing, the network is considered partially connected. Figure 4 shows an example of this type of architecture. The middle layer is hidden neurons. Acta Wasaensia 29 Figure 4. Fully connected Multilayer feedforward network Therefore, Multi-layer Feedforward Neural Networks are very suitable for complex classification such as heart signals, where the hidden layers compensate for the intermediate interconnected layers of the human brain and that made this architecture commonly used in pattern recognition (Fine, 1999) (Haykin, 2009). 3.1.3 Recurrent Networks This architecture has at least one feedback loop as opposed to only forward feedback in the previous architectures, which might have hidden layer or not. The feedback loop could be called self-feedback when the neuron feeds back into its own input. The feedback loops have a direct impact on the learning capability of the network, in addition, it implicates unit-delay elements that may lead to a nonlinear dynamical behaviour when the network contains nonlinear units. Figure 5 shows an example of this type of network. Figure 5. Simple Recurrent network (SRN) RNNs were developed in the 80s, and later many variations were developed to serve different purposes. Among which is Long Short-Term Memory (LSTM) that was proposed in the late 90s by Hochreiter and Schmidhuber. In 2018, a recurrent 30 Acta Wasaensia network was used to detect the position of the first heart sound (S1), also known as systole, and the second heart sound (S2), also known as diastole without using any a priori information about the state duration (Messner et al., 2018). 3.2 Learning Tasks The choice of a learning algorithm from the above is swayed by the learning task that the neural network is built to perform. There are at least six learning tasks that could be the purpose of neural networks, in this research the focus is put on pattern recognition (Haykin, 2009) . 3.2.1 Pattern Association This task expects the neural network to learn by association, more like the human memory. This could be auto-association, where a neural network is required to store set of patterns by repetitively showing them to the network using unsupervised learning. Or hetero-association, where an arbitrary set of patterns is paired with another arbitrary output patterns using supervised learning. 3.2.2 Pattern Recognition This task expects the neural network to assign a predefined classification or categorisation to a received pattern. This is done by enduring a training session, where sets of inputs along with their categorisation is presented to the network repetitively. Pattern recognition requires removing unwanted data/information from the input as much as possible, to reduce the error margin. This is known as denoising the signal, which is done by filtering. The filtering algorithm or method depends on the application, for instance for heart acoustic signals this could be, and most commonly is, Kalman Filter (Welch & Bishop, 2006) (Salleh et al., 2012). In rare cases when the signal-to-noise ratio is acceptable, this step is skipped. Next, the signal should be processed to identify unique features that distinguish one signal from another; this process is known as feature extraction. In the case of heart acoustic signals, this is about deriving the features of the disease. There are several methods and algorithms to extract features from signals; for example, Filter Banks and application of wavelet transform (Liung & Hartimo, 2002) (Tovar et al., 2010) , more about this topic in next chapter. Acta Wasaensia 31 The final stage of the pattern recognition is the classification, where the inputs are matched to particular category/class. The feedforward neural network is used widely in classification of signals, especially biological signals. There are many applications of this, for example, using deep feedforward neural networks, also known as convolutional neural networks, to classify heart sounds (Patel & Joshi, 2013) (Zhang & Han, 2017). 3.3 More Neural Networks Applications in Tele- cardiology The following applications show the impact of neural networks on tele-cardiology, and more specifically. And more so, the importance of feedforward neural networks in the field of classifying heart conditions. This section sheds light on relevant applications in the past three years. In 2016, feedforward neural network was used to predict the heart rate of a cyclist based on cycling cadence (Mutijarsa et al., 2016). The goal was to overcome the limitation of wearable sensors that do not read at regular intervals (e.g. 1 second, 2 seconds) by predicting the heart rate and complete the missing data. This should allow the cyclist to control the intensity of cycling and help them avoid risks of overtraining and heart attack. The feedforward neural network is used to model the mathematical relationship between the heart rate and cycling cadence. It expected the heart rate, the cadence on the second as input and gave the predictive value of the heart rate on the next second as output. They used large dataset for training and only 1% of the dataset size for testing, the experiment resulted in close prediction to measured heart rate values, with mean absolute error is 2.43 and 3.02 of training and test data, respectively. In 2017, two-layered perceptron neural network was used to analyse time- frequency cardio-rhythm-o-gram signal parameters using real-time heart rate value (Melnik et al., 2017). In another research in the same year Electrocardiogram (ECG) was used to detect several heart abnormalities, the accuracy of the ECG was improved using an artificial neural network with self-learning algorithm (Rastgar-Jazi et al., 2017). They used Main Lead II for extracting the features of the abnormalities in the ECG signals, which is known for feature extraction from ECG. Learning algorithms were not discussed in this research as they are out of scope, however, this does not reduce the relevance of this research. 32 Acta Wasaensia In 2018, a team analysed the spectral and statistical features of the Heart Rate Variability (HRV) signal to diagnose Obstructive Sleep Apnea (OSA) (Ali & Hossen, 2018) . HRV is a relatively new way to track well-being; it is a simple measurement of the variation in time between heartbeats (Campos, 2017). They used multiple artificial neural networks, including; single perceptron network, feedforward network with back-propagation, and the probabilistic neural network. The highest performance was achieved by feedforward network with back- propagation using wavelet-based frequency domain features with specificity, sensitivity, and accuracy of 90%, 100%, and 96.7%, respectively (Ali & Hossen, 2018). Another research in the same year, worked on classifying the fetal heart rate using convolutional neural network (Li et al., 2018). In their work, they divided the fetal heart rate into three classes; normal, suspicious, and abnormal. Then, they obtained records for each category from the hospital, and segmented each record into ten d-window segments and used convolutional neural network to process the data in parallel. And at the end, they used the voting method to determine the class of the record. Additionally, and to conduct a comparative study, they repeated the experiment using basic statistics feature extraction method and input the features into Support Vector Machine and Multi-layer perceptron to classify. Ultimately, the results had higher accuracy when using convolutional neural networks (Li et al., 2018). 3.4 Deep Learning Deep learning is new area of Machine Learning that was introduced as a step towards Artificial Intelligence (AI), it uses data to learn what was only thought to be possible for humans. This includes, perception; content recognition, prediction, and classification. It goes beyond simple learning algorithms to understand natural language and written documents, which makes it capable of making new discoveries. It is thought to be world changing science, especially for healthcare. It is not a separate science from Artificial Neural Networks; Deep Learning is a set of techniques that were discovered as an improvement to Neural Networks’ learning techniques. Table 1 below shows a simple comparison between Shallow and Deep Neural Networks (EDUCBA, 2018), although it is oversimplified, it gives a clear distinction between the two. Nonetheless, both are a class of machine learning algorithms where the artificial neuron forms the basic computational unit and networks are used to describe the interconnectivity among each other and can have units in multiple layers for feature transformation and extraction. Acta Wasaensia 33 Table 1. Shallow vs. Deep Neural Networks &&"% " #%&"! "* (% '*"% # (% '*"% ##%"+ '!' &&" " #"&'"! (!'"!&3 &%. ".5 "". :89?4 $(%&+#"!!',% !( %"'%!!#% '%& !& #" #+','! # 0 $(%&+#"!!',"*%!( %" '%!!#% '%&!& # " #+','!&"* 0 %''(% "%*% (% '*"%&/ " "!&'!" %''(%"!'!&'%&' ,%&'!#(',%*' &',%&'"('#(',%! '!'% %,,%&% '!,%&0 (%%!'!'*"%&/&!" %''(%"!&&'&"%' ,&!'"!!'"!%#0 "",%&' %''(%&!&"',"( %" *%,"(&'%'0 &%" #''"'%! !%+'% ,,! 0 , '%,"!!' !'*"%&/, '% "!!'"!"!%''(% *& "%"%&&' %(%%!'!'*"%&0,% %&'%'!!'(%('"'% (&"!%,(!'"!0 , '%,"!!'!'& *'!!'*"%&%!"*! &"'- !! !&*%& '"!&*'"(''! !'*"%%!"*!&"# !'&0 !&(#%)& %'%! '*"%&/!" "% '%!!.(''!'*"%&% #%'%!(&!#&'+#%!&0& !(&('"!"%&.# !'*"%&.!!%'))%&% !'*"%&0 "!)"('"! (%!'*"%&/ ' &'" %!%"%%'(%&(&! "!)"('"!&*''%&'  %"!'"!!!''"!(&% +#%!0 !''"!"&.&'%' &!&.#',#(&&!"'%"'& " &,(&!'&%''(%0 (%%!'!(%!'*"%&/,"  %" ' ,""%*%* &!&!!'%!"% '"!")% ' &'#&0 (%&)!(%!'*"%&/ '&" %& )%!'!#('0#% %, %!'*!%(%%!'! %(%&)&'''"% %&'', '")'%%&'%('(%&! ''%!!'&'*'''%&" #"&&'!"% '"!"('"*'' %%&'%('(%& !'!!' '&'0 However, there is a known problem in deep neural networks that is Steepest Descent, also known as vanishing gradient problem, which was limiting the depth of Neural Network severely. Neural Networks are trained using backpropagation gradient descent, that relies on updating the weights of each layer as a function of the derivative of the previous layer. However, the update signal was lost as the 34 Acta Wasaensia depth is increased and that is a problem, which causes two issues; local minima and saddle point. Local Minima can be explained as follows, during the iterative optimisation algorithm to minimise the loss function (i.e. how far is the performance of the network from being perfect) the Network finds a local minima and it stops optimising, while in fact the optimal performance (real minima of the loss function) has not been reached yet. Local minima problem is visualised in Figure 6. Saddle Points is a similar concept with local maxima in the other direction of the local minima. The network also has a learning rate that dictates the size of the step taken from one iteration to another when seeking optimal performance. Figure 6. Steepest Descent Problem (local minima) To mitigate this, Neural Networks were limited to smaller number of hidden layers and in some cases, they were preferred to have no loops; they were either feedforward or recurrent; because the more hidden layers you add to the network the worse the steepest descent problem becomes. However, in 2006 unsupervised pre-training before starting the gradient descent was suggested as a new mitigation method (Hinton et al., 2006). Stochastic Gradient Descent and Batch Gradient Descent have also been used to mitigate this problem. The basic equation to update gradient descent (optimise the network in every iteration), is 3;4 t F t Ë x I G» D TµØtÙµ~  Where t is the weight vector, which is used to subtract the gradient of the loss function with respect to the weights multiplied by the learning rate (x), since the steepest descent is the opposite to the gradient. The gradient is a vector that gives the direction of the loss function’s steepest ascent. k is the number of the iteration, and Tµ is the learning of the iteration. Now to improve the vanishing Acta Wasaensia 35 gradient problem using the techniques discussed above, Equation 3 is revised in Equation 4; 3<4 fmoi Î ;. -kÚt F t Ë x I G»TµØtÙÛ This means that the gradient of the loss function is taken at every step, which differs from the actual loss function that is summation of loss of every learning iteration. This gives one-iteration-loss, while Equation 3 gives all-iterations-loss which in turn may lead to a local minima or a saddle point. The depth of the Neural Network is measured with the number of hidden layers; at the beginning two or more hidden layers counted as deep network but this number had been increasing over the years, Figure 7 shows a simple deep neural network in comparison to a shallow neural network, this was adopted from (Nielsen, 2015). In 2014, GoogLeNet was release, which had 22 layers and 1024 weights, it had won Image-Net Large Scale Visual Recognition Competition (ILSVRC’14) that year (Szegedy et al., 2014). Figure 7. Simple Comparison between Shallow and Deep Neural Networks Apple uses deep learning for face recognition in its iPhone X as a biometric authentication method (YML, 2017). The phone is equipped with “neural engine” that is two processing cores to handle machine learning algorithms, such as face recognition and augmented reality apps. This could open the door for massive improvements to mHealth. It has been used largely in the medical field; to form diagnosis using images or sounds. When using images, it is referred to as “computer vision”, which is used by Microsoft’s InnerEye Initiative that started in 2010 (Microsoft, 2010). Currently working on image diagnostic tools for tumour detection, this went on to be used for automatic segmentation of aggressive brain tumours and overall simplification of otherwise delicate surgeries. 36 Acta Wasaensia It is doing an even better job at suggesting treatment ideas and options leaning from previous patients and finding what worked best on similar conditions. This was the result of the collaboration between the Oncology department at the Memorial Sloan Kettering with IBM Watson; the result an intelligence- augmenting tool that helps doctors deal with unique cancer cases (Memorial Sloan Kettering Cancer Center, 2018). Deep Learning has opened the door for insightful medical research and tremendous chances for improvements and simplification of difficult decision- making processes in the healthcare world. However, the core source of Deep Learning power is data; the bigger and relevant the data, the better the learning. Therefore, crowdsourced medical data collection is now important. It is simply, pooling data from mobile devices to aggregate health data; this is what Apple’s HealthKit, Samsung Health, FitBit, and the likes are doing. In fact, Apple is aiming to use the data collected through their devices to find treatment for Parkinson’s disease and Asperger’s syndrome. They have built interactive apps that patients can use to assess their conditions over time, of course the data collected through the apps are anonymised and fed into the data pool for future research into a possible treatment. Not only that, they actually have multiple apps; for autism, seizures, concussion, melanoma, and more (Apple, 2017). They have recently introduced a new model of Apple Watch that is equipped with three sensors to perform electrocardiogram that results an ECG signal of the heart. Traditionally, electrocardiogram is performed using electrodes on the skin; about six on the chest and in some cases one on every limb (Cables & Sensors, 2018). Therefore, Apple Watch is a practical step in the same direction of this research; performing diagnostics on the go. It is also equipped with an accelerometer and gyroscope to detect falls. The watch is expected to monitor the heart rate continuously and use a software application to “diagnose” the user’s condition based on this data that may result in contacting the emergency centre (Apple, 2018). The accuracy of this decision is now challenged by cardiologists who are emphasising the importance of having multiple electrodes and their placement around the body to obtain an ECG signal, however, until the date of this writing, there have not been concluding tests of this capability. Deep Learning is expected to change the future of cardiac imaging, especially segmentation. Segmentation is used to identify the pixels of interest in the cardiac image, it isolates the outline of the particular organ/area of interest from a Magnetic Resonance Imaging (MRI). It provides quantitative metrics of the segmented area of interest, and when combined with deep learning, it could be very insightful. For instance, for a cardiologist it would show the percentage of Acta Wasaensia 37 blood that is pumped out of the left ventricle during each heartbeat, this is known as Ejection Fraction (EF) (American Heart Association, 2017). It is usually performed manually, and it takes 30 minutes on average. When the quantitative metrics are fed into Convolutional Neural Networks (CNN) as a Deep Learning method, EF measurement is done in fraction of the time (Wetstone, 2018). The measurement of EF is quite essential to diagnose heart failure; heart’s ability to pump blood. This is measured in percentage; how much of the blood amount in the chamber is pumped out to the rest of the body. On average, a healthy human heart is expected to pump out 50 – 70% of the blood, this way the body can carry out biological processes as normal. 41 – 49% EF is called borderline EF, where the human body is expected to have shortness of breath during regular activities due to lack of oxygen. Less than 40% EF is called reduced EF, where shortness of breath can occur at rest. With such simple quantification designing a monitoring system with a built-in warnings mechanism will not be difficult, convolutional neural networks with deep learning algorithm is already in research to measure EF, the rest of about coding the warning mechanism. This would be very similar to the concept used in The Vital Transmitter that was developed by the author as part of the master’s thesis (Abdelmageed, 2012). 38 Acta Wasaensia 4 WAVELET TRANSFORMS As discussed in the previous chapter, the acoustic heart signal is very complex and requires advanced processing. Before inputting the signal into a feedforward neural network for classification with purpose of pattern recognition, the signal characteristics and features in time-frequency space should be extracted; and that is where wavelet transform comes to the picture. This chapter starts with overview of Wavelet Transform and then dives into Filter Banks as the technique that will be used later in the experiment for decompose the acoustic signal into single frequency sub-bands and ease the process of feature extraction. So, why wavelet transforms? In general, signals often have slowly changing trends or oscillations punctuated with transients, while images have smooth regions interrupted by edges and abrupt changes in contrast. These abrupt changes are frequently realised as the most interesting part of the data specially in terms of the information they provide. Fourier Transform is a powerful tool for data analysis, however, it does not represent abrupt changes efficiently. This is understandable, since Fourier Transform averages the signal over all time span, hence, information about when an abrupt change had occurred will disappear. Such abrupt changes can be observed in terms of frequency components, but it is not possible to determine at what time that had occurred. That is when the need for a representation that are well localised in time and frequency, something like short-time Fourier transom or wavelets (Devleker & MathWorks, 2016). In 1996, a group of engineers and physicians joined forces to analyse the heart sounds using wavelet transform (El-Asir et al., 1996). At the time and since 1979 (Blinowska et al., 1979), most of the computerised heart sound analysis was performed in the frequency domain and such applications were developed using Fourier transform or autoregressive spectral estimation techniques. As the case with Fourier Transform, Wavelet Transform is either Continuous (CWT) or Discrete (DWT). Where CWT is efficient for characterising oscillator behaviour in signals, while DWT is ideal for de-noising and compressing signals and images; as it represents many signals and images with fewer coefficients, this enables sparser representation. DWT process is equivalent to comparing a signal with discrete multi-rate filter banks (Devleker & MathWorks, 2016). The following example explains how DWT works; a signal S is first filtered with low-pass and high-pass filters A1 and D1, respectively. Half of the samples are discarded upon filtering as per Nyquist criterion. The filters, typically, have small Acta Wasaensia 39 number of coefficients and result in good computational performance, they also have the ability to reconstruct the sub-bands while cancelling any aliasing that occurs due to down-sampling. For the next level of decomposition, the low pass sub-band A1 is iteratively filtered by the same technique to yield narrower sub- bands A2, D2 and so on. The length of the coefficients in each sub-band is half of the number of coefficients in the proceeding stage. With this technique, the signal of interest can be captured with few large magnitude DWT coefficients while the noise in the signal results in smaller DWT coefficients, which helps analyse signals at progressively narrower sub-bands at different resolution along with de-noising and compressing signals (Devleker & MathWorks, 2016). In simple words, DWT is an operation that takes an input signal and decomposes it into its frequential components, a lot like Discrete Fourier Transform (DFT), and this operation is efficiently implemented by Filter Banks. Generally, Wavelet Transform is calculated using the following equation; 3=4 ^[ØfØa, bÙÙ Î ~C¬ J fØqÙ4I Þ º„­ ¬ ß dq ƒ§ „§  Where fØqÙ is the time-series signal being processed, a (a>0) is a scaling factor, b is a shift factor, and 4ØØ% Ë Ù0ØÙÙ is the daughter wavelet which is a scaled and shifted version of the mother wavelet 4Ø%Ù. in principle, the daughter wavelet is formed by scaling the mother wavelet by ØaÙ which relates to frequency and shifting it along fØqÙ by ØbÙ, then the similarity of the daughter wavelet to the original signal is calculated and recorded in the WT coefficient ^[fØa, bÙ. This process is repeated for all Øa, bÙ until the entire time-series signal and frequencies of interest are covered to obtain the coefficient matrix, which provides the spectral information through scaling while preserving the time domain information via shifting the wavelet across the signal. 4.1 Filter Banks A filter bank is an array of band-pass filters that separate the input signal into multiple components, where every component carries a single frequency sub-band of the original signal. Since Filter Banks involve various sampling rates, they are also referred to as multi-rate systems. They are widely used for multi-band equalization and in image and audio content analysis (Mertins, 1999). They are used to equalise graphics, and in digital signal processing at the receiver to down convert sub-bands to a low centre frequency that could be resampled at a lower rate. It is also used for signal compression, when some frequencies are more relevant than others. The Filter Bank of interest to this research is 40 Acta Wasaensia Multidimensional Filter Bank, which can be used to implement Wavelet Transform, and specifically DWT. For example, in an M-channel filter bank (see Figure 8), an input signal is decomposed into M sub-band signals by applying M analysis filters with different pass-bands. Each of the sub-band signals carry information on the input signal in a particular frequency band. Figure 8. M-channel filter bank (Mertins, 1999) The blocks with downward arrows indicate down-sampling (subsampling) by factor N, and the blocks with upwards arrows indicate up-sampling by factor N. Subsampling by N means that only the Nth sample is taken, which serves to reduce redundancies in the M sub-band signals. While up-sampling by N means insertion of N-1 consecutive zeros between the samples, which allows the original sampling rate to be recovered. Up-samplers are usually followed by filters that replace the inserted zeros with meaningful values. When M = N, the system represents critical subsampling, because that is when the maximum down-sampling factor of perfect reconstruction (the output signal is a copy of the input signal with no further distortion than a time shift and amplitude scaling) can be achieved (Mertins, 1999). Mathematically, a filter bank is a series expansion, where the sub-band signals are the coefficients and the time-shifted variants form the basis. The time- shifted variants represented in Equation 6 are of synthesis filter impulse response g´ØlÙ. 3>4 g´Øl Ë iVÙ, i H E Acta Wasaensia 41 4.1.1 Two-Channel Filter Bank Considering the two-channel filter bank in Figure 9, the signals can be represented as follow 3?4  }`ØwÙ Î ~ ÜR}ØwÙ_ØwÙ Ê R}ØËwÙ_ØËwÙÝ,   ~`ØwÙ Î ~ ÜR~ØwÙ_ØwÙ Ê R~ØËwÙ_ØËwÙÝ,  _ÒØwÙ Î Ü }`ØwÙQ}ØwÙ Ê ~`ØwÙQ~ØwÙÝ- From that, the input-output relation can be interpreted as 3@4 _ÒØwÙ Î ~ ÜR}ØwÙQ}ØwÙ Ê R~ØwÙQ~ØwÙÝ_ØwÙ   Ê~ ÜR}ØËwÙQ}ØwÙ Ê R~ØËwÙQ~ØwÙÝ_ØËwÙ Where the first term represents the transmission of signal _ØwÙ through the system, and the second term represents the aliasing component at the output of the filter bank. Figure 9. Two-channel filter bank The Perfect Reconstruction (PR) condition is achieved when the output signal is just a delayed version of the input signal. This is true when the transfer function for the signal component, ZØwÙ, satisfy 3A4 ZØwÙ Î R}ØwÙQ}ØwÙ Ê R~ØwÙQ~ØwÙ Î 4  Î : Ï I L  Î : Ï I Î  0 Ñ I Î 0’ Ñ I Which is equivalent to diodes (an electronic component that allows an electric current to flow in one direction). Diodes allows the current S to flow when the 52 Acta Wasaensia critical potential difference ]I with small resistance Y°. Consequently, the valves can be seen as the nonlinear component in this emulated system. 5.1.1.5 The Windkessel Model The previous electrical models are used in different version of models of the heart and its functions, for instance, the arterial system can be modelled by the Windkessel model. This one is based on the physical characteristics of blood vessels, it describes the pulse-wave propagation of the blood. Large vessels like the aorta and its branches have high elasticity and so they are called Windkessel vessels. Windkessel is a German term for air chamber (Åstrand & Rodahl, 1977), which stores air and affect the velocity and pressure of the fluid flowing through the system pipes. These vessels mimic the air chamber by storing a volume of the blood, which also affect the velocity and pressure of the blood flowing through them. In practice, this comprises of capacitor to represent compliant aorta and a resistor to represent stiffer peripheral vessels connected in parallel. 5.1.1.6 Mossa’s engineering model It is a generalisation of the Ursino model (Ursino, 1998) , except that in this model the heart is considered a pulsatile pump. This amend makes it possible to simulate the task of the ventricles by an elastance variable model. The hydraulic analogue of the cardiovascular system is represented in Figure 13. Where the vascular system is divided into eight compartments, five of which replicate the systemic circulation. They represent the systemic arteries (sa), splanchnic peripheral (sp), and the venous (sv) circulations. While the remaining three replicate the arterial (pa), peripheral (pp), and venous pulmonary (pv) circulation. Acta Wasaensia 53 Figure 13. Hydraulic analogue of the cardiovascular system C and R refer to capacitor and resistor, respectively. The subscripts next to each refers to the compartment it represents (refer to previous paragraph). The subscripts la, lv, ra, and rv refer to the left atrium, left ventricle, right atrium, and right ventricle, respectively. Rj is the hydraulic resistance that represents the pressure energy losses in the jth compartment. While Cj is the compliance that represents the amount of stressed blood volume stored at a given pressure W and flow P, unstressed blood volume Vu, j. According to (Ursino, 1998), the inertia effect could be ignored anywhere except for the large artery compartment where the blood acceleration is significant enough. Now for the equations; they were written by Mossa assuming few things: 1. Preservation of mass at the capacities in Figure 13. 2. Equilibrium of forces at inertances (is a measure of the pressure difference in a fluid required to cause a change in flow-rate with time) Lj. 3. Total amount of blood contained initially in the vascular system is 5,300 ml (Ursino et al., 1994). To make this easier, Figure 13 is redrawn with electrical components only. Figure 14 represents nonlinear closed-loop lumped parameter model of the intact pulsatile heart and its circulations. Left and right sides of the heart have the same model while the parameters’ values vary. The constant value of the compliance is 54 Acta Wasaensia used to characterize the linear capacity, which describes the atrium; the contractile activity of the atrium is neglected. Since the arteries and veins width vary through the human body, the representing capacitance and resistance for blood flow are dependent on that. The pressure generated by the hear movement is time-varying, which is indicated by the capacitors with arrow through them in Figure 14. To represent a reference pressure based on other pressure differences, electrical earth is used as an indication of zero voltage. As known, the blood goes from the atrium to the ventricle via the atrioventricular valve, simulated as the series arrangement of an ideal unidirectional valve in series with a constant resistance (Mossa, 2008). Following the assumptions listed above, conserving of mass and balance of forces across the different compartments in Figure 13 The shapes used in the closed-loop lumped parameter model of the cardio system are numbered on the side and explained below: (1)Electrical resistance used here as vessel resistance (2)Electrical capacitance used here as vessel compliance (3)Magnetic inductance used here as blood inertia (4)Diode used here as valve. Figure 14. Closed-loop lumped model of the cardiovascular system. Acta Wasaensia 55 The upper circuit of Figure 14 gives the pulmonary arteries. When the cardiac muscles are relaxed (diastole), the ventricle fills through an exponential pressure/volume function. This reflect elasticity on both the relaxed muscle and the external constraints related to it, which is mainly the pericardium. While on systole, the ventricle fills through a linear pressure/volume function with adopted slope. This is called elastance at the instant of maximal contraction, and it is demoted by E(max). The constants k (r and E) refer to the ventricle resistance and the end-diastolic pressure-volume relationship for the heart. Vu is the unstressed ventricle volume (Mossa, 2008). To move from end-diastolic to end-systolic the relationship is controlled by a pulsating activation function that is 3ØqÙ, with period T equal to the basal cardiac cycle that is 0.833 seconds. This period steers P(max, lv) the isometric left ventricle pressure. The controlling function is controlled by baroreflex control system, known as a highly complex function of the sinus nerves. As a try to simplify the process an approximation of the ventricle activation function is used, it is found with a simple sine function with | = 1.25 radian, and the signal frequency corresponds to the cardiac cycle. Finally, the blood flow. It leaves the ventricle only when the aortic valve is open, it also depends on the difference between the isometric ventricle pressure and arterial pressure (afterload). To solve these functions, Mossa came up with a strategy of two steps: 1. Desired average pressure and volume distribution and cardiac output determine the parameter values of compliance and resistance. 2. Inductance and elastance functions are assigned values that generate representative pressure and flow pulses. The cardiovascular parameters must be selected following guidance provided by data rescaled for a patient with 70KG body weight. The average pressure level should be realistic and so is the volume distribution. From there the computed ventricular pressure, root aortic pressure, and outflow curves should look as close as possible to the corresponding human values. In basal condition the following values characterise the cardiovascular system. 5.2 Phonocardiogram signal (PCG) Before discussing the mathematical models of the heart acoustic signal, it is important to discuss the heart sound signal and review the literature available in 56 Acta Wasaensia that area. The first step any physician does when checking on a patient is to listen to their heart and lungs using stethoscope. Whether the patient is here for an emergency or a regular check-up, this step is essential. That is because the heart movement generates a sound and so does the pulmonary system. This sound holds significant information about the health of the patient, specifically about their cardiovascular and pulmonary systems. The information held in the heart sound does not only indicate heart healthiness, but vessels and blood health as well. However, the heart sound signal that is heard by human ear is not informative, due to the subjectivity of its interpretation that even experienced cardiologist might misinterpret clear indications. Although the phonocardiogram signal does map directly to the ECG; where every pressure change in the heart is translated into electrical change that shows in the ECG signal, the limitation comes from the human hearing range. The human hearing range cover 20 Hz to 20k Hz, while the heart sound signal may contain other frequencies out of this range. However, some parts of the signal tend to be very weak to be captured by the standard analogue stethoscope. Moreover, the heart signal is considered quasi-periodic; with a period-time of about one second for adults (infants have a smaller period-time). Thus, the spectrum of the heart acoustic signal is discrete with large number of frequency components and the frequency resolution of the spectrum is around 1 Hz. Consequently, if the maximum frequency of the heart sound is considered to be 500 Hz, then the spectrum will have 500 frequency components. It is assumed that each frequency component along with the time information (information extracted by using for example Wavelet transform), may carry insightful information about the heart condition. However, to the best of the author’s knowledge, there has not been a study that claims or proves this yet in the literature. Furthermore, a direct mapping of the mathematical relation between each heart frequency component and all possible heart diseases is cumbersome if at all possible. Therefore, machine learning seems to be the optimal solution in this case. If the heart sound signal is obtained, it has marvellous potentials to diagnose heart conditions in timely fashion or even at an early stage. This is what sparked the idea of simplifying the acquisition and analysis of the heart sound by finding a new possible spot in the human body besides the chest area to capture the sound wave. The hardware realisation is not the focus of the research, however, the modelling of such system and the analysis of the acoustic signal for diagnosing purposes is. In this chapter, different approaches to model the acoustic wave propagation is studied to find the suitable approach for our experiment. Acta Wasaensia 57 5.2.1 Definitions and Descriptions To recap the cardiac cycle; the heart generates an electrical activity, which causes atrial and ventricular contractions. These contractions force the blood between the chambers of the heart and around the body. The blood accelerates and decelerate following the opening and closure of the heart valves, which in turn increases the vibrations of the entire cardiac structure (the heart sound and murmurs) (Leatham, 1975). Heart murmur is an unusual sound heard between heartbeats. Murmurs sometimes sound like a whooshing or swishing noise. Murmurs may be harmless, also called innocent, or abnormal (NHLBI, 2016). In other words, the heart sound reflects the turbulence created when the heart valves close. The vibrations are audible at the chest wall, that is where the physicians place the stethoscope. The physician moves the stethoscope in the chest area while the patient is sitting up or reclined about 45 degrees. The stethoscope is placed in four areas (Tidy, 2015); • Mitral area: at the apex beat, that is when the left ventricle is closest to the thoracic cage. The stethoscope is placed on the left side of the body at the end of the chest muscle, below the areolas to be. • Tricuspid area: inferior right sternal margin is the point closest to the valve. The stethoscope is placed on the level of the left areolas towards the middle of the chest. • Pulmonary area: left second intercostal space close to the sternum is where the infundibulum is closest to the thoracic cage. The stethoscope is placed on the on the same vertical line of the tricuspid area upwards, about 4 inches below the collar bone. • Aortic area: right second intercostal space close to the sternum, that is when the ascending aorta is nearest to the thoracic cage. The stethoscope is placed on the same level of the pulmonary area to the right. The stethoscope bell is used to hear the lower-frequency sounds, while the diaphragm is used for higher frequencies. However, if the physician suspects anything further, they call for an echocardiogram. The fundamental heart sounds (FHSs) give an indication of the health of the heart as said before. The graphical representation of the heart sound recording is called phonocardiogram (PCG). And it consists of the following: • S1 this is the first heart sound; it forms the “lub” in the “lub-dub”. Caused by the closure of the atrioventricular valves; tricuspid and mitral. It 58 Acta Wasaensia consists of the mitral valve closure M1 and the tricuspid valve closure T1, they normally come in this order. • S2 is the second heart sound, it forms the “dub” in the “lub-dub”. Caused by the closure of the semilunar valves; aortic and pulmonary. It consists of the aortic valve closure A2 and the pulmonary valve closure P2, they normally come in this order. In some cases, the heart has more sounds than first and second, those are rare cases, but they do not necessarily reflect any abnormality. Such as: • S3 which is called a proto-diastolic gallop, occurs at the beginning of diastole after S2 with lower pitch than S1. The heart sounds more like “lub- dub-ta” instead of the usual “lub-dub”. • S4 which is called the atrial gallop, it is produced by the sound of blood being forced info hypertrophic ventricle. The heart sounds more like “ta- lub-dub” instead of the usual “lub-dub”. A normal heart would have short sound wave at the end of the diastole phase and again at the end of the systole phases. Shorter and less significant waves during the second diastole and then it repeats. Figure 15 shows the signal with more details about the pressure caused by every valve closure. To analyse the FHSs, the first step is to segment the signal. It is essential to localise the FHSs correctly to identify the systolic and diastolic regions. Many segmentation methods have been developed before, first published algorithm was in 1997 (Liang et al., 1997), it was based on Heart Sound Envelogram. The latest was in 2015 (Springer et al., 2016) and it was based on hidden semi Markov model (HSMM), what makes this work special is that it segments the first and second heart sounds within noisy real-world PCG using HSMM extended with the use of logistic regression for emission probability estimation. Acta Wasaensia 59 Figure 15. Heart sound signal (Creative Commons). Classifying heart sound recordings is not a new science, there are about 50 years old trials, however, it is still considered a challenging task. The first try was in 1963 attempted by (Gerbarg et al., 1963), their focus was to identify children with rheumatic heart disease using threshold-based method. Looking at the history of heart sound classification, the most widely used method is Artificial Neural Networks (ANNs), as a machine learning approach for this classification. Some researchers used wavelet transform to extract features held in the frequency component (Liung & Hartimo, 2002), (Schmidt et al., 2011), and other use it to extract time-frequency features (de Vos & Blanckenberg, 2007). In more recent work, some have also used support vector machines (SVM). As a feature extraction method, in practice, some researchers used wavelet (Ari et al., 2010), time- frequency feature-based classifier (Maglogiannis et al., 2009) and in recent work, some used Hidden Markov Models (HMM) (Wang et al., 2007) (Saraçoğlu, 2012). Others used clustering-based classification; k-nearest neighbours (kNN) algorithm (Bentley et al., 1998) (Martínez-Vargas et al., 2012) . Although some of these tries demonstrated potential for accurate detect of pathology in PCG recordings, unfortunately, they all share the following flaws:  Good performance is limited to carefully selected data. 60 Acta Wasaensia  Lack of separate test dataset.  Failure to use a variety of PCG recordings.  Validation on only clean recordings. 5.2.2 Characteristics of the Heart Acoustic Wave It is important to remember that the cardiovascular system, the system that generates the acoustic wave in study is nonstationary; it changes with time. It is also semi-periodic and non-ergodic. On average, the human heart takes about 0.6 – 1 seconds to complete a cardiac cycle, which is described in Figure 15. The acoustic signal has information in both domains; time and frequency. Since it is semi-periodic in Time Domain and discrete pens in Frequency Domain. Figure 15 is a part of the Wiggers Diagram that was developed by dr. Carl J. Wiggers to explain the cardiac cycle. It is a single grid that shows; Electrocardiogram, Aortic Pressure, Ventricular Pressure, Atrial Pressure, Phonocardiogram, and Ventricular Volume. The complete Wiggers Diagram is shown in Figure 16; the X-axis represents time while the y-axis represents pressure, volume, or amplitude depending on the type of signal at hand. Acta Wasaensia 61 Figure 16. Wiggers Diagram (Creative Commons). Each of the signals shown in Figure 16 give insights into the cardiac cycle. In fact, every signal can be studied separately or in conjunction with another signal to diagnose the cardiovascular system and the heart condition. The heart acoustic wave has low intensity and low frequency, first sound (S1) has components spanning the band of 10 Hz – 140 Hz and second sound (S2) has components that span the band of 10 Hz – 400 Hz. Despite the low frequency, the energy content in the heart vibrations is much higher than that of a high frequency one. While the third and fourth sounds (S3, S4) have very low frequency. Moreover, heart sounds have short duration, they are transient for 50 – 100 milliseconds. Additionally, the properties of the frequency of heart sounds vary with time, which makes them non-stationary. Due to the complexity of the heart activity that generates these sounds, the components overlap. The heart acoustic wave velocity in human body is approximately 1540 m/s, and the velocity is high when the wave goes through solid/stiff medium, such as bones. While it is slower through fluid, such as blood and even slower through gas, such as air. Due to the medium impedance the velocity change. Table 2 below, shows different velocity values that corresponds to different media (Azhari, 2010). 62 Acta Wasaensia Table 2. Sound velocity in different media (   "',3 1&"!4 "" 9=?= (&3"!'%&4 9=?= (&3%"&&'%&4 9=A8 "! <8@8 "''&&( 9=<8 ' 9<=8 '% 9<@8 Having discussed this, it is important to realise that every medium in the body that the blood travels through or passes by has an impact on the quality of the signal. This is not limited to the velocity of flowing that is discussed above, but also the power of the signal. From physics point of view, each medium/organ has its own absorption rate that corresponds to its density. Furthermore, every organ in the propagation path causes the signal to bounce in different direction (scatter) and be absorbed by different organs in these new paths. Nonetheless, the sound waves of low frequency will not experience this effect; since the dimensions of the organ is smaller than the wavelength of the sound signal in this case. Not to mention that organs comprise of extremely small cells compared to the wavelength of the sound wave, which adds extremely small effect to low frequency sounds passing through, such effects are very difficult to measure and can be ignored for practical reasons (Skille & Nielsen, 2017). Acta Wasaensia 63 6 MODELLING HEART ACOUSTIC WAVE PROPAGATION The research in this area have been dominated by two approaches; deterministic and stochastic. The stochastic approach considers the uncertainties and noise in the model. However, both approaches are valid and have satisfactory results. One may interpret the deterministic approach as an averaging of the stochastic one. In this chapter, both approaches are briefly discussed to choose the model that will be used in the experiment. The term “Analytical approach” is used here to express the deterministic approach. 6.1 Analytical Approach: Attenuation This approach is popular for studying ultrasonic intra-body communications (IBC) and sensors. However, the frequency band does not force any limitation on the application of this model, in fact it could be equally applied to any other frequency band, such as low frequency. This assumption was made because the same approach was used to model ultrasonic communications (Galluccio et al., 2012) and to model opto-ultrasonic communications (Santagati et al., 2013). Where the first research limited the maximum frequency to 1GHz to be able to operate with small devices and limited maximum attenuation, and the second one discussed frequency band up to 1THz. Furthermore, the assumption was confirmed in 2016, when the same approach was used to model heart acoustic signals (Shi & Chiao, 2016). In this approach, the heart acoustic wave propagation system is straightforward, it is modelled in Figure 17 as an attenuation block function that transfer the heart sound into the recorded sound at the measuring spot. Figure 17. Propagation Model of heart acoustic wave (Analytical) when an acoustic wave propagates through the human body, the signal experiences exponential decay in its amplitude. This loss is known as “attenuation” and it happens due to the absorption, scattering, and reflection caused by the human tissue; bone, muscle, blood, fat, other organs, etc. Multipath is not discussed here because when heart acoustic wave propagates in the human body, it comes across 64 Acta Wasaensia irregular shaped organs that are small in relation to the wavelength of the acoustic wave (y Î s0fÙ , these organs are weak reflectors, because of that, scattered reflection is discussed instead. The amplitude decay is modelled as (Kinahan, 2006) ; 39?4 MØwÙ Î M}e„¾É¼ Where z¨is the amplitude attenuation factor, measured in ck„~. The amplitude loss is measured in decibel as 39@4 "&&B<: "~} ިؼ٨À ß The attenuation coefficient x of a material, also known as the absorption coefficient, is a variable of the tissue acoustic properties (a, b) and the frequency (f) of the acoustic wave, modelled in Equation 19 as: 39A4 x Î af­ x is measured in ln-k„~ , b is often roughly estimated as 1, while a can be measured as a Î x0f in ln-k„~Rw„­. The attenuation can also be defined as 3:84 x Î <: "~}ØÙ- 7¨ K A-@7¨ With that, Equation (16) can be reformulated as 3:94 MØw, fÙ Î M}e„¬±¼0‚- Table 3. Acoustic attenuation (5) of human body tissues &&( x3 #1 4';:- x3 #1 4';=:- x3 #1 4'=::- "" 9=+982= :9+982= ><+982= (& ;;+982= <@+982= 9=+982< "! ==+982< ?>+982< ::+982; ' :8+982= :@+982= @A+982= '% :=+982? ;=+982? 98+982> && ?8+982? A@+982= :@+982< In Table 3, the acoustic attenuation of the human body tissue is presented (Bryn A. Lloyd, 2010). On the propagation path between the heart and the wrist there is blood, blood vessels, muscle and bone tissues, fat cells and water. Each tissue causes different attenuation depending on the frequency of the acoustic wave. Acta Wasaensia 65 This can be mapped directly to heart acoustic wave propagating through human body, but since the entire heart mechanism is based on pressure changes (refer to chapter 4), the equation should be rewritten for pressure instead of amplitude, it is the initial pressure W}experiences exponential decay (Santagati & Melodia, 2013) (Santagati et al., 2013) (Galluccio et al., 2012). 3::4 WØdÙ Î W}e„½¯  While (d) represents the distance travelled by the acoustic wave. Agitation of the molecules of these tissues cause thermal noise (Galluccio et al., 2012). Moreover, there are many sources of noises and uncertainties at the band of wave propagation. Hence, and based on the central limit theorem, this noise can be expressed as additive Gaussian. The power spectral density (PSD) of this noise in dB can be given as (Santagati & Melodia, 2013) (Santagati et al., 2013) (Galluccio et al., 2012) 3:;4  ¯©ØÙ Î Ë;> Ê <: "ØÙ%μ #%- And consequently, the signal-to-noise-ratio (SNR) is calculated as 3:<4  Ø, Ù Î Š0†Ø‘,“ىؓÙ֓  Where W is the transmitted power, VØfÙ is the PSD of the thermal noise, MØd, fÙ is the total attenuation experienced by the transmitted pressure signal. 3:=4 Ø, Ù Î „¤‘ Towards the end of this research a team of two attempted building heart sound monitor using wearable wrist sensor (Shi & Chiao, 2016). Although they focused on the hardware, the mathematical model of the heart acoustic system was based on the analytical approach discussed here. The wrist sensor was designed to detect heart pulses by measuring pressure changes to calculate the heartbeats. As previously discussed in this research, the sound wave changes with the change of blood pressure, which is inevitable when the sound wave propagates through the blood vessels (following the blood flow), this applies to pulse wave as well. In (Shi & Chiao, 2016), they worked on finding the inverse function for the pulse wave between two locations along the same artery; chest and wrist and used that to estimate the recorded pulse wave at the chest from the one recorded at the wrist. Figure 18 shows the model that was presented in (Shi & Chiao, 2016), it represents the travel of the pulse wave sound in blood vessels. The job is now about inversing the effect of the blocks in this model, once the pulse wave was captured at the chest and the wrist, the two were compared to find the relation between them. The delay 66 Acta Wasaensia correlates with the distance travelled from the heart to the wrist; they relied on the pulse peaks in time domain to estimate the delay using Short-Time Fourier Transform (STFT). They then trained a two-layer feed-forward backpropagation neural networks to find the inverse attenuation function. The main task here was to validate the recorded pulse wave at the wrist and consequently validate the proposed hardware. Figure 18. Travel model of pulse wave sounds in blood vessels. 6.1.1 Discussion Assume the analytical approach is used to model the heart acoustic system propagating from the heart to the left wrist, now apply Equation 21. To do so, and to apply the proposed model along with the mathematical equations, few variables must be found first, starting with the known facts. • The tissues found in the path between the heart and the wrist are; bone, muscle, blood, blood vessels, and in some cases fat • The typical frequencies of the heart sounds; f«~; Î ;:Rw,f«~< Î ;=:Rw, f«; Î ;:Rw, f«< Î =::Rw. • The attenuation coefficient x of every tissue in every frequency is found in Table 3 • Tissue acoustic properties (a, b) o For simplicity, let us assume b Î ;. This is the common case o Find a Î x0f for every x and every f, see Table 4 for values • Now to find W} of the recorded heart sound wave using MATLAB o The heart sound used in this test is an mp3 audio file recorded with two stereo channels, total of 2383872 samples recorded with Acta Wasaensia 67 sample rate of 44.1 kHz and bit rate of 33.8140 bps, this information was found using the function audioinfo() in MATLAB o Initial pressure (W}) is the pressure of the recorded heart sound wave. It can be found using Sound Pressure Level calculator (SPL) function in MATLAB, this function was defined by Chad Greene (Chad Greene, 2012) o pnj gives the sound pressure in dB and it takes into consideration pressure reference of the medium (in this case, blood) o The average blood pressure of an adult is 105/70 mmHg, which is equivalent to 1.5 mmHg, this is equal to 0.01125 Pascal o W} in dB, spl (sound_wave_vector, pressure_reference), which was found to be 21.6937 dB o left channel sound was used here, this was found using the function sound_wave_vector = sound_wave(:,1) in MATLAB Table 4. Acoustic properties for human tissue between heart and wrist &&( a Î x0ff«~; f«~< f«; f«< "" 9=+982> 9=+982? 9=+982> 9>+982? (& ;;+982> ;<+982? ;;+982> ;?+982? "! ==+982= ==+982> ==+982= ==+982> ' :8+982> :8+982? :8+982> :: +982? "")&& ?+982> ?+982> ?+982> ?+982> • And finally, to calculate WØdÙ using Equation (22) o Distance from the heart to the left wrist is approximately 0.78 m o WØ:-@AÙ Î <;-?Be„}-‚¬± o WØ:-@AÙ will have different values for every frequency and tissue, following the acoustic properties of each. See Table 5 for all values The results shown in Table 5 indicate that the pressure of the measured sound wave at the wrist is almost the same as the initial pressure. However, this does not reflect the real case scenario (heart-wrist system) accurately because it is a known 68 Acta Wasaensia fact that the pressure is not uniform throughout the human body. This happens because the heart pumps the blood to the entire body, and it circulates back against gravity. The pressure drops as the distance from the heart increases (Luiz et al., 2006). In conclusion, this approach is simple to use, and it gives relatively good results, nevertheless, it might be more suitable for practical applications where the output signal can be measured rather than simulated as is the case here, which makes (Shi & Chiao, 2016) a valid use case. Table 5. Calculating P(d) for S1 and S2 frequencies in different tissues &&( WØ:-@AÙ Î <;-?Be„}-‚¬±f«~; f«~< f«; f«< "" :90>@ :9.>@ :90>@ :90>?A (& :90>A :90>@ :90>A :90>> "! :90=A? :90=> :90=A? :90;: ' :90>@> :90>A :90>@> :90>?= "")&& :90>A :90>? :90>A :90>< Allthough the heart sound spans through great number of frequencies depending on its condition, in this discussion the minimum and maximum values are used to simplify the concept. Technically, every frequency component of this signal is affected by an attenuation corresponding to its frequency value. 6.2 Stochastic Approach This approach has been adopted by Dr. Durand in 1989 (Durand et al., 1990). It is interested in studying the properties of the signal in frequency domain. Ideally, this would be achieved using Probability Density Function (PDF), however, it is not feasible to find PDF of semi-period signal such as the heart signal, and that is why this approach settled for Power Spectral Density (PSD). Durand’s work in analysing heart acoustic wave in dogs started early 80s, he and his team published series of papers documenting the modelling of the heart-thorax acoustic system followed by analysing the effects of different conditions using M1 (one of the major components of S1, the other is T1) and A2 (one of the major components of S2, the other is P2) from the first heart sound S1 and its second S2, respectively. Their work has been highly appreciated and has been since cited nonstop around heart sound wave modelling and analysis. Figure 19 shows the Acta Wasaensia 69 complete model that Durand had started in 1985 (Durand et al., 1985) with the additional Ø%Ù and &Ø%Ù as the instrumental noise. Figure 19. Durand’s basic model of the heart/thorax acoustic system Where; • PCG represented by uØqÙ • the instrumental noise of the recording system at the input is represented by kØqÙ • rØqÙ represents the PCG measured at the system input (sum of uØqÙ and kØqÙ) • lØqÙ is background noise at the output • vØqÙ is the PCG measured at the recording system at the output • sØqÙ is the contribution of the input to the measured PCG at the output • and h®Øq, {Ù is the time-varying impulse response of the system (transfer function). The mathematical formulation of this model is based on the time-frequency representation of digital signals and systems that was introduced in early 80s (Portnoff, 1980). In this model, Durand chose to calculate the time-varying Fourier transform _¹Øl, fÙ of the PCG uØqÙ on a single cardiac cycle basis, represented as follow 3:>4 _¹Øl, fÙ Î D tØkÙuØl Ê k ËU0<Ùe„³¿±µª„~µ…}  70 Acta Wasaensia Where r is the index of the cardiac cycle, (Ø Ù is a real window sequence of duration centred on the time index !, ! emphasises a portion of the PCG at a specific instant of the cardiac cycle. The time-varying Fourier transform ›Ø!, Ù of the PCG output signal vØqÙ on a cardiac cycle basis is calculated as follow 3:?4  ¹`Øl, fÙ Î D tØkÙvØl Ê k ËU0<Ùe„³¿±µª„~µ…}  Thus, the cross spectral density between PCGs is estimated by 3:@4 ¡¢Ø!, Ù Î ~‹D ›IØ!, Ù›Ø!, ً›…~  And  is the number of cardiac cycles averaged, and \ is the energy of the windowing function. Now, the auto spectral density function becomes 3:A4 ¡¡Ø!, Ù Î ~‹D 1›Ø!, Ù1‹›…~  3;84 ¢¢Ø!, Ù Î ~‹D 1›Ø!, Ù1‹›…~  3;94  Î D (Ø Ùˆ„~—…}  Referring to Figure 19, the instrumental noise of the recording system at the input can be neglected, because it is very small compared to the recorded signal. Hence, the system transfer function can be estimated using 3;:4 Ø!, Ù Î ¡¢Ø!, Ù Î ¡¢Ø!, Ù0¡¡Ø!, Ù Where Ø!, Ù is the transfer function that represents the heart-thorax acoustic system that describes the characteristics of the sound transmission in terms of the amplitude and phase (Durand et al., 1985) . The system time-varying coherence function is interpreted as the frequency distribution of the squared correlation function between the input and output signals of the system, it can be computed by 3;;4 6¡¢ Ø!, Ù Î 1ŒÇÈؘ,“Ù1 Á ŒÇÇؘ,“ÙŒÈÈؘ,“Ù Where 6¡¢ Ø!, Ù Ð ; , the coherence function value indicates the validity of the transfer function for different frequency spectrums; • if 6¡¢ Ø!, Ù Î ; the system is noise-free, and the estimate of the transfer function is exact Acta Wasaensia 71 • and if 6¡¢ Ø!, Ù Î : the system output is pure noise and the system transfer function is practically meaningless. For example, this could happen if the sensor is faulty. For any value between 0 and 1, the system might suffer from large noise, or its linearity might be questionable, or then the input signal is not only what’s indicated in Figure 17. If the instrumental noise is not negligible, it can be reflected in the transfer function as follow 3;<4 ž¢Ø!, Ù Î ¡¢Ø!, ÙØ ŒÇÇؘ,“ÙŒÇÇؘ,“ÙƒŒÄÄؘ,“ÙÙ Where ——Ø!, Ù is the auto spectral density function of the instrumental noise m(t) in Figure 19. This instrumental noise could cause negative bias that underestimates the real gain of the transfer function, which can be estimated and corrected using corrective function corresponding to the inverse ratio of Equation (34). 3;=4 Ø!, Ù Î ŒÇÇؘ,“ÙƒŒÄÄؘ,“ÙŒÇÇØÅ,ÃÙ Î ŒÆÆؘ,“Ù ŒÆÆؘ,“Ù„ŒÄÄؘ,“Ù Where žžØ!, Ù is the auto spectral density function of the PCG monitored by the input recording system. At and the coherence function between the instrumental noise and the output PCG is 3;>4 6žŸ Ø!, Ù Î 6¡¢ Ø!, ÙØ ŒÇÇؘ,“ÙŒÇÇؘ,“ÙƒŒÄÄؘ,“ÙÙ This means that 6žŸ Ø!, Ù Ï 6¡¢ Ø!, Ù. Durand and his team (Durand et al., 1985) and many others after their work have adopted this approach with success. It is worth to mention that a large portion of the work was practical; Durand and his team recorded the heart sound surgically from inside the heart and compared that to the recording they got from the apex. 6.2.1 Discussion (Realisation of Durand’s Research) This stochastic approach relies on the measurement to estimate the transfer function. It is a non-parametric approach as has been shown in the previous section. However, in this study, we have not performed similar measurements on living animals nor humans. This is due to the restrictions around such experiments; it requires ethical approval and medical doctor’s involvement, which was not seen necessary for this pure engineering research. Therefore, the analytical 72 Acta Wasaensia derived approach “analytical approach” was preferred to this. However, it is still relevant to realise Durand’s method from human PCG, for future reference when real data can be obtained. Using the same heart sound that was used in the discussion of the analytical approach, let us find the variables required for the stochastic approach. • Fourier Transform of the audio signal must be found. • Hanning window function is used here, and it can be found using the function, with alpha set to 0.5 3;?4 (Ø!Ù Î :-> Ë Ø; Ë "$ Þ¦˜‰„~ßÙ: Ð ! Ð Ë ; • Equation 26 can be constructed now. • Using tØlÙ, the energy of the window function \ can be calculated using Equation 29. • Number of cardiac cycles (Y) in the heart sound, this was found by counting the peaks of the sound signal plotted in MATLAB; which was found to be 48 cycles in this case. The number of cycles is subjective and cannot be generalised. • And now the cross- and auto- spectral density functions can be constructed. The problem with applying Durand’s model as-is is that there is lack of representation of the distance between the heart and the measuring point (wrist in this case). Due to that, it is cumbersome to reflect the difference between the output measured at the chest and the wrist. Durand’s direct measurements have replaced the need for this representation. Therefore, Durand’s model must be modified to accommodate the attenuation experienced by the heart sound wave after travelling the distance to the wrist. This is what will be simulated in this discussion. Acta Wasaensia 73 Figure 20. Heart acoustic signal measured at the chest (input) The realisation of Durand’s Model could be achieved in the simplest form using Fast Fourier Transform and Power Spectral Density (PSD) calculations. The process starts by converting the heart acoustic signal in Figure 20 into frequency- domain using Fast Fourier Transform (FFT), to calculate PSD. The signal’s frequency is then scaled to find the corresponding frequency values of every point for the entire signal. The absolute result is then multiplied into the FFT of the signal element-by-element (dot product), this can be seen in Figure 21. The received signal is calculated using Inverse Fast Fourier Transform (IFFT), this gives an estimate of the received heart sound signal at the wrist using Durand’s Model, and can be seen in Figure 22. The signal seems to be distorted and have missed some information when compared to Figure 20, however, there is no direct relation between the signal characteristics and where it was measured (how far it had travelled from the heart). 74 Acta Wasaensia Figure 21. Realisation of Durand’s Model (1) Figure 22. Realisation of Durand’s Model (2) As discussed earlier, Durand’s Model does not account for the travelled distance from the heart to reach the measurement point of the acoustic signal. Therefore, there is a need for a hybrid model that combines the two approaches; analytical Acta Wasaensia 75 and Stochastic. To reflect the attenuation caused by the travelled journey by the PCG signal, this will be called the Hybrid Model. Below are two attempts to design the analytical model as well as the hybrid model. A. Method 1 From the discussion about the analytical approach, the heart sound wave experiences attenuation x, which is a function of frequency and acoustic properties of the medium, x Î af­. Let us assume that b Î ; and ignore the frequency effect for simplicity, which leads to a uniform attenuation that is equal to a. To even simplify the attenuation further, let us use the average value of a from Table 4. Consequently, a Î @-:; Í ;:„ . The model used to demonstrate this method is shown in Figure 23. Figure 23. Simulation model of the stochastic approach – simplified method B. Method 2 This method is a hybrid between the analytical and the stochastic approach. The output signal is the sum of outputs that were found using the analytical approach. This method factors in the frequency effect on the attenuation. Figure 24 shows the model used to demonstrate this method. The calculated output following the analytical approach is shown in Table 5, the average value is calculated here • Average value of the acoustic property, a Î @-:; Í ;:„ • Frequency bands should remain the same; 10Hz, 140 Hz, 10 Hz, 400 Hz. • Averaged output is the sum of attenuated input signal in each frequency band divided by 4. 76 Acta Wasaensia Figure 24 Simulation model of the stochastic approach - hybrid method 6.3 Summary In this chapter, the analytical and stochastic models of the heart acoustic wave propagation were investigated. The focus point was the wrist, what does the heart acoustic wave propagation model look like between the heart and the wrist? And most importantly, which model is more realistic? The realism was measured by answering three questions: • Does the model account for the distance between the focus points? • Does the model account for the attenuation imposed by the organs on the propagation path? • Does the model preserve the human body characteristics? The analytical approach does account for the distance between the heart and the measuring point and it does account for the attenuation imposed by other organs on the propagation path, but it does not reflect real human body characteristics; it assumes uniform pressure throughout the propagation path. The stochastic approach (Durand’s model) relies on the measurements of living animals to get the system transfer function, which nullify the need to explicitly represent the distance between the focus points and the attenuation caused by other organs on the path. Since this approach is using the measurements to find the transfer function rather than studying the propagation path. However, this model can easily be modified to account for the missing measures; distance and another organs’ attenuation. As opposed to the analytical approach where there is Acta Wasaensia 77 no room for modification without over complicating the model. That is how the hybrid model was created. The Hybrid Model gives realistic realization of the heart acoustic wave propagation from the heart to the wrist accounting for human body characteristics; distance, effect of other organs, and frequency differences in the acoustic signal itself. However, the result of the realisation reveals that the Hybrid Model is not realistic enough to account for the distance travelled by the PCG signal nor is it descriptive enough for the impact of this distance. 78 Acta Wasaensia 7 THE EXPERIMENT The experiment here refers to modelling the heart-wrist acoustic wave propagation system using SIMULINK MATLAB, see Figure 25 for the model. The transfer function [ ~}-}}}~ ƒ}-}}Áƒ}-~~ƒ~] was found using the analytical approach, where x in Equation 22 is a function in frequency, [W· º Î W²¶e„½Ø±Ù¯]. This ensures the applicability of this model to the entire frequency range. By calculating the Laplace Transform of this equation, the model transfer function was found. Since the values in table 5 are general and applicable to adults, this model is assumed objective. The rest of the model shown in Figure 25 shows, a healthy heart acoustic signal is used as a reference, a random noise is added to account for measurement noise, the total signal is used as an input to the model. The resultant signal at output of the model is a distorted version of the input. This is when the actual experiment starts, and the discussion gets interesting. See Figure 26 and Figure 27 for the original and received signals, respectively. Figure 25. Heart-wrist acoustic propagation model (analytical approach) The reason behind choosing the analytical approach for carrying out the analysis is the lack of real measurements. Since the research is done by an engineer and despite having studied the cardiovascular system there is still need for medical supervision to acquire real human or animal measurements. In addition to the ethical approval requirements, the experiment had to be simulated and the analytical approach was most suitable for that. It, also, paints a clear picture of the propagation path effect on the heart acoustic signal. Figure 26 and Figure 27 show the healthy heart signal that is used as reference throughout this experiment (original and as received at the wrist). The same healthy signal referenced in chapter 6 (44100 sample/second) was down sampled by 100 sample/second to reduce the computation load and speed up the processing time. This sampling rate was used throughout the experiment. It should be noted that down sampling acts as a filter that removes the high frequencies, Acta Wasaensia 79 which works well in the case of low frequencies like the one at hand. The new resultant signal has slower sample rate than the original signal. This could affect the accuracy of the experiment negatively, since the down sampling of the signal could lose some of the disease’s indications that are held in high frequencies. All information and indications held beyond 200 Hz are lost, because the down sampled signal shows only 200 frequency components. Not to mention that it would limit the number of levels that can be in the Filter Bank, since some sub bands would show only noise. However, the purpose of this research is to diagnose the phonocardiogram signal (PCG) with machine learning with limited computation load and processing power (mobile devices), down sampling serves this purpose. At this stage, the main point is to prove that the received heart acoustic signal at the wrist is viable and informative. From Figure 27, the received signal does hold information and despite the distortion caused by the noise and the system, the signal is still comparable to the original signal measured at the chest. To prove this, the task is to determine the success of the diagnosis (i.e. classification) of the heart condition using the signal received at the wrist; if the result is satisfactory (>75% success rate and <25% probability of error), the signal does not need any restoration and can be used as is without any improvement. This is good news, because it reduces the system complexity and consequently reduce the computation load and required time and power for this system.  Figure 26. Original Signal: Healthy Heart Acoustic Signal Am plitude Time in seconds 80 Acta Wasaensia  Figure 27. Received signal at the wrist (heart-wrist propagation mode) The diagnostic system is based on multivariate classification from parametric estimation as a Machine Learning technique. This is used to classify the acoustic signal as received at the wrist and find the most probable diagnosis of the heart condition. The classification is based on the discriminant function of the unclassified signal. The most probable diagnosis is found by maximising the discriminant function and in other words, minimising the Mahalanobis distance (Alpaydin, 2016). The experiment starts by defining five hypotheses, each representing a heart disease, these hypotheses are used as reference classes for the diagnostic system. This is done by introducing a unique transfer function that alters the healthy heart acoustic signal shown in Figure 26 to form a hypothetical disease, the resultant is then sent through the heart-wrist model (acoustic propagation transfer function) to be received as hypothesis x. And here is what makes these hypotheses valid - In simple words, an unhealthy heart is a heart that produces faulty PCG. When the cardiovascular system has a problem, the acoustic signal of the heart should reflect that in time/frequency space. This appears as corruption in the heart sound. Such change in the acoustic signals might not be audible or sensed by human ears (not even experienced cardiologist). However, it could be sensed and potentially classified with sensitive sensors and proper machine learning algorithms. A heart disease could be modelled as a linear corruption of signal of the healthy sound. This corruption has been performed by applying the healthy heart sound signal to Am plitude Time in seconds Acta Wasaensia 81 different linear filters with different characteristics. There is no medical basis for selecting these filters during the simulation. However, this could be another research for modelling different heart problems with linear/nonlinear transfer functions. The diagnostic system’s job is to identify this noise and consequently conclude the most probable heart condition. Table 6 shows the hypotheses that were created for the diagnostics task.  82 Acta Wasaensia Table 6. Classes for The Diagnostic System ,#"'&&   %!&%(!'"!3!'"!4 ,#"'&&8384 "!3',&!4 ,#"'&&9394 ;>Z Ê ;::Z Ê ;-; ,#"'&&:3:4 ;Z Ê ;::: ,#"'&&;3;4 ;Z Ê :-:=Z Ê -BB ,#"'&&<3<4 ;Z Ê ; ,#"'&&=3=4 ;<Z Ê ; Figure 28 shows what these hypotheses look like to highlight how they differ from each other. These are the signals with added system noise only before going through the heart-wrist model. In this figure the signals are shown in time domain, where x axis represents the time in seconds and the y axis shows the amplitude. Acta Wasaensia 83 Figure 28 Heart Hypotheses: hypothetical heart conditions 7.1 Classification of Heart Conditions There are numerous classification algorithms in the literature. However, there is an important trade-off between complexity and accuracy. One application of this proposal is to be applied in limited computation power device (mobile devices). Hence, simple classification algorithms are used as a start. 7.1.1 Simplified Approach (Using two features) In this approach, each hypothesis is characterised with two features; the mean and covariance. Every new case would be characterised with the same features and the classification is based on minimising the result of the following equation 3;@4 g² Î MÔz Ë z²Ô Ê N1 Ë ²1 Where A and B are weights that are chosen to maximise the indication of the mean difference and/or the covariance difference. During the experiment, it was observed that the covariance difference is more significant. For that, A and B were selected as 0.5 and 1, respectively. And the equation is now 84 Acta Wasaensia 3;A4 g² Î :->Ôz Ë z²Ô Ê 1 Ë ²1 This equation represents the distance between the unknown condition and every known hypothesis. When minimised, the result is the closest and most probable condition. To assure practicality and most importantly discuss accuracy and probability of error in relation to signal to noise ratio; this approach was deployed in two different scenarios. Scenario 1. Quite environment, where the heart acoustic wave (PCG) is recorded indoors or at least inside an ambulance with closed doors. In the experiment, this is represented with stationary noise (z:,;), also known as Gaussian Noise. Although this scenario is applicable, it is rarely achievable in accidents and more so in catastrophes, which are the targeted scenarios with this research. Scenario 2. Chaotic/ noisy environment, where the heart acoustic wave (PCG) is recorded outdoors amid the accident where moving vehicles around, for example. In this experiment, this is represented with nonstationary noise (z :, ;). The classification accuracy is discussed at the end of every scenario. In both cases, 300 cases were generated with equal distribution (50 cases/hypothesis) between the Hypotheses. Although this is unrealistic; because in real life, there are some heart conditions there are more probable than others. If such scenario is explored, the experiment would be using Bayesian probability, where the impact of the most probable hypothesis (weight) would be factored in addition to the minimising the classification function. a) Scenario 1: Stationary Noise A random Gaussian noise is used to generate random variations of the heart signal, these signals represent unknown conditions. They are based on the hypotheses discussed at the beginning of this chapter. Total of 300 cases were generated and classified, all cases were classified successfully. This classification has percent error of 0%, with 100% reliability. If this approach with scenario 1 is considered system 1, the confidence interval can be calculated as follow 3<84 OS Î u Ì w I C¶ Where; Acta Wasaensia 85 • u is the mean of the classification function’s results • w is predefined value (Sullivan L.) based on the confidence level that this result can be reproduced • p is the standard deviation of the results of the classification function, and n is the number of samples. In this scenario, the confidence level of reproducing the same results is 95% since the result depend on the noise characteristics. And with that, the Confidence Interval is <-AA>?e Ë :> Ì -;B?Ae Ë :?. If the confidence level increase to 99%, the confidence interval would be <-AA>?e Ë :> Ì =-<:;>e Ë :?, which is clearly wider. With this, it can be concluded that system 1 is applicable with high accuracy due to the wide confidence interval. However, the wide confidence interval could also be caused by the relatively small sample size, which could also explain overfitting that has led to 100% successful classification. Nevertheless, the application is limited to quite environment. This system can be used to reach quick diagnosis in an ambulance on the way to the ER, however, that is not the scope of this research and for that this system will not be considered further. b) Scenario 2: Nonstationary Noise A noise with defined mean and covariance is used to generate random variations of the heart signal that represent unknown conditions based on the hypotheses discussed at the beginning of this chapter. The noise is defined as follow 3<94 zØVÙ Î ·»°¹ I oaldØ;Ù 3<:4  Î ·»°¹ I oaldØ;Ù Ê  3<;4 V Î  I oaldØl, ;Ù Ê z Where N is the random nonstationary noise, n is the number of samples, and z,are the mean and covariance, respectively. The noise power ( ·»°¹) value (:-:::;BAÙ is used to keep value of N reasonably low, in order to meet an SNR of 20 dB, C is a constant that was set to the same value of ·»°¹. Here and after SNR = 20 dB is used as a threshold; least accepted SNR. Total of 300 cases were generated and classified using this approach, and to maintain comparable results with the advanced approach. 86 Acta Wasaensia Only 300 cases were classified. Out of which, 217 cases were successfully classified, while 83 cases have failed; a success percentage of 69.6%. This approach with scenario 2, is considered system 2. Looking at the classification results, the system functions with this success percentage as long as the SNR is kept higher than or equal to 20 dB. However, this percentage drops to 18.6% as soon as the SNR is below 20 dB, until it simply stops distinguishing between the cases. In an attempt to show case an extreme scenario, SNR was set to -50 dB, where the noise power is kept the same, but the constant value was set to 1, in other words, the covariance was defined as: 3<<4  Î ·»°¹ I oaldØ;Ù Ê ; Consequently, this resulted in large confusion where all cases were classified as hypothesis H0. Only 17 cases were classified successfully, and the rest was false. In Table 7, the classification function shows how small is the distance between the unknown case and the Hypotheses. These small differences explain the high level of confusion and why this system in this scenario is simply useless when SNR is below 20 dB. 100% of the error occurs in classifying any case that is not related to hypothesis H0. Although it is an extreme case, but in this scenario, the result is particularly dangerous since the system perceives all conditions to be healthy.  Acta Wasaensia 87 Table 7. False classifications using simplified approach (scenario 2) & &&'"!(!'"!3g²4 &&'"! '"! "%%'&& &9 @0;8289 8 %( 8 @0;@289 @0;@289 @0;?289 @0;@289 @0;>289 &: @0:=289 8 & 9 @0;;289 @0;;289 @0;:289 @0;;289 @0;9289 &; @0:=289 8 & : @0;:289 @0;;289 @0;:289 @0;:289 @0;8289 &< @0:=289 8 & ; @0;;289 @0;;289 @0;:289 @0;;289 @0;9289 &= @0:=289 8 & < @0;:289 @0;;289 @0;:289 @0;:289 @0;8289 &> @0:?289 8 & = @0;<289 @0;=289 @0;<289 @0;<289 @0;:289 88 Acta Wasaensia Using the same Equation (40), the Confidence Interval for this scenario with 95% confidence is ;-?<Ì :-;:>A . If the confidence level increase to 99%, the confidence interval would be ;-?<Ì :-;B:, which is slightly wider than that of 95% confidence interval but still too narrow for an acceptable success. Figure 29 shows a comparison between the success rate granted by an SNR that is higher than or equal to 20 dB and that granted by an SNR that is lower than 20 dB. Figure 29. Comparison of the success rate around the SNR threshold. From this, it can be concluded that the simplified approach is reliable and theoretically applicable to closed and quite environment with zero probability of error. Nonetheless, classifying heart condition in an open chaotic environment with two features only is not reliable method. Evidently, the confidence level gets narrower when the noise is nonstationary in comparison to the stationary noise scenario. Moreover, the success percentage drops by 30%. In a pursuit of finding a higher accuracy, better success rate and a more reliable classification system, the advanced approach is proposed. In an attempt to introduce more features that identify and differentiate between the Hypotheses. A similar concept was used to distinguish between normal and abnormal heart sound using spectrogram analysis (Dey et al., 2012). However, the heart sound was measured at the chest with high quality.          (&& (%          (&& (% Acta Wasaensia 89 7.1.2 Advanced Approach (Using Wavelet Transform) As seen in Approach 1, diagnosing the acoustic signal using two features only in an open and chaotic environment is limited to low noise powers that achieve SNR close to 20 dB, the same system proves to be useless when the SNR reaches small values that are far below 20 dB. This approach proposes to use more features to define the cases; to improve the process accuracy of distinguishing between the hypotheses. This is achieved by decomposing the signal into a number of bands using Wavelet Transforms (specifically; Filter Banks), every band is then described by its mean and covariance, so that the entire case is defined by a number of features equal to 2x the number of bands. As the focus of this research is diagnosing heart condition in an open and chaotic environment, the noise is kept nonstationary. However, the noise power and SNR are discussed around the threshold of 20 dB to maintain comparable results to the previous approach. The stationary noise scenario was ignored here since the result was satisfactory using Approach 1. Same as in the previous approach, 300 cases were generated with nonstationary noise (50 cases for every hypothesis). For each hypothesis, the cases were divided 50% training and 50% testing sets. Then the following steps were followed (1)Decomposing the signal The signal is decomposed using Wavelet Transform and specifically Filter Banks with different number of levels for every cycle of this experiment. The training and test sets were decomposed into a number of bands equivalent to 2x Filter Levels. For each band, the mean and covariance were calculated, so that each case is described by a number of features equivalent to 2x the number of bands. These features are concatenated in a 1 x the number of features matrix named “descriptive matrix”. The number of levels was selected using trial and error as the method of optimisation. Starting the 1 level (2 bands), the experiment trials and compare results from 2 levels (4 bands), 3 levels (8 bands), 4 levels (16 bands), 5 levels (32 bands), and finally 6 levels (64 bands). Increasing the levels beyond 6 did not add any value to the classification nor did it improve the classification result, for that this experiment was stopped at 6 levels. (2)Training the System Training the system in this approach is about using the training set to construct the discriminant function. By calculating the mean of the descriptive matrix to get 90 Acta Wasaensia a 1x128 vector and calculate the covariance of the descriptive matrix, which is a square matrix of 128x128. These values are used to construct the Discriminant Function (DF) that is given by Equation 45. This equation represents quadratic discriminant function used for multivariant classification (Alpaydin, 2016). 3<=4 P Î Ë:-> "Ø%ØpÙÙ Ë :->Øu Ë kÙ I ilseopeØpÙ I Øu Ë kÙ Where s is the covariance matrix, m is the mean vector, and x is the training case. It certainly helped to add a confirmation step here that tests the DF using the training set, by simply calculating the DF for the training set and maximising the result, in order to confirm the validity of the training. This is a simple test; because if the training is valid the classification must be 100% correct. During the confirmation step, it became evident that the determinant of the covariance matrix is zero in many cases, which made the first component of the DF function (-infinity). For that, the discriminant function equation was rewritten as follow 3<>4 P Î Ë:->Øu Ë kÙ I ilseopeØpÙ I Øu Ë kÙ This formula was used when the filter levels reached 4 (16 bands, 32 features). (3)Testing the System To test the system, the discriminant function calculated in step (2) is used to classify the test set. The classification was 97.33% successful using 8 features (2 levels), with only 4 false classifications. With 128 features (6 levels), the classification was 79.33% successful. Total of 31 cases out of the 150 cases were falsely classified, while the rest 119 were correctly classified, see Figure 30 for a visual representation of the classification success across hypotheses. Acta Wasaensia 91 Figure 30. Results of Advanced Approach – 8 features (using Filter Banks) After repeating the above steps for every all Filter Levels from 1 to 6, it was concluded that the best possible configuration for this scenario using this approach is to use a Filter Bank with 2 levels. This decomposes the signal into total of 4 bands and allows the signal to be described by 8 features. This is an interesting result because it defies the proposition behind this approach, that is more sub-bands will improve the success rate of the classification. Figure 31 shows that this relation is non-existent when the SNR is higher than 50 dB. Despite defying the proposition of the approach, this result is eye-opening. Decomposing signals is considered insightful, because it provides more descriptive details about the original signal. For example, a signal that is decomposed to 4 bands is more descriptive than one decomposed to 2 bands, since the number of features is twice as much with 4 bands. What makes this result eye-opening, is that it argues against that. It is known that when the decomposition level is too large, the analysis goes too deep and might lead to analysing noise instead of the signal of interest. However, this was not expected with small decomposition levels as the ones used here. Looking at Figure 32, it is evident that when the SNR value is 20 dB, the success rate of the classification is inversely related to the filter level, and consequently, the number of features. 92 Acta Wasaensia Figure 31. Classification success vs. the number of levels (High SNR) Figure 32. Classification success vs. the number of levels (Low SNR) Referring to Figure 28, there is a clear correlation between the hypotheses that could explain this result. Evidently, the correlation coefficients between these hypotheses proves exactly that, especially between Hypothesis 1 and 2, Hypothesis 1 and 3, and also between Hypothesis 2 and 3, where the correlation coefficients are close to 1. Acta Wasaensia 93 From this discussion, it can be concluded that a larger number of smaller bands causes the classification to be more error prone. Such bands result from decomposing the signal using filter banks with large number of levels. This could have been caused by the down sampling, since it removed information held in high frequencies and left some bands with just noise. However, the trade-off was worthy, because the down sampling that was performed at the beginning of the experiment reduced the required computational load and processing power. Not to mention that it served the purpose of this research; limited energy and processing power (for potential use of mobile devices). Similar result had been noted in image texture recognition that filter banks with smaller number of levels performed better than larger ones (Randen, 1997). Despite that, this finding is considered a paradigm shift because it goes against expected performance. Let us use the analogy of localising a red ball inside a round field, Figure 33 demonstrates this visually, if the task is to find the location of _ØiÙ visualised as the red dot. _ØiÙ is in one of the circle slices or not in the circle at all.   Figure 33. Visual demonstration of the impact of the number of hypotheses. On the first circle, there are six slices. Therefore, the possible location of _ØiÙ is reduced to 1/6th of the circle only. On the middle circle, there are ten slices. Therefore, the possible location of _ØiÙ is reduced to 1/10th of the circle. The accuracy is higher here because the location is further reduced by ~ 0.07. And on the third circle, there are twelve slices. Therefore, the possible location of _ØiÙ is reduced to 1/12th of the circle. The accuracy is even higher here because the location is further reduced by 0.08 from first circle to the right and by 0.02 from the middle circle. The conclusion here is that the number of slices determines the level of accuracy when it comes to the location of _ØiÙ, because it reduces the pool of possible locations. In the same concept, the number of sub bands of the signal should determine the accuracy of the diagnostic system. However, it is important 94 Acta Wasaensia to maintain low noise to avoid analysing noise only and causing more errors in the classification. It is also important to maintain clear distinction between the classifications, to ensure low to no correlation between the sub bands. Based on this experiment, it can be stated that the number of levels used in the decomposing filter bank is inversely related to the accuracy of the classification. 7.1.3 Using Neural Network for Classification In this approach, a neural pattern recognition network from the Neural Network toolbox in MATLAB was used. That is a two-layer feedforward network, with a sigmoid transfer function in the hidden layer, and a softmax transfer function in the output layer. The simple network had one hidden layer with 10 neurons, the network diagram is shown in Figure 34. Figure 34. Neural Pattern Recognition Network Diagram The data set had 150 samples with 16 elements for the input. This input is the same decomposed signal using a 4-level filter bank (same as the previous approach). Similarly, it had 150 samples for the target with 6 elements. each element represented a class (Hypo 0, Hypo 1, Hypo 2, Hypo 3, Hypo 4, Hypo 5), and each target vector showed ‘1’ for the correct class and ‘0’ elsewhere. The data set was divided into 70% for training, 15% for validation and 15% for testing. Consequently, the network output has 6 elements. For training, the network used Scaled Conjugate Gradient training algorithm. The training was successful and so was the testing and validation. The performance was satisfactory as can be seen from Figure 35 Acta Wasaensia 95 Figure 35. Neural Pattern Recognition performance plot As can be seen in the confusion plot in Figure 38, 25 cases of every class were recognised, which represents all the dataset that was used for training, testing and validation, with zero% errors and zero confusion. That is when the training was concluded and there was no need for further testing. Following this, the network was used to classify a new dataset of 150 samples. The classification was 100% successful, the network was able to recognise the features of every sample with high accuracy; where the output showed 0.99999* for the correct class and <0.00001* for all other classes. The difference between the values of the output vector is very large, which proves low confusion and high performance, as can be seen in Figure 36. 96 Acta Wasaensia Figure 36. The confusion plot of the Pattern Recognition Neural training The accuracy of the classification must be coupled with trust level to understand the reliability of this classification. The Neural Network trust level can be measured using the formula in Equation 47 300=828? <0;?29? 80AAAAA@@ 90??28A =0<>28? 9 90?=29< 80AAAA@?:> <0A:28@ 909?28? 90:=28= :0:@28@ 9 80AAAAAA?; ;0;A29@ ;0<:2:> :08828? <0:928@ :029A :0@928> :0:<298 80AAAAA?9A 9 <0;=29> :0;A28? =0:>29> <0><299 80AAAAAA?> A0:8299 9 ;0=?29> :08828? >09829> :0??299 80AAAAAA@ >0AA299 9 A0@:29= 80AAAAA?8: 90=?28? >08?28@ :0?=28> 90;928@ 9 >0:?2:9 :0><28> 80AAAAA?;> :0A:29@ :0<:29< 908<29= 9 @0@=29= ?0?;28? <0@@29? 80AAAAA@:? 90?A28A A0=?28? 9 80AAAAAA?; ;0;>29@ ;0;A2:> 90AA28? <09@28@ :0<>28@ 9 Therefore, the trust level on the classification provided by this Neural Network is ; Ë :-;?@ Î :-A, that is 83.3%. This is quite satisfactory; however, it could be improved using more Hypotheses. The below discussion explains further. In this experiment, there are six classes that have equal presence in the studied sample. Therefore, the probability that a given case is either of these classes is 0.167 and since it is equal, it has been ignored. But let us consider a sample where the classes do not have equal presence, in other words, imbalanced sample. For example; assuming the following imbalanced sample was used for training W R}ÔZ× Î :-; W R~ÔZ× Î :-> W RÔZ× Î :-; W R€ÔZ× Î :-< 98 Acta Wasaensia W RÔZ× Î : W RÔZ× Î :-; The training process would have been more complicated, since it could suffer from local optima, where it keeps classifying cases to the class with the highest presence (highest probability). Not to mention that the accuracy of the classification would become a secondary measure of the network performance, since the cases that have been classified as Class (e.g. R~) are 50% correct. In fact, such sample would be more realistic than the balanced sample that was used in the experiment, since heart diseases, and diseases in general, have different probabilities. These probabilities are impacted by the gender of the patient, their age group, their life style, and for some diseases their origin and country of residence. To improve the performance of the classification of imbalanced sample such problem, a combination of the following techniques can be used; - Collect more data from the underrepresented classes to reach a balanced sample. This will simplify the problem but might not always be possible. And in other times, it might not be desired, in case of limited resources. - Simply duplicate training samples to match up the number of samples per class in the training dataset. However, this will increase the training time without adding any new information. This solution can have better impact if the copies are augmented and modified to add new information, for instance a distorted version of the same class. - Remove training sample from the overrepresented classes to match up the number of samples per class in the training dataset. This will reduce the training time but will worsen the network performance by removing useful information for the training. - Train for sensitivity and specificity. Sensitivity is the probability of classifying a case as R~, while it is in fact R~, which measures the accuracy of detecting a Hypothesis. While Specificity is the probability of classifying a case as R~, while it is in fact R, which measures the accuracy of detecting the absence of a Hypothesis. These two attributes describe the accuracy of the classification as a whole. For optimal performance, they should both be equally high (~1) due to their equal importance. However, this might be different in some application. For example, skin cancer detection system is better off with high sensitivity and low specificity. The two attributes can be traded-off depending on the application. Acta Wasaensia 99 - For every class, specify a weight that is inversely proportional to the probability of the class. For example, R~ should have a weight of 0.5, while R€ should have a weight of 0.8. These weights represent the contribution of the class to the loss function. A practical implementation of this was presented in 2015 and it is called “Keras Implementation” (Chollet et el., 2015). Another solution would be to introduce a new hypothesis “none of the known hypotheses”. This gives room to define an acceptable probability of error, where any classification that is lower than a certain minima would lead to “none of the known hypotheses”. This method increases the credibility and integrity of the system compared to the previous one, however, the fewer the hypotheses the more probable the neutral diagnosis is concluded. Moreover, the higher the acceptable probability, the more accurate the system is. Using neural networks opens the door for broadening the pool of classes with minimal work, by training the network on new classes (hypotheses). A similar concept was presented to detect abnormal heartbeat using recurrent neural networks in January 2018 (Latif et al., 2018). However, the heartbeat was measured at the chest assuming high quality. In conclusion, the number of the learnt hypotheses has a huge impact on the accuracy level of the diagnostic system. The more knowledgeable the diagnostic system, the more accurate the diagnosis will be, which is achieved by training ANN on larger number of hypotheses. 7.2 Summary For a chaotic environment, like the one represented with nonstationary noise, the advanced approach provided satisfactory results with 97.33% success rate. This is using the discriminant function for classifying. The signal is decomposed using 2- level filter bank into 4 bands that are each described with their mean and covariance; the signal is represented with 8 features. This success rate is elevated to 100% when pattern recognition neural network is used for classifying the 16- feature signal after decomposition into 8 subbands using 4-level filter bank. Moreover, the Neural Network-based classification had a trust level equal to 83.3% which correlates with the number of hypotheses included in the classification. Figure 37 shows two of the classification methods that were used in this research, these had the best results. 100 Acta Wasaensia Figure 37. PCG-based Diagnosis Using Machine Learning (three methods) Acta Wasaensia 101 8 CONCLUSIONS This research intended to study the feasibility of diagnosing the heart condition using low quality acoustic signal. In doing so, it would discuss multiple classification techniques that can handle distorted biomedical signals and analyse the reliability of such diagnosis. Lowering the quality was achieved by assuming the input to such system was taken from the wrist as opposed to the common spots on the chest. This input was simulated using a heart-wrist acoustic propagation model that was designed after cardiovascular models available in literature. On the journey towards this, eHealth history was studied, and more focus was put on mHealth tele-cardiology solutions and why there is still room for more research in this area. Especially when lowering the demands on the quality of the data and the computation load can mean saving more lives in chaotic accidents. This research used one healthy heart sound that was measured at the chest to explore the feasibility of the proposition. The signal was put through the heart- wrist acoustic propagation system to get the input to the experiment. In the experiment, this healthy signal was called hypothesis H0. Which was then used to create five more hypotheses using different transfer functions. These six hypotheses represented the reference classes for the classification task. This experiment aimed to diagnose the heart condition using the acoustic signal of the heart as measured at the wrist. For that, three machine learning approaches were attempted; simple, advanced, and using neural networks. The simple approach used two features to describe the heart condition acoustic signal in question. The approach proved to be successful in closed environments with stationary noise and the performance degraded by 30% in chaotic environment with nonstationary noise. Since the focus of this research is chaotic environments, this approach was rejected. The advanced approach studied the relation between the number of features and the accuracy of the classification. To do so, the heart acoustic signal was decomposed into sub-bands and each band was described with two features. The classification result was 27% better than that of the simple approach. However, the result introduced a new factor, that is the size of the bands being described. It was concluded that the smaller the bands, the stronger the correlation between the cases and, consequently, the less accurate the classification. This finding in itself is a paradigm shift, since signal classification is commonly thought to perform better with larger filter banks (larger number of levels and consequently 102 Acta Wasaensia subbands). However, this was not the case in this research due to multiple reasons that were discussed in the experiment results. The pattern recognition neural network was attempted to simplify the computation load required by the advanced approach and most importantly, automate the classification task. The result was more than satisfactory, with 100% accuracy. And in comparison, to the discriminant function approach, the neural network was lighter. Once the training phase was complete, the pattern recognition neural required less computing time than that of the advanced approach, and consequently less computing power. Considering the complexity of the advanced approach, this does not come as a surprise. The proposition of this research seems to check out successfully; the biomedical signal is still useful and can lead to a successful diagnosis with a signal-to-noise ratio as low as 19dB. This opens the door for performing diagnostic tasks during catastrophes and at accident spots with small processing units, limited energy (mobile devices), with bad connections and in chaotic environments. It also leaves room for new wave of wearable devices that measure more prominent vital signs, like recording the heart sound, from the wrist (i.e. wrist bands) or even the ankle. There is a chance to lower the required signal-to-noise ratio further by introducing restoration techniques to improve the received signal (refer to Methods in chapter 1). This, unfortunately, comes at the expense of more computation load. Another alternative would be to introduce amplification step before the signal is processed, which could also lead to lowering the required signal-to-noise ratio even further when coupled with restoration technique to mitigate the amplified distortion and noise. There is no need to assume perfect environment, this research proves that it is possible to get reliable diagnosis in a chaotic environment. 8.1 Future Work It is obvious that there is still room for more research in this area. Not just to lower the required data quality, but also to build the proposed system and reduce the computation load further. One way to do this could be a wrist band that is equipped with a sensitive recording microphone, which could be considered an improved version of the wearable wrist sensor proposed in (Shi & Chiao, 2016) . The recorded signal is then sent to a mobile device that is equipped with a processing unit that can decompose the signal and run the analysis and diagnosis. One interesting idea would be to use data fusion to diagnose the cardiovascular system using more descriptive signals than just PCG. Referring to Wiggers Acta Wasaensia 103 Diagram (Figure 16), there are multiple signals that could be used to describe the cardio cycle and consequently reveal diseases or defects. This concept of data fusion and how it could lead to better heart diagnosis was used early on by a group of a cardiologist, a paediatrician, and an engineer (Ninova et al., 1978). They attempted to automate PCG screening for heart disease in children using ECG signal as a reference. In a similar fashion to Wiggers diagram, QRS and R-R interval were used here as references for the heart sounds; S1 and S2. The classification was based on the amplitude of each sound to detect abnormality in the heart. The classification has a sensitivity of 96.5% and specificity of 92.4%. The same concept was used again in 2011 (Phanphaisarn et al., 2011). When a group of researchers detect and diagnose heart abnormality using a combination of subsystems. One subsystem studied the relation between the Electrocardiogram (ECG) and the Phonocardiogram (PCG) to drive a decision based on the linear predictive coding coefficients of the impulse response that represents the ECG- PCG relationship. and a second subsystem that studied the phase space of the normal and abnormal heart using the likelihood ratio test value. What if we combine the system in Figure 39 along with a similar one that is ECG based to deduce a better diagnosis of the cardiovascular system and the health of the heart? There is a large number of researches in the area of ECG-based analysis and diagnosis, many of which are following the same concepts used in this research (refer to Chapter 4, 4.3). It has actually started long before PCG-based analysis, however, it suffered from the assumptions discussed in Chapter two. So, what if ECG-based diagnosis is built for mobile devices, similar to the PCG-based system built in this research? The two systems can then be combined; one is PCG-based like the one presented in Figure 37, and the other is ECG-based following the same limitations enforced by mobile devices; low computational load, processing power, and energy. The complete system would look like Figure 38. 104 Acta Wasaensia Figure 38. Data Fusion-based Diagnosis using Machine Learning Each subsystem would have its own indications to the heart condition and combined they would complete the picture of the cardiovascular system’s heart. ECG analysis may detect; cholesterol clogs in the heart’s blood supply, history of heart attacks, enlargement of one side of the heart, and abnormal rhythms (Blood Pressure Association, 2008). While PCG analysis may detect; abnormal vibrations when the valves open or close, the speed of the blood flow through the chambers, and tension in the tissues that connect the valves to the heart muscle. The likes of heart murmur, friction rub, and gallop are sound indications that can be detected when analysing PCG (Altshul, 2015). This data-fused system will be able detect all of these diseases and analyse each indication. Not only that, but it can be extended to use the relationship between ECG and PCG to validate the indication into heart arrhythmias; since both analyses can detect arrhythmias. The novelty of such system would lay in getting reliable diagnosis using mobile devices’ restrictions; limited computational load, processing power, and energy. However, knowing what we know now from this research, down sampling would not have be the best approach since mapping the two signals would suffer greatly from that. Which consequently means that the computational load would not be small! Remember that longer recording of ECG is more insightful than shorter ones, in fact 12-Lead ECG is the shortest informative version of ECG. Although there are many researches that attempted analysing shorter recordings of the ECG; 5 – 60 minutes (Chudzik et al., 2014) , but should be further tested in this setup. But if 12-Lead is the shortest informative ECG, deep learning will be the right approach for this system. Now, envision using Neural Engine the one that Apple built in 2017 for iPhone X (A11© by Apple) to build this system (refer to Chapter 3, 3.4). It will most probably require some modifications to handle large signals. If the number of features Acta Wasaensia 105 required to describe the ECG and PCG signal reach 1024, the Filter Bank should have 512 sub bands, that is 256 levels. Every block in Figure 40 would have 1024 inputs and outputs, except for the Filter Banks where they’ll have one input and 1024 outputs. This system would be very interesting to study further! Another valuable addition to this data-fusion is Heart Rate Variability (HRV), which can be measured from multiple short ECG recordings, as short as 40 seconds as shown in (Lim et al., 2011). It provides more insights into the wellness in general besides its indications to heart conditions (mostly arrhythmias). For instance, it is used nowadays in wearables to analyse stress levels and recovery patterns after high intensity activities (Firstbeat Technologies Ltd., 2014). There are different ways to analyse HRV, all have been reviewed in this article (Abdelmageed & Virrankoski, 2015). In this research we proved that 20dB SNR is good enough for a reliable diagnosis, but it is only the beginning. Data fusion and neural chip will open the door for more reliable diagnosis and a lot fewer assumptions. This is because the powerfulness of the neural chip will mitigate the need to limit computational load and processing time, which were restricting factors in this research. The future work in this discussion will aim to reduce the impact of these restrictions relying on new technology to provide the required computation and processing power encapsulated for mobile devices. With the advancement made in neural chips, the likes of the neural engine, it is very likely to realise mobile diagnostic systems based on data fusion in the near future. So, watch this space… 106 Acta Wasaensia References A.C. Guyton. (1992). Textbook of medical physiology (8th ed.). Philadelphia, USA: Saunders. Abdelmageed, S. (2012). Wireless health monitoring system: Vital transmitter Abdelmageed, S., & Virrankoski, R. (2015). Evaluation of HRV analysis methods to perform heart condition diagnostics in embedded systems. Almadar Journal for Communications, Information Technology, and Applications (AJCITA), 2(1) Retrieved from http://almadar-rd.ly/AJCITA/AJCITA_April_2015.pdf Abdul Awal, Sheikh Shanawaz Mostafa, & Mohiuddin Ahmad. (2011). Simplified mathematical model for generating ECG signal and fitting the model using nonlinear least square technique. Paper presented at the International Conference on Mechanical Engineering, Ali, S. Q., & Hossen, A. (2018). (2018). Different neural networks approaches for identification of obstructive sleep apnea. Pp. 281-284. https://doi.org/10.1109/ISCAIE.2018.8405485 Alpaydin, E. (2016). Introduction to machine learning (3rd ed. ed.). Cambridge: The MIT Press. Retrieved from http://lib.myilibrary.com?ID=890587 Altshul, S. (2015). What the sound of your heart says about your health  Retrieved from https://www.everydayhealth.com/news/what-sound-your-heart- says-about-your-health/ American Heart Association. (2017). Ejection fraction heart failure measurement. Retrieved from https://www.heart.org/en/health-topics/heart- failure/diagnosing-heart-failure/ejection-fraction-heart-failure-measurement Amri, M. F., Rizqyawan, M. I., & Turnip, A. (2016). ECG signal processing using offline-wavelet transform method based on ECG-IoT device. Pp. 1-6. https://doi.org/10.1109/ICITACEE.2016.7892404 Anliker, U., Ward, J. A., Lukowicz, P., Troster, G., Dolveck, F., Baer, M., Vuskovic, M. (2004). AMON: A wearable multiparameter medical monitoring and alert system. IEEE Transactions on Information Technology in Biomedicine, 8(4), 415- 427. https://doi.org/10.1109/TITB.2004.837888 Apple. (2017). ResearchKit and CareKit. Retrieved from http://www.apple.com/researchkit/ Apple. (2018). Apple watch series 4 - health. Retrieved from https://www.apple.com/lae/apple-watch-series-4/health/ Ari, S., Hembram, K., & Saha, G. (2010). Detection of cardiac abnormality from PCG signal using LMS based least square SVM classifier. Expert Systems with Applications, 37(12), 8019-8026. https://doi.org/10.1016/j.eswa.2010.05.088 Acta Wasaensia 107 Åstrand, P., & Rodahl, K. (1977). Textbook of work physiology: Physiological bases of exercise (2nd ed.). New York: New York: McGraw-Hill. Retrieved from http://trove.nla.gov.au/version/45216664 Azhari, H. (2010). Basics of biomedical ultrasound for engineers. Hoboken, N.J: Wiley-IEEE Press. B.Prad Kumar, S.Balambigai, Dr.R.Asokan, ECG De-Noising Based On Hybrid Technique, 2012 Blinowska, K. J., Czerminska, M. A., Chomicki, O. A., & Górowski, T. (1979). Computer analysis of frequency spectrum of a phonothyreogram. Medical & Biological Engineering & Computing, 17(2), 207-210. https://doi.org/10.1007/BF02440930 Blood Pressure Association. (2008). Electrocardiogram (ECG) and high blood pressure. Retrieved from http://www.bloodpressureuk.org/BloodPressureandyou/Medicaltests/ECG Bryn A. Lloyd. (2010). Attenuation constant. Retrieved from https://www.itis.ethz.ch/virtual-population/tissue- properties/database/acoustic-properties/attenuation-constant/ Büşra Kübra Karaca, Burcu Oltu, Tuğçe Kantar, Erkin Kiliç, Mehmet Feyzi Akşahin, & Aykut Erdamar. (2018). Classification of heart sound recordings with continuous wavelet transform based algorithm. Paper presented at the 2018 26th Signal Processing and Communications Applications Conference (SIU) C. Kalaiselvi. (2016). Diagnosing of heart diseases using average k-nearest neighbor algorithm of data mining. Paper presented at the 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom), Cables & Sensors.12-lead ECG placement guide with illustrations. Retrieved from https://www.cablesandsensors.eu/pages/12-lead-ecg-placement-guide-with- illustrations. Campos, M. (2017). Heart rate variability: A new way to track well-being. Retrieved from https://www.health.harvard.edu/blog/heart-rate-variability-new-way- track-well-2017112212789 Canada, H., & Canada, H. (2003). Canada's health infostructure. Retrieved from https://www.canada.ca/en/health-canada/services/health-care- system/ehealth/canada-health-infostructure/history.html Chad Greene. (2012). Sound pressure level calculator - file exchange - MATLAB central. Retrieved from http://se.mathworks.com/matlabcentral/fileexchange/35876-sound-pressure- level-calculator Chollet, F., & et el. (2015). Keras documentation 108 Acta Wasaensia Chu, Y., & Ganz, A. (2004). A mobile teletrauma system using 3G networks. IEEE Transactions on Information Technology in Biomedicine, 8(4), 456-462. https://doi.org/10.1109/TITB.2004.837893 Chudzik, M., Cygankiewicz, I., Klimczak, A., Lewek, J., Bartczak, K., & Wranicz, J. K. (2014). Short-term ECG recordings for heart rate assessment in patients with chronic atrial fibrillation. Archives of Medical Science : AMS, 10(4), 676-683. https://doi.org/10.5114/aoms.2014.44859 Colin Tidy. (2015). Heart auscultation. read about heart auscultation. Retrieved from https://patient.info/doctor/heart-auscultation Coskun, H., Deperlioglu, O., & Yigit, T. (2017). Implementation of wavelet transform extrasystole heart sound with convolution method for feature extraction. https://doi.org/10.1109/SIU.2017.7960300 D Martínez-Vargas, J., I Godino-Llorente, J., & Castellanos/Dominguez, G. (2012). Time–frequency based feature selection for discrimination of non-stationary biosignals. EURASIP Journal on Advances in Signal Processing, 2012(1), 1-18. https://doi.org/10.1186/1687-6180-2012-219 de Vos, J. P., & Blanckenberg, M. M. (2007). Automated pediatric cardiac auscultation. IEEE Transactions on Biomedical Engineering, 54(2), 244-252. https://doi.org/10.1109/TBME.2006.886660 Dey, N., Das, A., & Chaudhuri, S. S. (2012). Wavelet based normal and abnormal heart sound identification using spectrogram analysis. Retrieved from http://arxiv.org/abs/1209.1224 Ding, H., Sun, H., & Hou, K. (2011). Abnormal ECG signal detection based on compressed sampling in wearable ECG sensor. Pp. 1-5. https://doi.org/10.1109/WCSP.2011.6096677 Durand, L., Genest, J., & Guardo, R. (1985). Modeling of the transfer function of the heart-thorax acoustic system in dogs. IEEE Transactions on Biomedical Engineering, BME-32(8), 592-601. https://doi.org/10.1109/TBME.1985.325598 Durand, Langlois, Y., Lanthier, T., Chiarella, R., Coppens, P., Carioto, S., & Bertrand-Bradley, S. (1990). Spectral analysis and acoustic transmission of mitral and aortic valve closure sounds in dogs. part 1 modelling the heart/ thorax acoustic system. Journal of Politeness Research, 10(1), 1. https://doi.org/10.1515/pr-2014- 0001 EDUCBA. (2018). Neural networks vs deep learning - useful comparisons to learn. Retrieved from https://www.educba.com/neural-networks-vs-deep-learning/ Einav, S., & Elad, D. (2009). Physical and flow properties of blood El-Asir, B., Khadra, L., Al-Abbasi, A. H., & Mohammed, M. M. J. (1996). Time– frequency analysis of heart sounds. Paper presented at the Proceedings of Digital Processing Applications (TENCON '96), https://doi.org/10.1109/TENCON.1996.608401 Acta Wasaensia 109 Eysenbach, G. (2001). What is e-health? JMIR Publications Inc., Toronto, Canada. https://doi.org/10.2196/jmir.3.2.e20 Fathurachman, M., Kalsum, U., Safitri, N., & Utomo, C. P. (2014). Heart disease diagnosis using extreme learning based neural networks. Pp. 23-27. https://doi.org/10.1109/ICAICTA.2014.7005909 Fine, T. L. (1999). Feedforward neural network methodology. New York, NY 10010, USA: Springer. Firstbeat Technologies Ltd. (2014). Stress and recovery analysis method based on 24-hour heart rate variability. ().Firstbeat Technologies Ltd. Retrieved from https://assets.firstbeat.com/firstbeat/uploads/2015/11/Stress-and- recovery_white-paper_20145.pdf Gallego, J. R., Hernandez-Solana, A., Canales, M., Lafuente, J., Valdovinos, A., & Fernandez-Navajas, J. (2005). Performance analysis of multiplexed medical data transmission for mobile emergency care over the UMTS channel. IEEE Transactions on Information Technology in Biomedicine, 9(1), 13-22. https://doi.org/10.1109/TITB.2004.838362 Galluccio, L., Melodia, T., Palazzo, S., & Santagati, G. E. (2012). Challenges and implications of using ultrasonic communications in intra-body area networks. Pp. 182-189. https://doi.org/10.1109/WONS.2012.6152227 Geoffrey E. Hinton, Simon Osindero, & Yee-Whye Teh. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18(7) https://doi.org/10.1162/neco.2006.18.7.1527 Gerbarg, D. S., Taranta, A., Spagnuolo, M., & Hofler, J. J. (1963). Computer analysis of phonocardiograms. Progress in Cardiovascular Diseases, Retrieved from http://www.onlinepcd.com/article/S0033-0620(63)80007-9/abstract Gomes Cabral, G., & de Oliveira, Adriano Lorena Inacio. (2014). One-class classification for heart disease diagnosis. Pp. 2551-2556. https://doi.org/10.1109/SMC.2014.6974311 Greg Welch, & Gary Bishop. (2006). An introduction to the kalman filter. An introduction to the kalman filter (). Chapel Hill: University of North Carolina at Chapel Hill Department of Computer Science. Retrieved from https://www.cs.unc.edu/~welch/media/pdf/kalman_intro.pdf Haykin, S. (1999). Neural networks a comprehensive foundation (2nd ed.). Upper Saddle River, NJ: Prentice Hall. Haykin, S. S. (2009). Neural networks and learning machines (3. ed. ed.). Upper Saddle River: Pearson. Health essentials. (2014). What your doc listens for in the stethoscope. Retrieved from https://health.clevelandclinic.org/what-your-doc-listens-for-in-the- stethoscope/ 110 Acta Wasaensia Hong Tang, Ting Li, Yongwan Park, & Tianshuang Qiu. (2010). Separation of heart sound signal from noise in joint cycle frequency-time-frequency domains based on fuzzy detection. IEEE Transactions on Biomedical Engineering, 57(10), 2438- 2447. https://doi.org/10.1109/TBME.2010.2051225 Huiying Liung, & Iiro Hartimo. (2002). A feature extraction algorithm based on wavelet packet decomposition for heart sound signals. Jabbar, M. A., Deekshatulu, B. L., & Chandra, P. (2015). Computational intelligence technique for early diagnosis of heart disease. Pp. 1-6. https://doi.org/10.1109/ICETECH.2015.7275001 Jabbar, M. A., Deekshatulu, B. L., & Chndra, P. (2014). Alternating decision trees for early diagnosis of heart disease. Pp. 322-328. https://doi.org/10.1109/CIMCA.2014.7057816 Jayashree, M., & Singhi, S. (2011). Initial assessment and triage in ER. The Indian Journal of Pediatrics, 78(9), 1100-1108. https://doi.org/10.1007/s12098-011- 0411-3 Jentsch, M., Ramirez, L., Wood, L., & Elmasllari, E. (2013). The reconfiguration of triage by introduction of technology. Pp. 55-64. https://doi.org/10.1145/2493190.2493212 Jones, K. (2008). New emergency department technology cutting triage time in half. Retrieved from http://hospitalnews.com/new-emergency-department- technology-cutting-triage-time-in-half/ Kadambe, S., Murray, R., & Boudreaux-Bartels, G. F. (1999). Wavelet transform- based QRS complex detector. IEEE Transactions on Biomedical Engineering, 46(7) https://doi.org/10.1109/10.771194 Karaca, B. K., Oltu, B., Kantar, T., Kilic, E., Aksahin, M. F., & Erdamar, A. (2018). Classification of heart sound recordings with continuous wavelet transform based algorithm. https://doi.org/10.1109/SIU.2018.8404450 Khoumbati, K., Dwivedi, Y.K., Srivastava, A. and Lal, B. (2010). Handbook of Research on Advances in Health Informatics and Electronic Healthcare Applications. US: Medical Information Science Reference. Kinahan, P. (2006). Physical aspects of medical imaging. Retrieved from http://courses.washington.edu/bioen508/Lecture6-US.pdf Kirthi Devleker, & MathWorks. (2016). Understanding wavelets. Retrieved from https://se.mathworks.com/videos/understanding-wavelets-part-1-what-are- wavelets-121279.html Krishnaiah, V., Srinivas, M., Narsimha, G., & Chandra, N. S. (2014). Diagnosis of heart disease patients using fuzzy classification technique. Pp. 1-7. https://doi.org/10.1109/ICCCT2.2014.7066746 Leatham, A. (1975). Auscultation of the heart and phonocardiography. Scotland: Retrieved from http://catalog.hathitrust.org/Record/000039700 Acta Wasaensia 111 Li, J., Huang, L., Shen, Z., Zhang, Y., Fang, M., Li, B., Wang, H. (2018). Automatic classification of fetal heart rate based on convolutional neural network. IEEE Internet of Things Journal, https://doi.org/10.1109/JIOT.2018.2845128 Liang, H., Lukkarinen, S., & Hartimo, I. (1997). Heart sound segmentation algorithm based on heart sound envelogram. Pp. 105-108. https://doi.org/10.1109/CIC.1997.647841 Liang-Yu Shyu, Ying-Hsuan Wu, & Hu, W. (2004). Using wavelet transform and fuzzy neural network for VPC detection from the holter ECG. IEEE Transactions on Biomedical Engineering, 51(7), 1269-1273. https://doi.org/10.1109/TBME.2004.824131 Lim, P., Malcolme-Lawes, L., Stuber, T., Koa-Wing, M., Wright, I., Tillin, T., Kanagaratnam, P. (2011). Feasibility of multiple short, 40-s, intra-procedural ECG recordings to detect immediate changes in heart rate variability during catheter ablation for arrhythmias. Journal of Interventional Cardiac Electrophysiology, 32(2), 163-171. https://doi.org/10.1007/s10840-011-9580-2 Liu, C., Li, Q., Suresh, P. B., Vest, A., & Clifford, G. D. (2017). Multi-source features and support vector machine for heart rhythm classification. Pp. 1-4. https://doi.org/10.22489/CinC.2017.162-294 Luiz Aparecido Bortolotto, & Michel E. Safar. (2006). Blood pressure profile along the arterial tree and genetics of hypertension. Arq. Bras. Cardiol., 86(3) Retrieved from http://www.scielo.br/scielo.php?lng=en&pid=S0066- 782X2006000300002&script=sci_arttext&tlng=en M. Alsalamah, S., & Amin and J. Halloran. (2014). Diagnosis of heart disease by using a radial basis function network classification technique on patients' medical records. Paper presented at the 2014 IEEE MTT-S International Microwave Workshop Series on RF and Wireless Technologies for Biomedical and Healthcare Applications (IMWS-Bio2014), 1-4. M. G. Feshki, & O. S. Shijani. (2016). Improving the heart disease diagnosis by evolutionary algorithm of PSO and feed forward neural network. Paper presented at the Artificial Intelligence and Robotics (IRANOPEN), 2016, 48-53. Maglogiannis, I., Loukis, E., Zafiropoulos, E., & Stasis, A. (2009). Support vectors machine-based identification of heart valve diseases using heart sounds. Computer Methods and Programs in Biomedicine, 95(1), 47-61. https://doi.org/10.1016/j.cmpb.2009.01.003 Mandal, D., Chattopadhyay, M., & Mishra, I. S. (2012). A low cost non-invasive digital signal processor based (TMS320C6713) heart diagnosis system. Pp. 848- 851. https://doi.org/10.1109/RAIT.2012.6194535 Manikandan, S. (2017). Heart attack prediction system. Pp. 817-820. https://doi.org/10.1109/ICECDS.2017.8389552 Mauro Ursino. (1998). Interaction between carotid bioregulation and the pulsating heart: A mathematical model. American Journal of Physiology - Heart and 112 Acta Wasaensia Circulatory Physiology, 275(5), 1733-1747. Retrieved from http://ajpheart.physiology.org/content/275/5/H1733 Meireles, A., Figueiredo, L., & Seabra Lopes, L. (2013). A portable spatial monitoring system for autonomous heart diagnosis. Pp. 449-453. https://doi.org/10.1109/HealthCom.2013.6720718 Meiyu Li, & Lin, J. (2018). Wavelet-transform-based data-length-variation technique for fast heart rate detection using 5.8-GHz CW doppler radar. IEEE Transactions on Microwave Theory and Techniques, 66(1), 568-576. https://doi.org/10.1109/TMTT.2017.2730182 Memorial Sloan Kettering Cancer Center. (2018). Watson oncology. Retrieved from https://www.mskcc.org/about/innovative-collaborations/watson-oncology Mertins, A. (1999). Signal analysis: Wavelets, filter banks, Time/Frequency transforms and applications. John Wiley & Sons Ltd. Messner, E., Zohrer, M., & Pernkopf, F. (2018). Heart sound segmentation - an event detection approach using deep recurrent neural networks. IEEE Transactions on Biomedical Engineering. https://doi.org/10.1109/TBME.2018.2843258 Mhaskar, H., Liao, Q., & Poggio, T. (2017). When and why are deep networks better than shallow ones? Paper presented at the Thirty-First AAAI Conference on Artificial Intelligence, Microsoft. (2010). Project InnerEye - medical imaging AI to empower clinicians. Retrieved from https://www.microsoft.com/en-us/research/project/medical- image-analysis/ Miller, R. L. (2006). Building foundations. American Journal of Evaluation, 37(4), 596-598. https://doi.org/10.1177/1098214016666712 Miller, R. L. (2008). Building foundations. American Journal of Evaluation, 37(4), 596-598. https://doi.org/10.1177/1098214016666712 Mossa, H. A. L. (2008). Engineering modeling of human cardiovascular system. Paper presented at the 1st Regional Conference of Eng. Sci. NUCEJ Spatial, 307- 314. Moti Nasrabadi, A., & Heidari Kani, M. (2011). Design of ECG acquisition and transmission via bluetooth with heart disease diagnosis. Pp. 55-58. https://doi.org/10.1109/MeMeA.2011.5966642 Mutijarsa, K., Ichwan, M., & Utami, D. B. (2016). Heart rate prediction based on cycling cadence using feedforward neural network. Pp. 72-76. https://doi.org/10.1109/IC3INA.2016.7863026 NHLBI, N. (2016). Heart murmur - NHLBI, NIH. Retrieved from https://www.nhlbi.nih.gov/health/health-topics/topics/heartmurmur Acta Wasaensia 113 Nielsen, M. A. (Ed.). (2015). Neural networks and deep learning Determination Press. Retrieved from http://neuralnetworksanddeeplearning.com Ninova, P. P., Dascalov, I. K., & Dimitrova, M. I. (1978). Automated phonocardiographic screening for heart disease in children. Cardiology, 63(1), 5- 13. https://doi.org/10.1159/000169876 O.V. Melnik, Yu.A. Chelebaeva, & S.V. Chelebaev. (2017). Real-time heart rate parameter analysis based on artificial neural networks. Paper presented at the 6th MEDITERRANEAN CONFERENCE ON EMBEDDED COMPUTING, Olaniyi, E. O., Oyedotun, O. K., Helwan, A., & Adnan, K. (2015). Neural network diagnosis of heart disease. Pp. 21-24. https://doi.org/10.1109/ICABME.2015.7323241 Orphanou, K., Dagliati, A., Sacchi, L., Stassopoulou, A., Keravnou, E., & Bellazzi, R. (2016). Combining naive bayes classifiers with temporal association rules for coronary heart disease diagnosis. Pp. 81-92. https://doi.org/10.1109/ICHI.2016.15 P.M. Bentley, P.M. Grant, & J.T.E. McDonnell. (1998). Time-frequency and time- scale techniques for the classification of native and bioprosthetic heart valve sounds. IEEE Transactions on Biomedical Engineering, , 125-128. Patel, A. R., & Joshi, M. M. (2013). Heart diseases diagnosis using neural network. Pp. 1-5. https://doi.org/10.1109/ICCCNT.2013.6726740 Pater, L. D., & Jw., V. D. B. (1964). An electrical analogue of the human circulatory system. Great Britain: Pergamon Press. Retrieved from http://catalog.hathitrust.org/Record/010201851 Paul, A. K., Shill, P. C., Rabin, M. R. I., & Akhand, M. A. H. (2016). Genetic algorithm based fuzzy decision support system for the diagnosis of heart disease. Pp. 145-150. https://doi.org/10.1109/ICIEV.2016.7759984 Pavlopoulos, S., Kyriacou, E., Berler, A., Dembeyiotis, S., & Koutsouris, D. (1998). A novel emergency telemedicine system based on wireless communication technology-AMBULANCE. IEEE Transactions on Information Technology in Biomedicine, 2(4), 261-267. https://doi.org/10.1109/4233.737581 Perera, I. S., Muthalif, F. A., Selvarathnam, M., Liyanaarachchi, M. R., & Nanayakkara, N. D. (2013). Automated diagnosis of cardiac abnormalities using heart sounds. Pp. 252-255. https://doi.org/10.1109/PHT.2013.6461332 Phanphaisarn, W., Roeksabutr, A., Wardkein, P., Koseeyaporn, J., & Yupapin, P. P. (2011). Heart detection and diagnosis based on ECG and EPCG relationships. Medical Devices: Evidence and Research, Retrieved from http://www.dovepress.com/getfile.php?fileID=10848 Poonam Gupta, G.H. Raisoni, Wagholi Wagholi, & Pune, Smart ambulance system, 2016. 114 Acta Wasaensia Portnoff, M. (1980). Time-frequency representation of. digital signals and systems based on short-time Fourier analysis. IEEE Transactions on Acoustics, Speech, and Signal Processing, 28(1), 55-69. Quarteroni, A. (2006). Cardiovascular mathematics, pp. 479-512. Retrieved from http://dialnet.unirioja.es/servlet/oaiart?codigo=3139076 Randen, T. (1997). Filter and filter bank design for image texture recognition Rastgar-Jazi, M., & Fernando, X. (2017). Detection of heart abnormalities via artificial neural network: An application of self learning algorithms. Pp. 66-69. https://doi.org/10.1109/IHTC.2017.8058202 S. P. Kumar, D., Akash, K. M., & R. Shriram. (2016). Call ambulance smart elderly monitoring system with nearest ambulance detection using android and bluetooth. S. Pavlopoulos, S. Dembeyiotis, G. Konnis, D. Koutsouris. (1996). "Ambulance" - mobile unit for health care provision. Salam Ismaeel, Ali Miri, & Dharmendra Chourishi. (2015). Using the extreme learning machine (ELM) technique for heart disease diagnosis. Samani, H., & Zhu, R. (2016). Robotic automated external defibrillator ambulance for emergency medical service in smart cities. IEEE Access, 4, 268-283. https://doi.org/10.1109/ACCESS.2016.2514263 Sani, A. S., Islam, A. K. M. Muzahidul, Naz'ri Mahrin, M., Al-Mamoon, I., Baharun, S., Komaki, S., & Imai, M. (2014). A framework for remote monitoring of early heart attack diagnosis system for ambulatory patient. Pp. 159-164. https://doi.org/10.1109/IECBES.2014.7047477 Santagati, G. E., Melodia, T., Galluccio, L., & Palazzo, S. (2013). Ultrasonic networking for E-health applications. IEEE Wireless Communications, 20(4), 74- 81. https://doi.org/10.1109/MWC.2013.6590053 Santagati, G. E., & Melodia, T. (2013). Opto-ultrasonic communications in wireless body area nanonetworks. Pp. 1066-1070. https://doi.org/10.1109/ACSSC.2013.6810455 Saraçoğlu, R. (2012). Hidden markov model-based classification of heart valve disease with PCA for dimension reduction. Engineering Applications of Artificial Intelligence, 25(7), 1523-1528. https://doi.org/10.1016/j.engappai.2012.07.005 Saravanan, M., Rajini, G. K., Naseera, S., & Priyadarsini, M. J. P. (2017). Diagnosis of abnormal heartbeat using wavelet transform. Pp. 1-6. https://doi.org/10.1109/IPACT.2017.8245139 Sarma, P., Nirmala, S. R., & Sarma, K. K. (2013). Classification of ECG using some novel features. Pp. 187-191. https://doi.org/10.1109/ICETACS.2013.6691420 Schmidt, S. E., Graebe, M., Toft, E., & Struijk, J. J. (2011). No evidence of nonlinear or chaotic behavior of cardiovascular murmurs. Biomedical Signal Processing and Control, 6(2), 157-163. https://doi.org/10.1016/j.bspc.2010.07.003 Acta Wasaensia 115 Sebastian Thelen, Marie-Thérése Schneiders, & Daniel Schilberg. (2013). A multifunctional telemedicine system for pre-hospital emergency medical services. Paper presented at the eTELEMED 2013: The Fifth International Conference on eHealth, Telemedicine, and Social Medicine, 53-58. Sheik H. Salleh, Hadrina S. Hussain, Tan T. Swee, Chee-Ming Ting, Alias M. Noor, Surasak Pipatsart, Preecha P. Yupapin. (2012). Acoustic cardiac signals analysis a kalman filter–based approach. Int J Nanomedicine, (7), 2873–2881. Shemi, P. M., & Shareena, E. M. (2016). (2016). Analysis of ECG signal denoising using discrete wavelet transform. Pp. 713-718. https://doi.org/10.1109/ICETECH.2016.7569341 Shi, W. Y., & Chiao, J. (2016). Neural network based real-time heart sound monitor using a wireless wearable wrist sensor. Pp. 1-4. https://doi.org/10.1109/DCAS.2016.7791150 Shi, W. Y., Mays, J., & Chiao, J. (2016). Wireless stethoscope for recording heart and lung sound. Pp. 1-4. https://doi.org/10.1109/BIOWIRELESS.2016.7445545 Siddique Latif, Muhammad Usman, Junaid Qadi, & Rajib Rana. (2018). Abnormal heartbeat detection using recurrent neural networks. Neural Networks, SINTEF ICT. (2012). eTriage: BRIDGE project. Retrieved from http://www.bridgeproject.eu/en/bridge-results/concept-cases/etriage2 Skille, O., & Nielsen, N. (2017). Acoustics - how may sound waves behave inside the human body? Retrieved from https://physics.stackexchange.com/questions/360351/how-may-sound-waves- behave-inside-the-human-body Springer, D. B., Zühlke, L. J., Mayosi, B. M., Tarassenko, L., & Clifford, G. D. (2014). Mobile phone-based rheumatic heart disease diagnosis. https://doi.org/10.1049/cp.2014.0761 Springer, D. B., Tarassenko, L., & Clifford, G. D. (2016). Logistic regression- HSMM-based heart sound segmentation. IEEE Transactions on Biomedical Engineering, 63(4), 822-832. https://doi.org/10.1109/TBME.2015.2475278 Storve, S., Grue, J. F., Samstad, S., Dalen, H., Haugen, B. O., & Torp, H. (2016). Realtime automatic assessment of cardiac function in echocardiography. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 63(3), 358- 368. https://doi.org/10.1109/TUFFC.2016.2518306 Su, W., Huang, F., & Mo, S. (2011). An automatic wavelet transform based ECG detection algorithm. Pp. 1-3. https://doi.org/10.1109/icbbe.2011.5780255 Sufi, F., Khalil, I., & Tari, Z. (2010). A cardiod based technique to identify cardiovascular diseases using mobile phones and body sensors. Pp. 5500-5503. https://doi.org/10.1109/IEMBS.2010.5626578 116 Acta Wasaensia Suseno, J. E., & Burhanudin, M. (2017). The signal processing of heart sound from digital stethoscope for identification of heart condition using wavelet transform and neural network. Pp. 153-158. https://doi.org/10.1109/ICICOS.2017.8276354 Sveinsson, J. R., Benediktsson, J. A., & Hilmarsson, O. (1997). Feature extraction for neural network classifiers using wavelets and tree structured filter banks. vol.2. https://doi.org/10.1109/IGARSS.1997.615331 Szegedy, C., et al. (2015). Going Deeper with Convolutions. IEEE. Thiyagaraja, S. R., Vempati, J., Dantu, R., Sarma, T., & Dantu, S. (2014). Smart phone monitoring of second heart sound split. Pp. 2181-2184. https://doi.org/10.1109/EMBC.2014.6944050 Tovar, B., Wang, W. J., & Torry, J. N. (2010). Extraction of vibration feature using purpose designed wavelets for heart diagnosis. Pp. 390-404. https://doi.org/10.1109/ICWAPR.2010.5576406 Ursino, M., Antonucci, M., & Belardinelli, E. (1994). Role of active changes in venous capacity by the carotid baroreflex: Analysis with a mathematical model. AJP - Heart and Circulatory Physiology, 267(6), H2531. Retrieved from http://ajpheart.physiology.org/cgi/content/abstract/267/6/H2531 Uslu, E., & Bilgin, G. (2012). Exploiting locality based fourier transform for ECG signal diagnosis. Pp. 323-326. Retrieved from http://ieeexplore.ieee.org/document/6328891 Wang, P., Lim, C., Chauhan, S., Foo, J., & Anantharaman, V. (2007). Phonocardiographic signal analysis method using a modified hidden markov model. Annals of Biomedical Engineering, 35(3), 367-374. https://doi.org/10.1007/s10439-006-9232-3 Wang, Z., He, Z., & Chen, J. (1999). Filter banks and neural network-based feature extraction and automatic classification of electrogastrogram. Annals of Biomedical Engineering, 27(1), 88-95. https://doi.org/10.1114/1.151 Wetstone, J. (2018, -02-05T22:31:03.848Z). Getting to the heart of it: How deep learning is transforming cardiac imaging. Retrieved from https://medium.com/stanford-ai-for-healthcare/getting-to-the-heart-of-it-how- deep-learning-is-transforming-cardiac-imaging-22d34bf91a4e World Health Organization. (2013). mHealth: New horizons for health through mobile technologies: Second global survey on eHealth. Retrieved from http://search.credoreference.com/content/entry/fadtabers/mhealth/0 YML. (2017). A deeper dive into iPhone X face recognition. Retrieved from https://ymedialabs.com/a-deeper-dive-into-iphone-x-face-recognition Zhang, W., & Han, J. (Sep 2017). Towards heart sound classification without segmentation using convolutional neural network. Pp. 1-4. https://doi.org/10.22489/CinC.2017.254-164