Recursive Parameter Estimation and Its Convergence for Multivariate Normal Hidden Markov Model

Miftahul  Fikri; Zulkurnain  Abdul-Malek; Kurniawan  Atmadja

Authors

Miftahul Fikri Faculty of Electricity and Renewable Energy, Institut Teknologi PLN, Jakarta, Indonesia
Zulkurnain Abdul-Malek High Voltage and High Current Institute, Faculty of Electrical Engineering, Universiti Teknologi Malaysia, Johor, Malaysia
Kurniawan Atmadja Mathematics Study Program, Institut Sains dan Teknologi Nasional, Jakarta, Indonesia

Keywords:

Multivariate analysis, Markov chain, maximum likelihood, expectation maximization algorithm, monotonically increasing

Abstract

Hidden Markov models (HMMs) are models consisting pair of stochastic process which are commonly called observation process and a process that affect observation. Stochastic processes that affect this observation is assumed unobserving and form a Markov chain. HMM is often applied in time series data but still little application to longitudinal data because it requires more complex analysis. One of the HMMs is the multivariate normal hidden Markov model (MNHMM). The MNHMM is a HMMs which the probability of observation if the affect is known and assumed as multivariate normal distribution. This multivariate assumption causes the MNHMM applicable to longitudinal data. The main problem of MNHMM is parameter estimation and the convergence of the parameter estimator sequences. The novelty of this research is the method of estimating the MNHMM parameters used and the analysis of its convergence. Estimation of parameters is done by maximizing the likelihood function. The likelihood function is calculated using the forward-backward algorithm, then maximized recursively using the expectation maximization algorithm (EM algorithm) for obtain a model parameter estimator formula. The MNHM parameter estimator sequence obtained using the EM algorithm converges to the stationary point of the likelihood function monotonically increasing.

References

Barbu VS, Limnios N. Semi-Markov chains and hidden semi-Markov models: Toward applications. Springer; 2009.

Baum LE. An inequality and associated maximization technique in statistical estimation of probabilistic functions of Markov processes. In: Proceedings of the Third Symposium on Inequalities. 1972. p. 1-8.

Baum LE, Petrie T, Soules G, Weiss N. A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann Math Stat. 1970; 41(1): 164-171.

Cappe O. Inference in hidden Markov models. New York: Springer; 2005.

Cinlar E. Probability and stochastics. New York: Springer; 2011.

Cinlar E. Introduction to stochastic processes. Mineola: Dover Publications; 2013.

Cutajar M, Gatt E, Grech I, Casha O, Micallef J. Comparative study of automatic speech recognition techniques. IET Signal Process. 2013; 7(1): 25-46.

Dempster AP, Laird NM, Rubin DB. Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B. 1977; 39(1): 1-22.

Fikri M, Abdul-Malek Z, Esa MRM, Supriyanto E. Recursive parameter estimation and its convergence for multivariate normal hidden Markov inhomogeneous models. Malays J Fundam Appl Sci. 2023;19(5): 840-854.

Fikri M, Christiono MKIG, Abdul-Malek Z, Romadhonid ML, Esa MRM, et al. Comparison of corona discharge identification in 20 kV cubicles based on voltage and noise using ED, HMM, and FCM. J Teknol (Sci Eng). 2024; 86(5): 11-22.

Fikri M, Samsurizal, Christiono MKT. Pemodelan cuaca menggunakan model hidden Markov untuk pemanfaatan energi surya (Weather modeling using hidden Markov models for solar energy utilization). Kilat. 2020; 9(2): 217-224.

Fikri M, Setiawaty B, Purnaba IGP. Pendugaan parameter dan kekonvergenan penduga parameter model Poisson hidden Markov (Parameter Estimation and Convergence of Parameter Estimators of Poisson Hidden Markov Models). J Math Its Appl. 2016; 15(1): 1-10.

Grimmett GR, Stirzaker DR. Probability and random processes. Oxford University Press; 2001.

Gupta A, Dhingra B. Stock market prediction using hidden Markov models. In: 2012 Students Conference on Engineering and Systems (SCES). 2012. p. 1-4.

Huang C, Farewell D, Pan J. A calibration method for non-positive definite covariance matrix in multivariate data analysis. J Multivar Anal. 2017; 157: 45-52.

Khiatani D, Ghose U. Weather forecasting using hidden Markov model. In: 2017 International Conference on Computing and Communication Technologies for Smart Nation (IC3TSN). 2017. p. 220-225.

Lam C. Nonparametric eigenvalue-regularized precision or covariance matrix estimator. Ann Stat. 2016; 44(3): 928-953.

Ledoit O, Wolf M. A well-conditioned estimator for large-dimensional covariance matrices. J Multivar Anal. 2004; 88(2): 365-411.

Ledoit O, Wolf M. Analytical nonlinear shrinkage of large-dimensional covariance matrices. Ann Stat. 2020; 48(5): 3043-3065.

Leroux G, Puterman ML. Maximum-penalized-likelihood estimation for independent and Markov-dependent mixture models. Biometrics. 1992; 48(2): 545-565.

Li J, Pedrycz W, Jamal I. Multivariate time series anomaly detection: A framework of hidden Markov models. Appl Soft Comput. 2017; 60: 229-240.

Luck A, Giehr P, Nordstrom K, Walter J, Wolf V. Hidden Markov modelling reveals neighborhood dependence of Dnmt3a and 3b activity. IEEE/ACM Trans Comput Biol Bioinform. 2019; 16(5): 1598-1609.

Macdonald IL, Zucchini W. Hidden Markov and other models for discrete-valued time series. Chapman & Hall; 1997.

Pourahmadi M. Covariance estimation: The GLM and regularization perspectives. Stat Sci. 2011; 26(3): 369-387.

Mouaz B, Abderrahim BH, Abdelmajid E. Speech recognition of Moroccan dialect using hidden Markov models. Procedia Comput Sci. 2019; 151: 985-991.

Nguyen N. Hidden Markov model for stock trading. Int J Financ Stud. 2018; 6(36): 1-17.

Paroli R, Redaelli G, Spezia L. Poisson hidden Markov models for time series of overdispersed insurance counts. Astin Colloq. 2000: 461-474.

Paroli R, Spezia L. Gaussian hidden Markov models: Parameters estimation and applications to air pollution data. Univ Cattol Sacro Cuore. 1999; 94: 1-24.

Pourahmadi M. Joint mean-covariance models with applications to longitudinal data: Unconstrained parameterisation. Biometrika. 1999;86(3): 677-690.

Rabiner LR. A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE. 1989; 77(2): 257-288.

Ross SM. Introduction to probability models. 12th ed. Cambridge: Academic Press; 2019.

Rothman AJ, Levina E, Zhu J. A new approach to Cholesky-based covariance regularization in high dimensions. Biometrika. 2010; 97(3): 539-550.

Somani P, Talele S, Sawant S. Stock market prediction using hidden Markov model. In: 2014 IEEE 7th Joint International Information Technology and Artificial Intelligence Conference. 2014. p. 89-92.

Spezia L. Bayesian analysis of multivariate Gaussian hidden Markov models with an unknown number of regimes. J Time Ser Anal. 2010; 31: 1-11.

Spezia L, Futter MN, Brewer MJ. Periodic multivariate normal hidden Markov models for the analysis of water quality time series. Environmetrics. 2011; 22: 304-317.

Tao H, Lu X. Smoke vehicle detection based on multi-feature fusion and hidden Markov model. J Real-Time Image Process. 2019; 17(3): 745-758.

Trichilli Y, Boujelbène Abbes M, Masmoudi A. Predicting the effect of Googling investor sentiment on Islamic stock market returns: A five-state hidden Markov model. Int J Islam Middle East Financ Manag. 2020; 13(2): 165-193.

Wu J. On the convergence properties of the EM algorithm. Ann Stat. 1983;11(1): 95-103.

Young C, Chen C, Wu W. Numerical instability of calculating inverse of spatial covariance matrices. Stat Prob Lett. 2017; 129: 182-188.

Zangwill WI. Nonlinear programming: A unified approach. New Jersey: Prentice Hall; 1969.

Zarrabi N, Schluesche P, Meisterernst M, Börsch M, Lamb DC. Analyzing the dynamics of single TBP-DNA-NC2 complexes using hidden Markov models. Biophys J. 2018; 115(12): 2310-2326.

Zhang M, Jiang X, Fang Z, Zeng Y, Xu K. High-order hidden Markov model for trend prediction in financial time series. Phys A. 2019; 517: 1-12.