Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

This Springer brief addresses the challenges encountered in the study of the optimization of time-nonhomogeneous Markov chains. It develops new insights and new methodologies for systems in which concepts such as stationarity, ergodicity, periodicity and connectivity do not apply. This brief introduces the novel concept of confluencity and applies a relative optimization approach. It develops a comprehensive theory for optimization of the long-run average of time-nonhomogeneous Markov chains. The book shows that confluencity is the most fundamental concept in optimization, and that relative optimization is more suitable for treating the systems under consideration than standard ideas of dynamic programming. Using confluencity and relative optimization, the author classifies states as confluent or branching and shows how the under-selectivity issue of the long-run average can be easily addressed, multi-class optimization implemented, and Nth biases and Blackwell optimality conditions derived. These results are presented in a book for the first time and so may enhance the understanding of optimization and motivate new research ideas in the area.

Markov processes. --- Analysis, Markov --- Chains, Markov --- Markoff processes --- Markov analysis --- Markov chains --- Markov models --- Models, Markov --- Processes, Markov --- Stochastic processes

Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

Stochastic processes --- Markov processes --- 519.217 --- Analysis, Markov --- Chains, Markov --- Markoff processes --- Markov analysis --- Markov chains --- Markov models --- Models, Markov --- Processes, Markov --- Markov processes. --- 519.217 Markov processes

Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

Hidden semi-Markov models (HSMMs) are among the most important models in the area of artificial intelligence / machine learning. Since the first HSMM was introduced in 1980 for machine recognition of speech, three other HSMMs have been proposed, with various definitions of duration and observation distributions. Those models have different expressions, algorithms, computational complexities, and applicable areas, without explicitly interchangeable forms. Hidden Semi-Markov Models: Theory, Algorithms and Applications provides a unified and foundational approach to HSMMs, including various HSMMs (such as the explicit duration, variable transition, and residential time of HSMMs), inference and estimation algorithms, implementation methods and application instances. Learn new developments and state-of-the-art emerging topics as they relate to HSMMs, presented with examples drawn from medicine, engineering and computer science. Discusses the latest developments and emerging topics in the field of HSMMs Includes a description of applications in various areas including, Human Activity Recognition, Handwriting Recognition, Network Traffic Characterization and Anomaly Detection, and Functional MRI Brain Mapping. Shows how to master the basic techniques needed for using HSMMs and how to apply them.

Markov processes. --- Renewal theory. --- Stochastic processes --- Analysis, Markov --- Chains, Markov --- Markoff processes --- Markov analysis --- Markov chains --- Markov models --- Models, Markov --- Processes, Markov

Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

Semi-Markov Processes: Applications in System Reliability and Maintenance is a modern view of discrete state space and continuous time semi-Markov processes and their applications in reliability and maintenance. The book explains how to construct semi-Markov models and discusses the different reliability parameters and characteristics that can be obtained from those models. The book is a useful resource for mathematicians, engineering practitioners, and PhD and MSc students who want to understand the basic concepts and results of semi-Markov process theory. Clearly defines the properties and

Reliability (Engineering) --- Markov processes. --- Statistical methods. --- Analysis, Markov --- Chains, Markov --- Markoff processes --- Markov analysis --- Markov chains --- Markov models --- Models, Markov --- Processes, Markov --- Stochastic processes --- Mathematical statistics

Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

Meyn and Tweedie is back! The bible on Markov chains in general state spaces has been brought up to date to reflect developments in the field since 1996 - many of them sparked by publication of the first edition. The pursuit of more efficient simulation algorithms for complex Markovian models, or algorithms for computation of optimal policies for controlled Markov models, has opened new directions for research on Markov chains. As a result, new applications have emerged across a wide range of topics including optimisation, statistics, and economics. New commentary and an epilogue by Sean Meyn summarise recent developments and references have been fully updated. This second edition reflects the same discipline and style that marked out the original and helped it to become a classic: proofs are rigorous and concise, the range of applications is broad and knowledgeable, and key ideas are accessible to practitioners with limited mathematical background.

Stochastic processes --- Markov processes --- Markov processes. --- Mathematics. --- Math --- Science --- Analysis, Markov --- Chains, Markov --- Markoff processes --- Markov analysis --- Markov chains --- Markov models --- Models, Markov --- Processes, Markov