Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

This book concerns continuous-time controlled Markov chains, also known as continuous-time Markov decision processes. They form a class of stochastic control problems in which a single decision-maker wishes to optimize a given objective function. This book is also concerned with Markov games, where two decision-makers (or players) try to optimize their own objective function. Both decision-making processes appear in a large number of applications in economics, operations research, engineering, and computer science, among other areas. An extensive, self-contained, up-to-date analysis of basic o

Markov processes. --- Analysis, Markov --- Chains, Markov --- Markoff processes --- Markov analysis --- Markov chains --- Markov models --- Models, Markov --- Processes, Markov --- Stochastic processes

Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

Featuring previously unpublished results, Semi-Markov Models: Control of Restorable Systems with Latent Failures describes valuable methodology which can be used by readers to build mathematical models of a wide class of systems for various applications. In particular, this information can be applied to build models of reliability, queuing systems, and technical control. Beginning with a brief introduction to the area, the book covers semi-Markov models for different control strategies in one-component systems, defining their stationary characteristics of reliability and efficiency, and uti

System analysis --- Markov processes. --- Mathematical models. --- Analysis, Markov --- Chains, Markov --- Markoff processes --- Markov analysis --- Markov chains --- Markov models --- Models, Markov --- Processes, Markov --- Stochastic processes

Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

Meyn and Tweedie is back! The bible on Markov chains in general state spaces has been brought up to date to reflect developments in the field since 1996 - many of them sparked by publication of the first edition. The pursuit of more efficient simulation algorithms for complex Markovian models, or algorithms for computation of optimal policies for controlled Markov models, has opened new directions for research on Markov chains. As a result, new applications have emerged across a wide range of topics including optimisation, statistics, and economics. New commentary and an epilogue by Sean Meyn summarise recent developments and references have been fully updated. This second edition reflects the same discipline and style that marked out the original and helped it to become a classic: proofs are rigorous and concise, the range of applications is broad and knowledgeable, and key ideas are accessible to practitioners with limited mathematical background.

Stochastic processes --- Markov processes --- Markov processes. --- Mathematics. --- Math --- Science --- Analysis, Markov --- Chains, Markov --- Markoff processes --- Markov analysis --- Markov chains --- Markov models --- Models, Markov --- Processes, Markov

Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

The theory of Markov Processes has become a powerful tool in partial differential equations and potential theory with important applications to physics. Professor Dynkin has made many profound contributions to the subject and in this volume are collected several of his most important expository and survey articles. The content of these articles has not been covered in any monograph as yet. This account is accessible to graduate students in mathematics and operations research and will be welcomed by all those interested in stochastic processes and their applications.

Markov processes. --- Stochastic analysis. --- Analysis, Stochastic --- Mathematical analysis --- Stochastic processes --- Analysis, Markov --- Chains, Markov --- Markoff processes --- Markov analysis --- Markov chains --- Markov models --- Models, Markov --- Processes, Markov

Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

"Stochastic Analysis and Diffusion Processes presents a simple, mathematical introduction to Stochastic Calculus and its applications. The book builds the basic theory and offers a careful account of important research directions in Stochastic Analysis. The breadth and power of Stochastic Analysis, and probabilistic behavior of diffusion processes are told without compromising on the mathematical details. Starting with the construction of stochastic processes, the book introduces Brownian motion and martingales. The book proceeds to construct stochastic integrals, establish the Itô formula, and discuss its applications. Next, attention is focused on stochastic differential equations (SDEs) which arise in modeling physical phenomena, perturbed by random forces. Diffusion processes are solutions of SDEs and form the main theme of this book. The Stroock-Varadhan martingale problem, the connection between diffusion processes and partial differential equations, Gaussian solutions of SDEs, and Markov processes with jumps are presented in successive chapters. The book culminates with a careful treatment of important research topics such as invariant measures, ergodic behavior, and large deviation principle for diffusions. Examples are given throughout the book to illustrate concepts and results. In addition, exercises are given at the end of each chapter that will help the reader to understand the concepts better. The book is written for graduate students, young researchers and applied scientists who are interested in stochastic processes and their applications. The reader is assumed to be familiar with probability theory at graduate level. The book can be used as a text for a graduate course on Stochastic Analysis." -- Provided by publisher.

Stochastic analysis. --- Markov processes. --- Analysis, Markov --- Chains, Markov --- Markoff processes --- Markov analysis --- Markov chains --- Markov models --- Models, Markov --- Processes, Markov --- Stochastic processes --- Analysis, Stochastic --- Mathematical analysis --- Diffusion processes.