SU\G(𝔸)/K·U
Sign Up for free Log In

Author Description

Login to generate an author description

Ask a Question About This Mathematician

Bayesian mixture of autoregressive models

2008-06-11
John W. Lau , Mike K. P. So

Gibbs Partitions (EPPF's) Derived From a Stable Subordinator are Fox H and Meijer G Transforms

2007-01-01
Man-Wai Ho , Lancelot F. James , John W. Lau
This paper derives explicit results for the infinite Gibbs partitions generated by the jumps of an $α-$stable subordinator, derived in Pitman \cite{Pit02, Pit06}. We first show that for general $α$ … This paper derives explicit results for the infinite Gibbs partitions generated by the jumps of an $α-$stable subordinator, derived in Pitman \cite{Pit02, Pit06}. We first show that for general $α$ the conditional EPPF can be represented as ratios of Fox-$H$ functions, and in the case of rational $α,$ Meijer-G functions. Furthermore the results show that the resulting unconditional EPPF's, can be expressed in terms of H and G transforms indexed by a function h. Hence when h is itself a H or G function the EPPF is also an H or G function. An implication, in the case of rational $α,$ is that one can compute explicitly thousands of EPPF's derived from possibly exotic special functions. This would also apply to all $α$ except that computations for general Fox functions are not yet available. However, moving away from special functions, we demonstrate how results from probability theory may be used to obtain calculations. We show that a forward recursion can be applied that only requires calculation of the simplest components. Additionally we identify general classes of EPPF's where explicit calculations can be carried out using distribution theory.

Bayesian nonparametric estimation and consistency of mixed multinomial logit choice models

2010-08-01
Pierpaolo De Blasi , Lancelot F. James , John W. Lau
This paper develops nonparametric estimation for discrete choice models based on the mixed multinomial logit (MMNL) model. It has been shown that MMNL models encompass all discrete choice models derived … This paper develops nonparametric estimation for discrete choice models based on the mixed multinomial logit (MMNL) model. It has been shown that MMNL models encompass all discrete choice models derived under the assumption of random utility maximization, subject to the identification of an unknown distribution $G$. Noting the mixture model description of the MMNL, we employ a Bayesian nonparametric approach, using nonparametric priors on the unknown mixing distribution $G$, to estimate choice probabilities. We provide an important theoretical support for the use of the proposed methodology by investigating consistency of the posterior distribution for a general nonparametric prior on the mixing distribution. Consistency is defined according to an $L_1$-type distance on the space of choice probabilities and is achieved by extending to a regression model framework a recent approach to strong consistency based on the summability of square roots of prior probabilities. Moving to estimation, slightly different techniques for non-panel and panel data models are discussed. For practical implementation, we describe efficient and relatively easy-to-use blocked Gibbs sampling procedures. These procedures are based on approximations of the random probability measure by classes of finite stick-breaking processes. A simulation study is also performed to investigate the performance of the proposed methods.
View PDF Chat

Viterbi-Based Estimation for Markov Switching GARCH Model

2012-01-04
Robert J. Elliott , John W. Lau , Hong Miao +1 more
Abstract We outline a two-stage estimation method for a Markov-switching Generalized Autoregressive Conditional Heteroscedastic (GARCH) model modulated by a hidden Markov chain. The first stage involves the estimation of a … Abstract We outline a two-stage estimation method for a Markov-switching Generalized Autoregressive Conditional Heteroscedastic (GARCH) model modulated by a hidden Markov chain. The first stage involves the estimation of a hidden Markov chain using the Vitberi algorithm given the model parameters. The second stage uses the maximum likelihood method to estimate the model parameters given the estimated hidden Markov chain. Applications to financial risk management are discussed through simulated data.
View PDF Chat

On Bayesian Mixture Credibility

2006-11-01
John W. Lau , Tak Kuen Siu , Hailiang Yang
We introduce a class of Bayesian infinite mixture models first introduced by Lo (1984) to determine the credibility premium for a non-homogeneous insurance portfolio. The Bayesian infinite mixture models provide … We introduce a class of Bayesian infinite mixture models first introduced by Lo (1984) to determine the credibility premium for a non-homogeneous insurance portfolio. The Bayesian infinite mixture models provide us with much flexibility in the specification of the claim distribution. We employ the sampling scheme based on a weighted Chinese restaurant process introduced in Lo et al. (1996) to estimate a Bayesian infinite mixture model from the claim data. The Bayesian sampling scheme also provides a systematic way to cluster the claim data. This can provide some insights into the risk characteristics of the policyholders. The estimated credibility premium from the Bayesian infinite mixture model can be written as a linear combination of the prior estimate and the sample mean of the claim data. Estimation results for the Bayesian mixture credibility premiums will be presented.
View PDF Chat

Bayesian semi-parametric modeling for mixed proportional hazard models with right censoring

2005-11-08
John W. Lau

A Monte Carlo Markov chain algorithm for a class of mixture time series models

2009-08-28
John W. Lau , Mike K. P. So

A conjugate class of random probability measures based on tilting and with its posterior analysis

2013-11-01
John W. Lau
This article constructs a class of random probability measures based on exponentially and polynomially tilting operated on the laws of completely random measures. The class is proved to be conjugate … This article constructs a class of random probability measures based on exponentially and polynomially tilting operated on the laws of completely random measures. The class is proved to be conjugate in that it covers both prior and posterior random probability measures in the Bayesian sense. Moreover, the class includes some common and widely used random probability measures, the normalized completely random measures (James (Poisson process partition calculus with applications to exchangeable models and Bayesian nonparametrics (2002) Preprint), Regazzini, Lijoi and Pr\"{u}nster (Ann. Statist. 31 (2003) 560-585), Lijoi, Mena and Pr\"{u}nster (J. Amer. Statist. Assoc. 100 (2005) 1278-1291)) and the Poisson-Dirichlet process (Pitman and Yor (Ann. Probab. 25 (1997) 855-900), Ishwaran and James (J. Amer. Statist. Assoc. 96 (2001) 161-173), Pitman (In Science and Statistics: A Festschrift for Terry Speed (2003) 1-34 IMS)), in a single construction. We describe an augmented version of the Blackwell-MacQueen P\'{o}lya urn sampling scheme (Blackwell and MacQueen (Ann. Statist. 1 (1973) 353-355)) that simplifies implementation and provide a simulation study for approximating the probabilities of partition sizes.
View PDF Chat

Modelling long-term investment returns via Bayesian infinite mixture time series models

2008-05-12
John W. Lau , Tak Kuen Siu
Abstract This paper introduces the class of Bayesian infinite mixture time series models first proposed in Lau & So (2004) for modelling long-term investment returns. It is a flexible class … Abstract This paper introduces the class of Bayesian infinite mixture time series models first proposed in Lau & So (2004) for modelling long-term investment returns. It is a flexible class of time series models and provides a flexible way to incorporate full information contained in all autoregressive components with various orders by utilizing the idea of Bayesian averaging or mixing. We adopt a Bayesian sampling scheme based on a weighted Chinese restaurant process for generating partitions of investment returns to estimate the Bayesian infinite mixture time series models. Instead of using the point estimates, as in the classical or non-Bayesian approach, the estimation in this paper is performed by the full Bayesian approach, utilizing the idea of Bayesian averaging to incorporate all information contained in the posterior distributions of the random parameters. This provides a natural way to incorporate model risk or uncertainty. The proposed models can also be used to perform clustering of investment returns and detect outliers of returns. We employ the monthly data from the Toronto Stock Exchange 300 (TSE 300) indices to illustrate the implementation of our models and compare the simulated results from the estimated models with the empirical characteristics of the TSE 300 data. We apply the Bayesian predictive distribution of the logarithmic returns obtained by the Bayesian averaging or mixing to evaluate the quantile-based and conditional tail expectation risk measures for segregated fund contracts via stochastic simulation. We compare the risk measures evaluated from our models with those from some well-known and important models in the literature, and highlight some features that can be obtained from our models. Keywords: Bayesian MAR modelsBayesian mixture AR-ARCH modelsWeighted Chinese restaurant processClustering of returnsOutliers detectionDirichlet prior processQuantile-based risk measuresConditional tail expectation Acknowledgements We would like to thank the referee for many helpful and valuable comments and suggestions.

Coagulation Fragmentation Laws Induced By General Coagulations of Two-Parameter Poisson-Dirichlet Processes

2006-01-01
Man-Wai Ho , Lancelot F. James , John W. Lau
Pitman~(1999) describes a duality relationship between fragmentation and coagulation operators. An explicit relationship is described for the two-parameter Poisson-Dirichlet laws, with parameters {\footnotesize $(α,θ)$} and $(β,θ/α)$, wherein $PD(α, θ)$ is … Pitman~(1999) describes a duality relationship between fragmentation and coagulation operators. An explicit relationship is described for the two-parameter Poisson-Dirichlet laws, with parameters {\footnotesize $(α,θ)$} and $(β,θ/α)$, wherein $PD(α, θ)$ is coagulated by $PD(β,θ/α)$ for $0

Gibbs partitions, Riemann–Liouville fractional operators, Mittag–Leffler functions, and fragmentations derived from stable subordinators

2021-06-01
Man-Wai Ho , Lancelot F. James , John W. Lau
Abstract Pitman (2003), and subsequently Gnedin and Pitman (2006), showed that a large class of random partitions of the integers derived from a stable subordinator of index $\alpha\in(0,1)$ have infinite … Abstract Pitman (2003), and subsequently Gnedin and Pitman (2006), showed that a large class of random partitions of the integers derived from a stable subordinator of index $\alpha\in(0,1)$ have infinite Gibbs (product) structure as a characterizing feature. The most notable case are random partitions derived from the two-parameter Poisson–Dirichlet distribution, $\textrm{PD}(\alpha,\theta)$ , whose corresponding $\alpha$ -diversity/local time have generalized Mittag–Leffler distributions, denoted by $\textrm{ML}(\alpha,\theta)$ . Our aim in this work is to provide indications on the utility of the wider class of Gibbs partitions as it relates to a study of Riemann–Liouville fractional integrals and size-biased sampling, and in decompositions of special functions, and its potential use in the understanding of various constructions of more exotic processes. We provide characterizations of general laws associated with nested families of $\textrm{PD}(\alpha,\theta)$ mass partitions that are constructed from fragmentation operations described in Dong et al. (2014). These operations are known to be related in distribution to various constructions of discrete random trees/graphs in [ n ], and their scaling limits. A centerpiece of our work is results related to Mittag–Leffler functions, which play a key role in fractional calculus and are otherwise Laplace transforms of the $\textrm{ML}(\alpha,\theta)$ variables. Notably, this leads to an interpretation within the context of $\textrm{PD}(\alpha,\theta)$ laws conditioned on Poisson point process counts over intervals of scaled lengths of the $\alpha$ -diversity.
View PDF Chat

Bayesian Non-Parametric Mixtures of GARCH(1,1) Models

2012-01-01
John W. Lau , Edward Cripps
Traditional GARCH models describe volatility levels that evolve smoothly over time, generated by a single GARCH regime. However, nonstationary time series data may exhibit abrupt changes in volatility, suggesting changes … Traditional GARCH models describe volatility levels that evolve smoothly over time, generated by a single GARCH regime. However, nonstationary time series data may exhibit abrupt changes in volatility, suggesting changes in the underlying GARCH regimes. Further, the number and times of regime changes are not always obvious. This article outlines a nonparametric mixture of GARCH models that is able to estimate the number and time of volatility regime changes by mixing over the Poisson-Kingman process. The process is a generalisation of the Dirichlet process typically used in nonparametric models for time-dependent data provides a richer clustering structure, and its application to time series data is novel. Inference is Bayesian, and a Markov chain Monte Carlo algorithm to explore the posterior distribution is described. The methodology is illustrated on the Standard and Poor's 500 financial index.
View PDF Chat

Filtering a Double Threshold Model With Regime Switching

2013-05-02
Robert J. Elliott , Tak Kuen Siu , John W. Lau
We introduce a new double threshold model with regime switches. New filtering equations are derived based on a reference probability approach. We also propose a new and practically useful method … We introduce a new double threshold model with regime switches. New filtering equations are derived based on a reference probability approach. We also propose a new and practically useful method for implementing the filtering equations.

Stick-Breaking Representation and Computation for Normalized Generalized Gamma Processes

2015-07-22
John W. Lau , Edward Cripps

Variational inference for multiplicative intensity models

2020-02-04
John W. Lau , Edward Cripps , Wendy Hui

On Bayesian Mixture Credibility

2006-11-01
John W. Lau , Tak Kuen Siu , Hailiang Yang
We introduce a class of Bayesian infinite mixture models first introduced by Lo (1984) to determine the credibility premium for a non-homogeneous insurance portfolio. The Bayesian infinite mixture models provide … We introduce a class of Bayesian infinite mixture models first introduced by Lo (1984) to determine the credibility premium for a non-homogeneous insurance portfolio. The Bayesian infinite mixture models provide us with much flexibility in the specification of the claim distribution. We employ the sampling scheme based on a weighted Chinese restaurant process introduced in Lo et al. (1996) to estimate a Bayesian infinite mixture model from the claim data. The Bayesian sampling scheme also provides a systematic way to cluster the claim data. This can provide some insights into the risk characteristics of the policyholders. The estimated credibility premium from the Bayesian infinite mixture model can be written as a linear combination of the prior estimate and the sample mean of the claim data. Estimation results for the Bayesian mixture credibility premiums will be presented.
View PDF Chat

Remaining useful life prediction: A multiple product partition approach

2020-05-19
John W. Lau , Edward Cripps , Sally Cripps
This article introduces a Bayesian multiple change point model for a collection of degradation signals in order to predict remaining useful life of rotational bearings. The model is designed for … This article introduces a Bayesian multiple change point model for a collection of degradation signals in order to predict remaining useful life of rotational bearings. The model is designed for longitudinal data, where each trajectory is a time series segmented into multiple states of degradation using a product partition structure. An efficient Markov chain Monte Carlo algorithm is designed to implement the model. The model is run on in situ data, where vibration measurements are taken to indicate bearing degradation. The results suggest that bearing degradation exhibit an auto-correlation structure that we incorporate into the product partition model and often experience more than one degradation phase.

A Class of Generalized Hyperbolic Continuous Time Integrated Stochastic Volatility Likelihood Models

2005-01-01
Lancelot F. James , John W. Lau
This paper discusses and analyzes a class of likelihood models which are based on two distributional innovations in financial models for stock returns. That is, the notion that the marginal … This paper discusses and analyzes a class of likelihood models which are based on two distributional innovations in financial models for stock returns. That is, the notion that the marginal distribution of aggregate returns of log-stock prices are well approximated by generalized hyperbolic distributions, and that volatility clustering can be handled by specifying the integrated volatility as a random process such as that proposed in a recent series of papers by Barndorff-Nielsen and Shephard (BNS). The BNS models produce likelihoods for aggregate returns which can be viewed as a subclass of latent regression models where one has n conditionally independent Normal random variables whose mean and variance are representable as linear functionals of a common unobserved Poisson random measure. James (2005b) recently obtains an exact analysis for such models yielding expressions of the likelihood in terms of quite tractable Fourier-Cosine integrals. Here, our idea is to analyze a class of likelihoods, which can be used for similar purposes, but where the latent regression models are based on n conditionally independent models with distributions belonging to a subclass of the generalized hyperbolic distributions and whose corresponding parameters are representable as linear functionals of a common unobserved Poisson random measure. Our models are perhaps most closely related to the Normal inverse Gaussian/GARCH/A-PARCH models of Brandorff-Nielsen (1997) and Jensen and Lunde (2001), where in our case the GARCH component is replaced by quantities such as INT-OU processes. It is seen that, importantly, such likelihood models exhibit quite different features structurally. One nice feature of the model is that it allows for more flexibility in terms of modelling of external regression parameters.

Gibbs Partitions, Riemann-Liouville Fractional Operators, Mittag-Leffler Functions, and Fragmentations Derived From Stable Subordinators

2018-01-01
Man-Wai Ho , Lancelot F. James , John W. Lau
Pitman(2003)(and subsequently Gnedin and Pitman (2006) showed that a large class of random partitions of the integers derived from a stable subordinator of index $\alpha\in(0,1)$ have infinite Gibbs (product) structure … Pitman(2003)(and subsequently Gnedin and Pitman (2006) showed that a large class of random partitions of the integers derived from a stable subordinator of index $\alpha\in(0,1)$ have infinite Gibbs (product) structure as a characterizing feature. The most notable case are random partitions derived from the two-parameter Poisson-Dirichlet distribution, $\mathrm{PD}(\alpha,\theta)$, which are induced by mixing over variables with generalized Mittag-Leffler distributions, denoted by $\mathrm{ML}(\alpha,\theta).$ Our aim in this work is to provide indications on the utility of the wider class of Gibbs partitions as it relates to a study of Riemann-Liouville fractional integrals and size-biased sampling, decompositions of special functions, and its potential use in the understanding of various constructions of more exotic processes. We provide novel characterizations of general laws associated with two nested families of $\mathrm{PD}(\alpha,\theta)$ mass partitions that are constructed from notable fragmentation operations described in Dong, Goldschmidt and Martin(2006) and Pitman(1999), respectively. These operations are known to be related in distribution to various constructions of discrete random trees/graphs in $[n],$ and their scaling limits, such as stable trees. A centerpiece of our work are results related to Mittag-Leffler functions, which play a key role in fractional calculus and are otherwise Laplace transforms of the $\mathrm{ML}(\alpha,\theta)$ variables. Notably, this leads to an interpretation of $\mathrm{PD}(\alpha,\theta)$ laws within a mixed Poisson waiting time framework based on $\mathrm{ML}(\alpha,\theta)$ variables, which suggests connections to recent construction of P\'olya urn models with random immigration by Pek\"oz, R\"ollin and Ross(2018). Simplifications in the Brownian case are highlighted.
View PDF Chat

Option Pricing When the Regime-Switching Risk is Priced

2007-11-01
Tak Kuen Siu , Hailiang Yang Unim , John W. Lau
Recently, there has been considerable interest in investigating option valuation problem in the context of regime-switching models. However, most of the literature consider the case that the risk due to … Recently, there has been considerable interest in investigating option valuation problem in the context of regime-switching models. However, most of the literature consider the case that the risk due to switching regimes is not priced. Relatively little attention has been paid to investigate the impact of switching regimes on the option price when this source of risk is priced. In this paper, we shall articulate this important problem and consider the pricing of an option when the price dynamic of the underlying risky asset is governed by a Markov-modulated geometric Brownian motion. We suppose that the drift and volatility of the underlying risky asset switch over time according to the state of an economy, which is modeled by a continuous-time hidden Markov chain. We shall develop a two-stage pricing model which can price both the diffusion risk and the regime-switching risk based on the Esscher transform and the minimization of the maximum entropy between an equivalent martingale measure and the real-world probability measure over different states. The latter is called a min-max entropy problem. We shall conduct numerical experiments to illustrate the effect of pricing regime-switching risk. The results of the numerical experiments reveal that the impact of pricing regime-switching risk on the option prices is significant.

Inverse clustering of Gibbs Partitions via independent fragmentation and dual dependent coagulation operators

2022-01-01
Man Wai Ho , Lancelot F. James , John W. Lau
Gibbs partitions of the integers generated by stable subordinators of index $\alpha\in(0,1)$ form remarkable classes of random partitions where in principle much is known about their properties, including practically effortless … Gibbs partitions of the integers generated by stable subordinators of index $\alpha\in(0,1)$ form remarkable classes of random partitions where in principle much is known about their properties, including practically effortless obtainment of otherwise complex asymptotic results potentially relevant to applications in general combinatorial stochastic processes, random tree/graph growth models and Bayesian statistics. This class includes the well-known models based on the two-parameter Poisson-Dirichlet distribution which forms the bulk of explicit applications. This work continues efforts to provide interpretations for a larger classes of Gibbs partitions by embedding important operations within this framework. Here we address the formidable problem of extending the dual, infinite-block, coagulation/fragmentation results of Jim Pitman (1999, Annals of Probability), where in terms of coagulation they are based on independent two-parameter Poisson-Dirichlet distributions, to all such Gibbs (stable Poisson-Kingman) models. Our results create nested families of Gibbs partitions, and corresponding mass partitions, over any $0<\beta<\alpha<1.$ We primarily focus on the fragmentation operations, which remain independent in this setting, and corresponding remarkable calculations for Gibbs partitions derived from that operation. We also present definitive results for the dual coagulation operations, now based on our construction of dependent processes, and demonstrate its relatively simple application in terms of Mittag-Leffler and generalized gamma models. The latter demonstrates another approach to recover the duality results in Pitman (1999).
View PDF Chat

Detecting and Mitigating Algorithmic Bias in Binary Classification using Causal Modeling

2023-01-01
Wendy Hui , John W. Lau
This paper proposes the use of causal modeling to detect and mitigate algorithmic bias. We provide a brief description of causal modeling and a general overview of our approach. We … This paper proposes the use of causal modeling to detect and mitigate algorithmic bias. We provide a brief description of causal modeling and a general overview of our approach. We then use the Adult dataset, which is available for download from the UC Irvine Machine Learning Repository, to develop (1) a prediction model, which is treated as a black box, and (2) a causal model for bias mitigation. In this paper, we focus on gender bias and the problem of binary classification. We show that gender bias in the prediction model is statistically significant at the 0.05 level. We demonstrate the effectiveness of the causal model in mitigating gender bias by cross-validation. Furthermore, we show that the overall classification accuracy is improved slightly. Our novel approach is intuitive, easy-to-use, and can be implemented using existing statistical software tools such as "lavaan" in R. Hence, it enhances explainability and promotes trust.

Mitigating Nonlinear Algorithmic Bias in Binary Classification

2023-01-01
Wendy Hui , John W. Lau
This paper proposes the use of causal modeling to detect and mitigate algorithmic bias that is nonlinear in the protected attribute. We provide a general overview of our approach. We … This paper proposes the use of causal modeling to detect and mitigate algorithmic bias that is nonlinear in the protected attribute. We provide a general overview of our approach. We use the German Credit data set, which is available for download from the UC Irvine Machine Learning Repository, to develop (1) a prediction model, which is treated as a black box, and (2) a causal model for bias mitigation. In this paper, we focus on age bias and the problem of binary classification. We show that the probability of getting correctly classified as "low risk" is lowest among young people. The probability increases with age nonlinearly. To incorporate the nonlinearity into the causal model, we introduce a higher order polynomial term. Based on the fitted causal model, the de-biased probability estimates are computed, showing improved fairness with little impact on overall classification accuracy. Causal modeling is intuitive and, hence, its use can enhance explicability and promotes trust among different stakeholders of AI.
View PDF Chat

Mitigating Nonlinear Algorithmic Bias in Binary Classification

2024-06-25
Wendy Hui , John W. Lau
View PDF Chat

Mitigating Algorithmic Bias in Multiclass CNN Classifications Using Causal Modeling

2025-01-14
Min Kwang Byun , Wendy Hui , John W. Lau
This study describes a procedure for applying causal modeling to detect and mitigate algorithmic bias in a multiclass classification problem. The dataset was derived from the FairFace dataset, supplemented with … This study describes a procedure for applying causal modeling to detect and mitigate algorithmic bias in a multiclass classification problem. The dataset was derived from the FairFace dataset, supplemented with emotional labels generated by the DeepFace pre-trained model. A custom Convolutional Neural Network (CNN) was developed, consisting of four convolutional blocks, followed by fully connected layers and dropout layers to mitigate overfitting. Gender bias was identified in the CNN model's classifications: Females were more likely to be classified as "happy" or "sad," while males were more likely to be classified as "neutral." To address this, the one-vs-all (OvA) technique was applied. A causal model was constructed for each emotion class to adjust the CNN model's predicted class probabilities. The adjusted probabilities for the various classes were then aggregated by selecting the class with the highest probability. The resulting debiased classifications demonstrated enhanced gender fairness across all classes, with negligible impact--or even a slight improvement--on overall accuracy. This study highlights that algorithmic fairness and accuracy are not necessarily trade-offs. All data and code for this study are publicly available for download.
View PDF Chat

Mixtures of generalized gamma convolution processes

2025-04-27
John W. Lau

Mixtures of generalized gamma convolution processes

2025-04-27
John W. Lau

Mitigating Algorithmic Bias in Multiclass CNN Classifications Using Causal Modeling

2025-01-14
Min Kwang Byun , Wendy Hui , John W. Lau
This study describes a procedure for applying causal modeling to detect and mitigate algorithmic bias in a multiclass classification problem. The dataset was derived from the FairFace dataset, supplemented with … This study describes a procedure for applying causal modeling to detect and mitigate algorithmic bias in a multiclass classification problem. The dataset was derived from the FairFace dataset, supplemented with emotional labels generated by the DeepFace pre-trained model. A custom Convolutional Neural Network (CNN) was developed, consisting of four convolutional blocks, followed by fully connected layers and dropout layers to mitigate overfitting. Gender bias was identified in the CNN model's classifications: Females were more likely to be classified as "happy" or "sad," while males were more likely to be classified as "neutral." To address this, the one-vs-all (OvA) technique was applied. A causal model was constructed for each emotion class to adjust the CNN model's predicted class probabilities. The adjusted probabilities for the various classes were then aggregated by selecting the class with the highest probability. The resulting debiased classifications demonstrated enhanced gender fairness across all classes, with negligible impact--or even a slight improvement--on overall accuracy. This study highlights that algorithmic fairness and accuracy are not necessarily trade-offs. All data and code for this study are publicly available for download.
View PDF Chat

Mitigating Nonlinear Algorithmic Bias in Binary Classification

2024-06-25
Wendy Hui , John W. Lau
View PDF Chat

Detecting and Mitigating Algorithmic Bias in Binary Classification using Causal Modeling

2023-01-01
Wendy Hui , John W. Lau
This paper proposes the use of causal modeling to detect and mitigate algorithmic bias. We provide a brief description of causal modeling and a general overview of our approach. We … This paper proposes the use of causal modeling to detect and mitigate algorithmic bias. We provide a brief description of causal modeling and a general overview of our approach. We then use the Adult dataset, which is available for download from the UC Irvine Machine Learning Repository, to develop (1) a prediction model, which is treated as a black box, and (2) a causal model for bias mitigation. In this paper, we focus on gender bias and the problem of binary classification. We show that gender bias in the prediction model is statistically significant at the 0.05 level. We demonstrate the effectiveness of the causal model in mitigating gender bias by cross-validation. Furthermore, we show that the overall classification accuracy is improved slightly. Our novel approach is intuitive, easy-to-use, and can be implemented using existing statistical software tools such as "lavaan" in R. Hence, it enhances explainability and promotes trust.

Mitigating Nonlinear Algorithmic Bias in Binary Classification

2023-01-01
Wendy Hui , John W. Lau
This paper proposes the use of causal modeling to detect and mitigate algorithmic bias that is nonlinear in the protected attribute. We provide a general overview of our approach. We … This paper proposes the use of causal modeling to detect and mitigate algorithmic bias that is nonlinear in the protected attribute. We provide a general overview of our approach. We use the German Credit data set, which is available for download from the UC Irvine Machine Learning Repository, to develop (1) a prediction model, which is treated as a black box, and (2) a causal model for bias mitigation. In this paper, we focus on age bias and the problem of binary classification. We show that the probability of getting correctly classified as "low risk" is lowest among young people. The probability increases with age nonlinearly. To incorporate the nonlinearity into the causal model, we introduce a higher order polynomial term. Based on the fitted causal model, the de-biased probability estimates are computed, showing improved fairness with little impact on overall classification accuracy. Causal modeling is intuitive and, hence, its use can enhance explicability and promotes trust among different stakeholders of AI.
View PDF Chat

Inverse clustering of Gibbs Partitions via independent fragmentation and dual dependent coagulation operators

2022-01-01
Man Wai Ho , Lancelot F. James , John W. Lau
Gibbs partitions of the integers generated by stable subordinators of index $\alpha\in(0,1)$ form remarkable classes of random partitions where in principle much is known about their properties, including practically effortless … Gibbs partitions of the integers generated by stable subordinators of index $\alpha\in(0,1)$ form remarkable classes of random partitions where in principle much is known about their properties, including practically effortless obtainment of otherwise complex asymptotic results potentially relevant to applications in general combinatorial stochastic processes, random tree/graph growth models and Bayesian statistics. This class includes the well-known models based on the two-parameter Poisson-Dirichlet distribution which forms the bulk of explicit applications. This work continues efforts to provide interpretations for a larger classes of Gibbs partitions by embedding important operations within this framework. Here we address the formidable problem of extending the dual, infinite-block, coagulation/fragmentation results of Jim Pitman (1999, Annals of Probability), where in terms of coagulation they are based on independent two-parameter Poisson-Dirichlet distributions, to all such Gibbs (stable Poisson-Kingman) models. Our results create nested families of Gibbs partitions, and corresponding mass partitions, over any $0<\beta<\alpha<1.$ We primarily focus on the fragmentation operations, which remain independent in this setting, and corresponding remarkable calculations for Gibbs partitions derived from that operation. We also present definitive results for the dual coagulation operations, now based on our construction of dependent processes, and demonstrate its relatively simple application in terms of Mittag-Leffler and generalized gamma models. The latter demonstrates another approach to recover the duality results in Pitman (1999).
View PDF Chat

Gibbs partitions, Riemann–Liouville fractional operators, Mittag–Leffler functions, and fragmentations derived from stable subordinators

2021-06-01
Man-Wai Ho , Lancelot F. James , John W. Lau
Abstract Pitman (2003), and subsequently Gnedin and Pitman (2006), showed that a large class of random partitions of the integers derived from a stable subordinator of index $\alpha\in(0,1)$ have infinite … Abstract Pitman (2003), and subsequently Gnedin and Pitman (2006), showed that a large class of random partitions of the integers derived from a stable subordinator of index $\alpha\in(0,1)$ have infinite Gibbs (product) structure as a characterizing feature. The most notable case are random partitions derived from the two-parameter Poisson–Dirichlet distribution, $\textrm{PD}(\alpha,\theta)$ , whose corresponding $\alpha$ -diversity/local time have generalized Mittag–Leffler distributions, denoted by $\textrm{ML}(\alpha,\theta)$ . Our aim in this work is to provide indications on the utility of the wider class of Gibbs partitions as it relates to a study of Riemann–Liouville fractional integrals and size-biased sampling, and in decompositions of special functions, and its potential use in the understanding of various constructions of more exotic processes. We provide characterizations of general laws associated with nested families of $\textrm{PD}(\alpha,\theta)$ mass partitions that are constructed from fragmentation operations described in Dong et al. (2014). These operations are known to be related in distribution to various constructions of discrete random trees/graphs in [ n ], and their scaling limits. A centerpiece of our work is results related to Mittag–Leffler functions, which play a key role in fractional calculus and are otherwise Laplace transforms of the $\textrm{ML}(\alpha,\theta)$ variables. Notably, this leads to an interpretation within the context of $\textrm{PD}(\alpha,\theta)$ laws conditioned on Poisson point process counts over intervals of scaled lengths of the $\alpha$ -diversity.
View PDF Chat

Remaining useful life prediction: A multiple product partition approach

2020-05-19
John W. Lau , Edward Cripps , Sally Cripps
This article introduces a Bayesian multiple change point model for a collection of degradation signals in order to predict remaining useful life of rotational bearings. The model is designed for … This article introduces a Bayesian multiple change point model for a collection of degradation signals in order to predict remaining useful life of rotational bearings. The model is designed for longitudinal data, where each trajectory is a time series segmented into multiple states of degradation using a product partition structure. An efficient Markov chain Monte Carlo algorithm is designed to implement the model. The model is run on in situ data, where vibration measurements are taken to indicate bearing degradation. The results suggest that bearing degradation exhibit an auto-correlation structure that we incorporate into the product partition model and often experience more than one degradation phase.

Variational inference for multiplicative intensity models

2020-02-04
John W. Lau , Edward Cripps , Wendy Hui

Gibbs Partitions, Riemann-Liouville Fractional Operators, Mittag-Leffler Functions, and Fragmentations Derived From Stable Subordinators

2018-01-01
Man-Wai Ho , Lancelot F. James , John W. Lau
Pitman(2003)(and subsequently Gnedin and Pitman (2006) showed that a large class of random partitions of the integers derived from a stable subordinator of index $\alpha\in(0,1)$ have infinite Gibbs (product) structure … Pitman(2003)(and subsequently Gnedin and Pitman (2006) showed that a large class of random partitions of the integers derived from a stable subordinator of index $\alpha\in(0,1)$ have infinite Gibbs (product) structure as a characterizing feature. The most notable case are random partitions derived from the two-parameter Poisson-Dirichlet distribution, $\mathrm{PD}(\alpha,\theta)$, which are induced by mixing over variables with generalized Mittag-Leffler distributions, denoted by $\mathrm{ML}(\alpha,\theta).$ Our aim in this work is to provide indications on the utility of the wider class of Gibbs partitions as it relates to a study of Riemann-Liouville fractional integrals and size-biased sampling, decompositions of special functions, and its potential use in the understanding of various constructions of more exotic processes. We provide novel characterizations of general laws associated with two nested families of $\mathrm{PD}(\alpha,\theta)$ mass partitions that are constructed from notable fragmentation operations described in Dong, Goldschmidt and Martin(2006) and Pitman(1999), respectively. These operations are known to be related in distribution to various constructions of discrete random trees/graphs in $[n],$ and their scaling limits, such as stable trees. A centerpiece of our work are results related to Mittag-Leffler functions, which play a key role in fractional calculus and are otherwise Laplace transforms of the $\mathrm{ML}(\alpha,\theta)$ variables. Notably, this leads to an interpretation of $\mathrm{PD}(\alpha,\theta)$ laws within a mixed Poisson waiting time framework based on $\mathrm{ML}(\alpha,\theta)$ variables, which suggests connections to recent construction of P\'olya urn models with random immigration by Pek\"oz, R\"ollin and Ross(2018). Simplifications in the Brownian case are highlighted.
View PDF Chat

Stick-Breaking Representation and Computation for Normalized Generalized Gamma Processes

2015-07-22
John W. Lau , Edward Cripps

A conjugate class of random probability measures based on tilting and with its posterior analysis

2013-11-01
John W. Lau
This article constructs a class of random probability measures based on exponentially and polynomially tilting operated on the laws of completely random measures. The class is proved to be conjugate … This article constructs a class of random probability measures based on exponentially and polynomially tilting operated on the laws of completely random measures. The class is proved to be conjugate in that it covers both prior and posterior random probability measures in the Bayesian sense. Moreover, the class includes some common and widely used random probability measures, the normalized completely random measures (James (Poisson process partition calculus with applications to exchangeable models and Bayesian nonparametrics (2002) Preprint), Regazzini, Lijoi and Pr\"{u}nster (Ann. Statist. 31 (2003) 560-585), Lijoi, Mena and Pr\"{u}nster (J. Amer. Statist. Assoc. 100 (2005) 1278-1291)) and the Poisson-Dirichlet process (Pitman and Yor (Ann. Probab. 25 (1997) 855-900), Ishwaran and James (J. Amer. Statist. Assoc. 96 (2001) 161-173), Pitman (In Science and Statistics: A Festschrift for Terry Speed (2003) 1-34 IMS)), in a single construction. We describe an augmented version of the Blackwell-MacQueen P\'{o}lya urn sampling scheme (Blackwell and MacQueen (Ann. Statist. 1 (1973) 353-355)) that simplifies implementation and provide a simulation study for approximating the probabilities of partition sizes.
View PDF Chat

Filtering a Double Threshold Model With Regime Switching

2013-05-02
Robert J. Elliott , Tak Kuen Siu , John W. Lau
We introduce a new double threshold model with regime switches. New filtering equations are derived based on a reference probability approach. We also propose a new and practically useful method … We introduce a new double threshold model with regime switches. New filtering equations are derived based on a reference probability approach. We also propose a new and practically useful method for implementing the filtering equations.

Viterbi-Based Estimation for Markov Switching GARCH Model

2012-01-04
Robert J. Elliott , John W. Lau , Hong Miao +1 more
Abstract We outline a two-stage estimation method for a Markov-switching Generalized Autoregressive Conditional Heteroscedastic (GARCH) model modulated by a hidden Markov chain. The first stage involves the estimation of a … Abstract We outline a two-stage estimation method for a Markov-switching Generalized Autoregressive Conditional Heteroscedastic (GARCH) model modulated by a hidden Markov chain. The first stage involves the estimation of a hidden Markov chain using the Vitberi algorithm given the model parameters. The second stage uses the maximum likelihood method to estimate the model parameters given the estimated hidden Markov chain. Applications to financial risk management are discussed through simulated data.
View PDF Chat

Bayesian Non-Parametric Mixtures of GARCH(1,1) Models

2012-01-01
John W. Lau , Edward Cripps
Traditional GARCH models describe volatility levels that evolve smoothly over time, generated by a single GARCH regime. However, nonstationary time series data may exhibit abrupt changes in volatility, suggesting changes … Traditional GARCH models describe volatility levels that evolve smoothly over time, generated by a single GARCH regime. However, nonstationary time series data may exhibit abrupt changes in volatility, suggesting changes in the underlying GARCH regimes. Further, the number and times of regime changes are not always obvious. This article outlines a nonparametric mixture of GARCH models that is able to estimate the number and time of volatility regime changes by mixing over the Poisson-Kingman process. The process is a generalisation of the Dirichlet process typically used in nonparametric models for time-dependent data provides a richer clustering structure, and its application to time series data is novel. Inference is Bayesian, and a Markov chain Monte Carlo algorithm to explore the posterior distribution is described. The methodology is illustrated on the Standard and Poor's 500 financial index.
View PDF Chat

Bayesian nonparametric estimation and consistency of mixed multinomial logit choice models

2010-08-01
Pierpaolo De Blasi , Lancelot F. James , John W. Lau
This paper develops nonparametric estimation for discrete choice models based on the mixed multinomial logit (MMNL) model. It has been shown that MMNL models encompass all discrete choice models derived … This paper develops nonparametric estimation for discrete choice models based on the mixed multinomial logit (MMNL) model. It has been shown that MMNL models encompass all discrete choice models derived under the assumption of random utility maximization, subject to the identification of an unknown distribution $G$. Noting the mixture model description of the MMNL, we employ a Bayesian nonparametric approach, using nonparametric priors on the unknown mixing distribution $G$, to estimate choice probabilities. We provide an important theoretical support for the use of the proposed methodology by investigating consistency of the posterior distribution for a general nonparametric prior on the mixing distribution. Consistency is defined according to an $L_1$-type distance on the space of choice probabilities and is achieved by extending to a regression model framework a recent approach to strong consistency based on the summability of square roots of prior probabilities. Moving to estimation, slightly different techniques for non-panel and panel data models are discussed. For practical implementation, we describe efficient and relatively easy-to-use blocked Gibbs sampling procedures. These procedures are based on approximations of the random probability measure by classes of finite stick-breaking processes. A simulation study is also performed to investigate the performance of the proposed methods.
View PDF Chat

A Monte Carlo Markov chain algorithm for a class of mixture time series models

2009-08-28
John W. Lau , Mike K. P. So

Bayesian mixture of autoregressive models

2008-06-11
John W. Lau , Mike K. P. So

Modelling long-term investment returns via Bayesian infinite mixture time series models

2008-05-12
John W. Lau , Tak Kuen Siu
Abstract This paper introduces the class of Bayesian infinite mixture time series models first proposed in Lau & So (2004) for modelling long-term investment returns. It is a flexible class … Abstract This paper introduces the class of Bayesian infinite mixture time series models first proposed in Lau & So (2004) for modelling long-term investment returns. It is a flexible class of time series models and provides a flexible way to incorporate full information contained in all autoregressive components with various orders by utilizing the idea of Bayesian averaging or mixing. We adopt a Bayesian sampling scheme based on a weighted Chinese restaurant process for generating partitions of investment returns to estimate the Bayesian infinite mixture time series models. Instead of using the point estimates, as in the classical or non-Bayesian approach, the estimation in this paper is performed by the full Bayesian approach, utilizing the idea of Bayesian averaging to incorporate all information contained in the posterior distributions of the random parameters. This provides a natural way to incorporate model risk or uncertainty. The proposed models can also be used to perform clustering of investment returns and detect outliers of returns. We employ the monthly data from the Toronto Stock Exchange 300 (TSE 300) indices to illustrate the implementation of our models and compare the simulated results from the estimated models with the empirical characteristics of the TSE 300 data. We apply the Bayesian predictive distribution of the logarithmic returns obtained by the Bayesian averaging or mixing to evaluate the quantile-based and conditional tail expectation risk measures for segregated fund contracts via stochastic simulation. We compare the risk measures evaluated from our models with those from some well-known and important models in the literature, and highlight some features that can be obtained from our models. Keywords: Bayesian MAR modelsBayesian mixture AR-ARCH modelsWeighted Chinese restaurant processClustering of returnsOutliers detectionDirichlet prior processQuantile-based risk measuresConditional tail expectation Acknowledgements We would like to thank the referee for many helpful and valuable comments and suggestions.

Option Pricing When the Regime-Switching Risk is Priced

2007-11-01
Tak Kuen Siu , Hailiang Yang Unim , John W. Lau
Recently, there has been considerable interest in investigating option valuation problem in the context of regime-switching models. However, most of the literature consider the case that the risk due to … Recently, there has been considerable interest in investigating option valuation problem in the context of regime-switching models. However, most of the literature consider the case that the risk due to switching regimes is not priced. Relatively little attention has been paid to investigate the impact of switching regimes on the option price when this source of risk is priced. In this paper, we shall articulate this important problem and consider the pricing of an option when the price dynamic of the underlying risky asset is governed by a Markov-modulated geometric Brownian motion. We suppose that the drift and volatility of the underlying risky asset switch over time according to the state of an economy, which is modeled by a continuous-time hidden Markov chain. We shall develop a two-stage pricing model which can price both the diffusion risk and the regime-switching risk based on the Esscher transform and the minimization of the maximum entropy between an equivalent martingale measure and the real-world probability measure over different states. The latter is called a min-max entropy problem. We shall conduct numerical experiments to illustrate the effect of pricing regime-switching risk. The results of the numerical experiments reveal that the impact of pricing regime-switching risk on the option prices is significant.

Gibbs Partitions (EPPF's) Derived From a Stable Subordinator are Fox H and Meijer G Transforms

2007-01-01
Man-Wai Ho , Lancelot F. James , John W. Lau
This paper derives explicit results for the infinite Gibbs partitions generated by the jumps of an $α-$stable subordinator, derived in Pitman \cite{Pit02, Pit06}. We first show that for general $α$ … This paper derives explicit results for the infinite Gibbs partitions generated by the jumps of an $α-$stable subordinator, derived in Pitman \cite{Pit02, Pit06}. We first show that for general $α$ the conditional EPPF can be represented as ratios of Fox-$H$ functions, and in the case of rational $α,$ Meijer-G functions. Furthermore the results show that the resulting unconditional EPPF's, can be expressed in terms of H and G transforms indexed by a function h. Hence when h is itself a H or G function the EPPF is also an H or G function. An implication, in the case of rational $α,$ is that one can compute explicitly thousands of EPPF's derived from possibly exotic special functions. This would also apply to all $α$ except that computations for general Fox functions are not yet available. However, moving away from special functions, we demonstrate how results from probability theory may be used to obtain calculations. We show that a forward recursion can be applied that only requires calculation of the simplest components. Additionally we identify general classes of EPPF's where explicit calculations can be carried out using distribution theory.

On Bayesian Mixture Credibility

2006-11-01
John W. Lau , Tak Kuen Siu , Hailiang Yang
We introduce a class of Bayesian infinite mixture models first introduced by Lo (1984) to determine the credibility premium for a non-homogeneous insurance portfolio. The Bayesian infinite mixture models provide … We introduce a class of Bayesian infinite mixture models first introduced by Lo (1984) to determine the credibility premium for a non-homogeneous insurance portfolio. The Bayesian infinite mixture models provide us with much flexibility in the specification of the claim distribution. We employ the sampling scheme based on a weighted Chinese restaurant process introduced in Lo et al. (1996) to estimate a Bayesian infinite mixture model from the claim data. The Bayesian sampling scheme also provides a systematic way to cluster the claim data. This can provide some insights into the risk characteristics of the policyholders. The estimated credibility premium from the Bayesian infinite mixture model can be written as a linear combination of the prior estimate and the sample mean of the claim data. Estimation results for the Bayesian mixture credibility premiums will be presented.
View PDF Chat

On Bayesian Mixture Credibility

2006-11-01
John W. Lau , Tak Kuen Siu , Hailiang Yang
We introduce a class of Bayesian infinite mixture models first introduced by Lo (1984) to determine the credibility premium for a non-homogeneous insurance portfolio. The Bayesian infinite mixture models provide … We introduce a class of Bayesian infinite mixture models first introduced by Lo (1984) to determine the credibility premium for a non-homogeneous insurance portfolio. The Bayesian infinite mixture models provide us with much flexibility in the specification of the claim distribution. We employ the sampling scheme based on a weighted Chinese restaurant process introduced in Lo et al. (1996) to estimate a Bayesian infinite mixture model from the claim data. The Bayesian sampling scheme also provides a systematic way to cluster the claim data. This can provide some insights into the risk characteristics of the policyholders. The estimated credibility premium from the Bayesian infinite mixture model can be written as a linear combination of the prior estimate and the sample mean of the claim data. Estimation results for the Bayesian mixture credibility premiums will be presented.
View PDF Chat

Coagulation Fragmentation Laws Induced By General Coagulations of Two-Parameter Poisson-Dirichlet Processes

2006-01-01
Man-Wai Ho , Lancelot F. James , John W. Lau
Pitman~(1999) describes a duality relationship between fragmentation and coagulation operators. An explicit relationship is described for the two-parameter Poisson-Dirichlet laws, with parameters {\footnotesize $(α,θ)$} and $(β,θ/α)$, wherein $PD(α, θ)$ is … Pitman~(1999) describes a duality relationship between fragmentation and coagulation operators. An explicit relationship is described for the two-parameter Poisson-Dirichlet laws, with parameters {\footnotesize $(α,θ)$} and $(β,θ/α)$, wherein $PD(α, θ)$ is coagulated by $PD(β,θ/α)$ for $0

Bayesian semi-parametric modeling for mixed proportional hazard models with right censoring

2005-11-08
John W. Lau

A Class of Generalized Hyperbolic Continuous Time Integrated Stochastic Volatility Likelihood Models

2005-01-01
Lancelot F. James , John W. Lau
This paper discusses and analyzes a class of likelihood models which are based on two distributional innovations in financial models for stock returns. That is, the notion that the marginal … This paper discusses and analyzes a class of likelihood models which are based on two distributional innovations in financial models for stock returns. That is, the notion that the marginal distribution of aggregate returns of log-stock prices are well approximated by generalized hyperbolic distributions, and that volatility clustering can be handled by specifying the integrated volatility as a random process such as that proposed in a recent series of papers by Barndorff-Nielsen and Shephard (BNS). The BNS models produce likelihoods for aggregate returns which can be viewed as a subclass of latent regression models where one has n conditionally independent Normal random variables whose mean and variance are representable as linear functionals of a common unobserved Poisson random measure. James (2005b) recently obtains an exact analysis for such models yielding expressions of the likelihood in terms of quite tractable Fourier-Cosine integrals. Here, our idea is to analyze a class of likelihoods, which can be used for similar purposes, but where the latent regression models are based on n conditionally independent models with distributions belonging to a subclass of the generalized hyperbolic distributions and whose corresponding parameters are representable as linear functionals of a common unobserved Poisson random measure. Our models are perhaps most closely related to the Normal inverse Gaussian/GARCH/A-PARCH models of Brandorff-Nielsen (1997) and Jensen and Lunde (2001), where in our case the GARCH component is replaced by quantities such as INT-OU processes. It is seen that, importantly, such likelihood models exhibit quite different features structurally. One nice feature of the model is that it allows for more flexibility in terms of modelling of external regression parameters.
Coauthor Papers Together
Lancelot F. James 7
Tak Kuen Siu 6
Wendy Hui 5
Edward Cripps 4
Man-Wai Ho 4
Robert J. Elliott 2
Mike K. P. So 2
Hailiang Yang 2
Man Wai Ho 1
Pierpaolo De Blasi 1
Hailiang Yang Unim 1
Min Kwang Byun 1
Hong Miao 1
Sally Cripps 1

On a Class of Bayesian Nonparametric Estimates: I. Density Estimates

1984-03-01
Albert Y. Lo
Given a positive, normalized kernel and a finite measure on an Euclidean space, we construct a random density by convoluting the kernel with the Dirichlet random probability indexed by the … Given a positive, normalized kernel and a finite measure on an Euclidean space, we construct a random density by convoluting the kernel with the Dirichlet random probability indexed by the finite measure. The posterior distribution of the random density given a sample is classified. The Bayes estimator of the density function is given.
View PDF Chat

Gibbs Sampling Methods for Stick-Breaking Priors

2001-03-01
Hemant Ishwaran , Lancelot F. James
AbstractA rich and flexible class of random probability measures, which we call stick-breaking priors, can be constructed using a sequence of independent beta random variables. Examples of random measures that … AbstractA rich and flexible class of random probability measures, which we call stick-breaking priors, can be constructed using a sequence of independent beta random variables. Examples of random measures that have this characterization include the Dirichlet process, its two-parameter extension, the two-parameter Poisson–Dirichlet process, finite dimensional Dirichlet priors, and beta two-parameter processes. The rich nature of stick-breaking priors offers Bayesians a useful class of priors for nonparametric problems, while the similar construction used in each prior can be exploited to develop a general computational procedure for fitting them. In this article we present two general types of Gibbs samplers that can be used to fit posteriors of Bayesian hierarchical models based on stick-breaking priors. The first type of Gibbs sampler, referred to as a Pólya urn Gibbs sampler, is a generalized version of a widely used Gibbs sampling method currently employed for Dirichlet process computing. This method applies to stick-breaking priors with a known Pólya urn characterization, that is, priors with an explicit and simple prediction rule. Our second method, the blocked Gibbs sampler, is based on an entirely different approach that works by directly sampling values from the posterior of the random measure. The blocked Gibbs sampler can be viewed as a more general approach because it works without requiring an explicit prediction rule. We find that the blocked Gibbs avoids some of the limitations seen with the Pólya urn approach and should be simpler for nonexperts to use.KEY WORDS: Blocked Gibbs samplerDirichlet processGeneralized Dirichlet distributionPitman–Yor processPólya urn Gibbs samplerPrediction ruleRandom probability measureRandom weightsStable law

A Bayesian Analysis of Some Nonparametric Problems

1973-03-01
Thomas S. Ferguson
The Bayesian approach to statistical problems, though fruitful in many ways, has been rather unsuccessful in treating nonparametric problems. This is due primarily to the difficulty in finding workable prior … The Bayesian approach to statistical problems, though fruitful in many ways, has been rather unsuccessful in treating nonparametric problems. This is due primarily to the difficulty in finding workable prior distributions on the parameter space, which in nonparametric ploblems is taken to be a set of probability distributions on a given sample space. There are two desirable properties of a prior distribution for nonparametric problems. (I) The support of the prior distribution should be large--with respect to some suitable topology on the space of probability distributions on the sample space. (II) Posterior distributions given a sample of observations from the true probability distribution should be manageable analytically. These properties are antagonistic in the sense that one may be obtained at the expense of the other. This paper presents a class of prior distributions, called Dirichlet process priors, broad in the sense of (I), for which (II) is realized, and for which treatment of many nonparametric statistical problems may be carried out, yielding results that are comparable to the classical theory. In Section 2, we review the properties of the Dirichlet distribution needed for the description of the Dirichlet process given in Section 3. Briefly, this process may be described as follows. Let $\mathscr{X}$ be a space and $\mathscr{A}$ a $\sigma$-field of subsets, and let $\alpha$ be a finite non-null measure on $(\mathscr{X}, \mathscr{A})$. Then a stochastic process $P$ indexed by elements $A$ of $\mathscr{A}$, is said to be a Dirichlet process on $(\mathscr{X}, \mathscr{A})$ with parameter $\alpha$ if for any measurable partition $(A_1, \cdots, A_k)$ of $\mathscr{X}$, the random vector $(P(A_1), \cdots, P(A_k))$ has a Dirichlet distribution with parameter $(\alpha(A_1), \cdots, \alpha(A_k)). P$ may be considered a random probability measure on $(\mathscr{X}, \mathscr{A})$, The main theorem states that if $P$ is a Dirichlet process on $(\mathscr{X}, \mathscr{A})$ with parameter $\alpha$, and if $X_1, \cdots, X_n$ is a sample from $P$, then the posterior distribution of $P$ given $X_1, \cdots, X_n$ is also a Dirichlet process on $(\mathscr{X}, \mathscr{A})$ with a parameter $\alpha + \sum^n_1 \delta_{x_i}$, where $\delta_x$ denotes the measure giving mass one to the point $x$. In Section 4, an alternative definition of the Dirichlet process is given. This definition exhibits a version of the Dirichlet process that gives probability one to the set of discrete probability measures on $(\mathscr{X}, \mathscr{A})$. This is in contrast to Dubins and Freedman [2], whose methods for choosing a distribution function on the interval [0, 1] lead with probability one to singular continuous distributions. Methods of choosing a distribution function on [0, 1] that with probability one is absolutely continuous have been described by Kraft [7]. The general method of choosing a distribution function on [0, 1], described in Section 2 of Kraft and van Eeden [10], can of course be used to define the Dirichlet process on [0, 1]. Special mention must be made of the papers of Freedman and Fabius. Freedman [5] defines a notion of tailfree for a distribution on the set of all probability measures on a countable space $\mathscr{X}$. For a tailfree prior, posterior distribution given a sample from the true probability measure may be fairly easily computed. Fabius [3] extends the notion of tailfree to the case where $\mathscr{X}$ is the unit interval [0, 1], but it is clear his extension may be made to cover quite general $\mathscr{X}$. With such an extension, the Dirichlet process would be a special case of a tailfree distribution for which the posterior distribution has a particularly simple form. There are disadvantages to the fact that $P$ chosen by a Dirichlet process is discrete with probability one. These appear mainly because in sampling from a $P$ chosen by a Dirichlet process, we expect eventually to see one observation exactly equal to another. For example, consider the goodness-of-fit problem of testing the hypothesis $H_0$ that a distribution on the interval [0, 1] is uniform. If on the alternative hypothesis we place a Dirichlet process prior with parameter $\alpha$ itself a uniform measure on [0, 1], and if we are given a sample of size $n \geqq 2$, the only nontrivial nonrandomized Bayes rule is to reject $H_0$ if and only if two or more of the observations are exactly equal. This is really a test of the hypothesis that a distribution is continuous against the hypothesis that it is discrete. Thus, there is still a need for a prior that chooses a continuous distribution with probability one and yet satisfies properties (I) and (II). Some applications in which the possible doubling up of the values of the observations plays no essential role are presented in Section 5. These include the estimation of a distribution function, of a mean, of quantiles, of a variance and of a covariance. A two-sample problem is considered in which the Mann-Whitney statistic, equivalent to the rank-sum statistic, appears naturally. A decision theoretic upper tolerance limit for a quantile is also treated. Finally, a hypothesis testing problem concerning a quantile is shown to yield the sign test. In each of these problems, useful ways of combining prior information with the statistical observations appear. Other applications exist. In his Ph. D. dissertation [1], Charles Antoniak finds a need to consider mixtures of Dirichlet processes. He treats several problems, including the estimation of a mixing distribution, bio-assay, empirical Bayes problems, and discrimination problems.
View PDF Chat

The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator

1997-04-01
Jim Pitman , Marc Yor
The two-parameter Poisson-Dirichlet distribution, denoted $\mathsf{PD}(\alpha, \theta)$ is a probability distribution on the set of decreasing positive sequences with sum 1. The usual Poisson-Dirichlet distribution with a single parameter $\theta$, … The two-parameter Poisson-Dirichlet distribution, denoted $\mathsf{PD}(\alpha, \theta)$ is a probability distribution on the set of decreasing positive sequences with sum 1. The usual Poisson-Dirichlet distribution with a single parameter $\theta$, introduced by Kingman, is $\mathsf{PD}(0, \theta)$. Known properties of $\mathsf{PD}(0, \theta)$, including the Markov chain description due to Vershik, Shmidt and Ignatov, are generalized to the two-parameter case. The size-biased random permutation of $\mathsf{PD}(\alpha, \theta)$ is a simple residual allocation model proposed by Engen in the context of species diversity, and rediscovered by Perman and the authors in the study of excursions of Brownian motion and Bessel processes. For $0 < \alpha < 1, \mathsf{PD}(\alpha, 0)$ is the asymptotic distribution of ranked lengths of excursions of a Markov chain away from a state whose recurrence time distribution is in the domain of attraction of a stable law of index $\alpha$. Formulae in this case trace back to work of Darling, Lamperti and Wendel in the 1950s and 1960s. The distribution of ranked lengths of excursions of a one-dimensional Brownian motion is $\mathsf{PD}(1/2, 0)$, and the corresponding distribution for a Brownian bredge is $\mathsf{PD}(1/2, 1/2)$. The $\mathsf{PD}(\alpha, 0)$ and $\mathsf{PD}(\alpha, \alpha)$ distributions admit a similar interpretation in terms of the ranked lengths of excursions of a semistable Markov process whose zero set is the range of a stable subordinator of index $\alpha$.
View PDF Chat

Poisson Process Partition Calculus with applications to Exchangeable models and Bayesian Nonparametrics

2002-01-01
Lancelot F. James
This article discusses the usage of a partiton based Fubini calculus for Poisson processes. The approach is an amplification of Bayesian techniques developed in Lo and Weng for gamma/Dirichlet processes. … This article discusses the usage of a partiton based Fubini calculus for Poisson processes. The approach is an amplification of Bayesian techniques developed in Lo and Weng for gamma/Dirichlet processes. Applications to models are considered which all fall within an inhomogeneous spatial extension of the size biased framework used in Perman, Pitman and Yor. Among some of the results; an explicit partition based calculus is then developed for such models, which also includes a series of important exponential change of measure formula. These results are applied to obtain results for Levy-Cox models, identities related to the two-parameter Poisson-Dirichlet process and other processes, generalisations of the Markov-Krein correspondence, calculus for extended Neutral to the Right processes, among other things.

Random Discrete Distributions

1975-09-01
J. F. C. Kingmán
Summary It is impossible to choose at random a probability distribution on a countably infinite set in a manner invariant under permutations of that set. However, approximations to such a … Summary It is impossible to choose at random a probability distribution on a countably infinite set in a manner invariant under permutations of that set. However, approximations to such a choice can be made by considering exchangeable probability measures on the class of probability distributions over a finite set, and letting the size of that set increase without limit. Under suitable conditions the resulting probabilities, when arranged in descending order, have non-degenerate limiting distributions. These apparently arcane considerations lead to rather concrete conclusions in certain problems in applied probability.

Generalized weighted Chinese restaurant processes for species sampling mixture models

2003-10-01
Hemant Ishwaran , Lancelot F. James
The class of species sampling mixture models is introduced as an exten- sion of semiparametric models based on the Dirichlet process to models based on the general class of species … The class of species sampling mixture models is introduced as an exten- sion of semiparametric models based on the Dirichlet process to models based on the general class of species sampling priors, or equivalently the class of all exchangeable urn distributions. Using Fubini calculus in conjunction with Pitman (1995, 1996), we derive characterizations of the posterior distribution in terms of a posterior par- tition distribution that extend the results of Lo (1984) for the Dirichlet process. These results provide a better understanding of models and have both theoretical and practical applications. To facilitate the use of our models we generalize the work in Brunner, Chan, James and Lo (2001) by extending their weighted Chinese restaurant (WCR) Monte Carlo procedure, an i.i.d. sequential importance sampling (SIS) procedure for approximating posterior mean functionals based on the Dirich- let process, to the case of approximation of mean functionals and additionally their posterior laws in species sampling mixture models. We also discuss collapsed Gibbs sampling, Polya urn Gibbs sampling and a Polya urn SIS scheme. Our framework allows for numerous applications, including multiplicative counting process models subject to weighted gamma processes, as well as nonparametric and semiparamet- ric hierarchical models based on the Dirichlet process, its two-parameter extension, the Pitman-Yor process and finite dimensional Dirichlet priors.

Poisson-Kingman partitions

2003-01-01
Jim Pitman
This paper presents some general formulas for random partitions of a finite set derived by Kingman's model of random sampling from an interval partition generated by subintervals whose lengths are … This paper presents some general formulas for random partitions of a finite set derived by Kingman's model of random sampling from an interval partition generated by subintervals whose lengths are the points of a Poisson point process.These lengths can be also interpreted as the jumps of a subordinator, that is an increasing process with stationary independent increments.Examples include the two-parameter family of Poisson-Dirichlet models derived from the Poisson process of jumps of a stable subordinator.Applications are made to the random partition generated by the lengths of excursions of a Brownian motion or Brownian bridge conditioned on its local time at zero.
View PDF Chat

Bayesian Poisson process partition calculus with an application to Bayesian Lévy moving averages

2005-08-01
Lancelot F. James
This article develops, and describes how to use, results concerning disintegrations of Poisson random measures. These results are fashioned as simple tools that can be tailor-made to address inferential questions … This article develops, and describes how to use, results concerning disintegrations of Poisson random measures. These results are fashioned as simple tools that can be tailor-made to address inferential questions arising in a wide range of Bayesian nonparametric and spatial statistical models. The Poisson disintegration method is based on the formal statement of two results concerning a Laplace functional change of measure and a Poisson Palm/Fubini calculus in terms of random partitions of the integers {1,…,n}. The techniques are analogous to, but much more general than, techniques for the Dirichlet process and weighted gamma process developed in [Ann. Statist. 12 (1984) 351–357] and [Ann. Inst. Statist. Math. 41 (1989) 227–245]. In order to illustrate the flexibility of the approach, large classes of random probability measures and random hazards or intensities which can be expressed as functionals of Poisson random measures are described. We describe a unified posterior analysis of classes of discrete random probability which identifies and exploits features common to all these models. The analysis circumvents many of the difficult issues involved in Bayesian nonparametric calculus, including a combinatorial component. This allows one to focus on the unique features of each process which are characterized via real valued functions h. The applicability of the technique is further illustrated by obtaining explicit posterior expressions for Lévy–Cox moving average processes within the general setting of multiplicative intensity models. In addition, novel computational procedures, similar to efficient procedures developed for the Dirichlet process, are briefly discussed for these models.
View PDF Chat

Ferguson Distributions Via Polya Urn Schemes

1973-03-01
David Blackwell , James B. MacQueen
The Polya urn scheme is extended by allowing a continuum of colors. For the extended scheme, the distribution of colors after $n$ draws is shown to converge as $n \rightarrow … The Polya urn scheme is extended by allowing a continuum of colors. For the extended scheme, the distribution of colors after $n$ draws is shown to converge as $n \rightarrow \infty$ to a limiting discrete distribution $\mu^\ast$. The distribution of $\mu^\ast$ is shown to be one introduced by Ferguson and, given $\mu^\ast$, the colors drawn from the urn are shown to be independent with distribution $\mu^\ast$.

Estimating normal means with a conjugate style dirichlet process prior

1994-01-01
Steven N. MacEachern
Abstract The problem of estimating many normal means is approached by means of an hierarchical model. The hierarchical model is the standard conjugate model with one exception: the normal distribution … Abstract The problem of estimating many normal means is approached by means of an hierarchical model. The hierarchical model is the standard conjugate model with one exception: the normal distribution at the middle stage is replaced by a Dirichlet process with a normal shape. Estimation for this model is accomplished through the implementation of the Gibbs sampler (see Escobar and West,1991)Thisarticle describes a new Gibbs sampler algorithm that is implemented on a collapsed state space Results that apply to a general setting are obtained, suggesting that a collapse of the state space willimprove the rate of convergence of the Gibbs sampler. An example shows that the proposed collapse of the state space may result in a dramatically improved algorithm Keywords: Gibbs samplerhierarchical modelMarkov chainnonparametric Bayes

Computational Methods for Multiplicative Intensity Models Using Weighted Gamma Processes

2004-03-01
Hemant Ishwaran , Lancelot F. James
We develop computational procedures for a class of Bayesian nonparametric and semiparametric multiplicative intensity models incorporating kernel mixtures of spatial weighted gamma measures. A key feature of our approach is … We develop computational procedures for a class of Bayesian nonparametric and semiparametric multiplicative intensity models incorporating kernel mixtures of spatial weighted gamma measures. A key feature of our approach is that explicit expressions for posterior distributions of these models share many common structural features with the posterior distributions of Bayesian hierarchical models using the Dirichlet process. Using this fact, along with an approximation for the weighted gamma process, we show that with some care, one can adapt efficient algorithms used for the Dirichlet process to this setting. We discuss blocked Gibbs sampling procedures and Pólya urn Gibbs samplers. We illustrate our methods with applications to proportional hazard models, Poisson spatial regression models, recurrent events, and panel count data.

Markov Chain Sampling Methods for Dirichlet Process Mixture Models

2000-06-01
Radford M. Neal
Abstract This article reviews Markov chain methods for sampling from the posterior distribution of a Dirichlet process mixture model and presents two new classes of methods. One new approach is … Abstract This article reviews Markov chain methods for sampling from the posterior distribution of a Dirichlet process mixture model and presents two new classes of methods. One new approach is to make Metropolis—Hastings updates of the indicators specifying which mixture component is associated with each observation, perhaps supplemented with a partial form of Gibbs sampling. The other new approach extends Gibbs sampling for these indicators by using a set of auxiliary parameters. These methods are simple to implement and are more efficient than previous ways of handling general Dirichlet process mixture models with non-conjugate priors.

Some developments of the Blackwell-MacQueen urn scheme

1996-01-01
Jim Pitman
The Blackwell-MacQueen description of sampling from a Dirichlet random distribution on an abstract space is reviewed, and extended to a general family of random discrete distributions.Results are obtained by application … The Blackwell-MacQueen description of sampling from a Dirichlet random distribution on an abstract space is reviewed, and extended to a general family of random discrete distributions.Results are obtained by application of Kingman's theory of partition structures.
View PDF Chat

Posterior Analysis for Normalized Random Measures with Independent Increments

2008-07-24
Lancelot F. James , Antonio Lijoi , Igor Prünster
Abstract. One of the main research areas in Bayesian Nonparametrics is the proposal and study of priors which generalize the Dirichlet process. In this paper, we provide a comprehensive Bayesian … Abstract. One of the main research areas in Bayesian Nonparametrics is the proposal and study of priors which generalize the Dirichlet process. In this paper, we provide a comprehensive Bayesian non‐parametric analysis of random probabilities which are obtained by normalizing random measures with independent increments (NRMI). Special cases of these priors have already shown to be useful for statistical applications such as mixture models and species sampling problems. However, in order to fully exploit these priors, the derivation of the posterior distribution of NRMIs is crucial: here we achieve this goal and, indeed, provide explicit and tractable expressions suitable for practical implementation. The posterior distribution of an NRMI turns out to be a mixture with respect to the distribution of a specific latent variable. The analysis is completed by the derivation of the corresponding predictive distributions and by a thorough investigation of the marginal structure. These results allow to derive a generalized Blackwell–MacQueen sampling scheme, which is then adapted to cover also mixture models driven by general NRMIs.

On a class of Bayesian nonparametric estimates: II. Hazard rate estimates

1989-06-01
Albert Y. Lo , Chung‐Sing Weng

Combinatorial Stochastic Processes

2006-01-01
Jim Pitman , Jean Picard
View PDF Chat

Distributional results for means of normalized random measures with independent increments

2003-04-01
Eugenio Regazzini , Antonio Lijoi , Igor Prünster
We consider the problem of determining the distribution of means of random probability measures which are obtained by normalizing increasing additive processes. A solution is found by resorting to a … We consider the problem of determining the distribution of means of random probability measures which are obtained by normalizing increasing additive processes. A solution is found by resorting to a well-known inversion formula for characteristic functions due to Gurland. Moreover, expressions of the posterior distributions of those means, in the presence of exchangeable observations, are given. Finally, a section is devoted to the illustration of two examples of statistical relevance.
View PDF Chat

Exchangeable and partially exchangeable random partitions

1995-06-01
Jim Pitman
View PDF Chat

Some further developments for stick-breaking priors: Finite and infinite clustering and classification

2003-01-01
Hemant Ishwaran , Lancelot F. James

On Subordinators, Self-Similar Markov Processes and Some Factorizations of the Exponential Variable

2001-01-01
Jean Bertoin , Marc Yor
Let $\xi$ be a subordinator with Laplace exponent $\Phi$, $I=\int_{0}^{\infty}\exp(-\xi_s)ds$ the so-called exponential functional, and $X$ (respectively, $\hat X$) the self-similar Markov process obtained from $\xi$ (respectively, from $\hat{\xi}=-\xi$) by … Let $\xi$ be a subordinator with Laplace exponent $\Phi$, $I=\int_{0}^{\infty}\exp(-\xi_s)ds$ the so-called exponential functional, and $X$ (respectively, $\hat X$) the self-similar Markov process obtained from $\xi$ (respectively, from $\hat{\xi}=-\xi$) by Lamperti's transformation. We establish the existence of a unique probability measure $\rho$ on $]0,\infty[$ with $k$-th moment given for every $k\in N$ by the product $\Phi(1)\cdots\Phi(k)$, and which bears some remarkable connections with the preceding variables. In particular we show that if $R$ is an independent random variable with law $\rho$ then $IR$ is a standard exponential variable, that the function $t\to E(1/X_t)$ coincides with the Laplace transform of $\rho$, and that $\rho$ is the $1$-invariant distribution of the sub-markovian process $\hat X$. A number of known factorizations of an exponential variable are shown to be of the preceding form $IR$ for various subordinators $\xi$.
View PDF Chat

Size-biased sampling of Poisson point processes and excursions

1992-03-01
Mihael Perman , Jim Pitman , Marc Yor
View PDF Chat

On a Mixture Autoregressive Conditional Heteroscedastic Model

2001-09-01
Chun Shan Wong , Wai Keung Li
AbstractWe propose a mixture autoregressive conditional heteroscedastic (MAR-ARCH) model for modeling nonlinear time series. The models consist of a mixture of K autoregressive components with autoregressive conditional heteroscedasticity; that is, … AbstractWe propose a mixture autoregressive conditional heteroscedastic (MAR-ARCH) model for modeling nonlinear time series. The models consist of a mixture of K autoregressive components with autoregressive conditional heteroscedasticity; that is, the conditional mean of the process variable follows a mixture AR (MAR) process, whereas the conditional variance of the process variable follows a mixture ARCH process. In addition to the advantage of better description of the conditional distributions from the MAR model, the MARARCH model allows a more flexible squared autocorrelation structure. The stationarity conditions, autocorrelation function, and squared autocorrelation function are derived. Construction of multiple step predictive distributions is discussed. The estimation can be easily done through a simple EM algorithm, and the model selection problem is addressed. The shape-changing feature of the conditional distributions makes these models capable of modeling time series with multimodal conditional distributions and with heteroscedasticity. The models are applied to two real datasets and compared to other competing models. The MAR-ARCH models appear to capture features of the data better than the competing models.KEY WORDS: AutocorrelationEM algorithmModel selectionPredictive distributionsStationarity

Approximate Dirichlet Process Computing in Finite Normal Mixtures

2002-09-01
Hemant Ishwaran , Lancelot F. James
AbstractA rich nonparametric analysis of the finite normal mixture model is obtained by working with a precise truncation approximation of the Dirichlet process. Model fitting is carried out by a … AbstractA rich nonparametric analysis of the finite normal mixture model is obtained by working with a precise truncation approximation of the Dirichlet process. Model fitting is carried out by a simple Gibbs sampling algorithm that directly samples the nonparametric posterior. The proposed sampler mixes well, requires no tuning parameters, and involves only draws from simple distributions, including the draw for the mass parameter that controls clustering, and the draw for the variances with the use of a nonconjugate uniform prior. Working directly with the nonparametric prior is conceptually appealing and among other things leads to graphical methods for studying the posterior mixing distribution as well as penalized MLE procedures for deriving point estimates. We discuss methods for automating selection of priors for the mean and variance components to avoid over or undersmoothing the data. We also look at the effectiveness of incorporating prior information in the form of frequentist point estimates.Key Words: Almost sure truncationBlocked gibbs samplerNonparametric hierarchical modelPenalized MLEPolya urn gibbs samplingRandom probability measure

Lamperti-type laws

2010-07-20
Lancelot F. James
This paper explores various distributional aspects of random variables defined as the ratio of two independent positive random variables where one variable has an α-stable law, for 0 < α … This paper explores various distributional aspects of random variables defined as the ratio of two independent positive random variables where one variable has an α-stable law, for 0 < α < 1, and the other variable has the law defined by polynomially tilting the density of an α-stable random variable by a factor θ > −α. When θ = 0, these variables equate with the ratio investigated by Lamperti [Trans. Amer. Math. Soc. 88 (1958) 380–387] which, remarkably, was shown to have a simple density. This variable arises in a variety of areas and gains importance from a close connection to the stable laws. This rationale, and connection to the PD (α, θ) distribution, motivates the investigations of its generalizations which we refer to as Lamperti-type laws. We identify and exploit links to random variables that commonly appear in a variety of applications. Namely Linnik, generalized Pareto and z-distributions. In each case we obtain new results that are of potential interest. As some highlights, we then use these results to (i) obtain integral representations and other identities for a class of generalized Mittag–Leffler functions, (ii) identify explicitly the Lévy density of the semigroup of stable continuous state branching processes (CSBP) and hence corresponding limiting distributions derived in Slack and in Zolotarev [Z. Wahrsch. Verw. Gebiete 9 (1968) 139–145, Teor. Veroyatn. Primen. 2 (1957) 256–266], which are related to the recent work by Berestycki, Berestycki and Schweinsberg, and Bertoin and LeGall [Ann. Inst. H. Poincaré Probab. Statist. 44 (2008) 214–238, Illinois J. Math. 50 (2006) 147–181] on beta coalescents. (iii) We obtain explicit results for the occupation time of generalized Bessel bridges and some interesting stochastic equations for PD (α, θ)-bridges. In particular we obtain the best known results for the density of the time spent positive of a Bessel bridge of dimension 2 − 2α.
View PDF Chat

Estimating Mixture of Dirichlet Process Models

1998-06-01
Steven N. MacEachern , Peter Müller
Abstract Current Gibbs sampling schemes in mixture of Dirichlet process (MDP) models are restricted to using “conjugate” base measures that allow analytic evaluation of the transition probabilities when resampling configurations, … Abstract Current Gibbs sampling schemes in mixture of Dirichlet process (MDP) models are restricted to using “conjugate” base measures that allow analytic evaluation of the transition probabilities when resampling configurations, or alternatively need to rely on approximate numeric evaluations of some transition probabilities. Implementation of Gibbs sampling in more general MDP models is an open and important problem because most applications call for the use of nonconjugate base measures. In this article we propose a conceptual framework for computational strategies. This framework provides a perspective on current methods, facilitates comparisons between them, and leads to several new methods that expand the scope of MDP models to nonconjugate situations. We discuss one in detail. The basic strategy is based on expanding the parameter vector, and is applicable for MDP models with arbitrary base measure and likelihood. Strategies are also presented for the important class of normal-normal MDP models and for problems with fixed or few hyperparameters. The proposed algorithms are easily implemented and illustrated with an application.

Hierarchical Mixture Modeling With Normalized Inverse-Gaussian Priors

2005-11-06
Antonio Lijoi , Ramsés H. Mena , Igor Prünster
In recent years the Dirichlet process prior has experienced a great success in the context of Bayesian mixture modeling. The idea of overcoming discreteness of its realizations by exploiting it … In recent years the Dirichlet process prior has experienced a great success in the context of Bayesian mixture modeling. The idea of overcoming discreteness of its realizations by exploiting it in hierarchical models, combined with the development of suitable sampling techniques, represent one of the reasons of its popularity. In this article we propose the normalized inverse-Gaussian (N–IG) process as an alternative to the Dirichlet process to be used in Bayesian hierarchical models. The N–IG prior is constructed via its finite-dimensional distributions. This prior, although sharing the discreteness property of the Dirichlet prior, is characterized by a more elaborate and sensible clustering which makes use of all the information contained in the data. Whereas in the Dirichlet case the mass assigned to each observation depends solely on the number of times that it occurred, for the N–IG prior the weight of a single observation depends heavily on the whole number of ties in the sample. Moreover, expressions corresponding to relevant statistical quantities, such as a priori moments and the predictive distributions, are as tractable as those arising from the Dirichlet process. This implies that well-established sampling schemes can be easily extended to cover hierarchical models based on the N–IG process. The mixture of N–IG process and the mixture of Dirichlet process are compared using two examples involving mixtures of normals.

A Bayesian Nonparametric Approach to Reliability

1981-03-01
Richard L. Dykstra , Purushottam W. Laud
It is suggested that problems in a reliability context may be handled by a Bayesian nonparametric approach. A stochastic process is defined whose sample paths may be assumed to be … It is suggested that problems in a reliability context may be handled by a Bayesian nonparametric approach. A stochastic process is defined whose sample paths may be assumed to be increasing hazard rates by properly choosing the parameter functions of the process. The posterior distribution of the hazard rates is derived for both exact and censored data. Bayes estimates of hazard rates and $\operatorname{cdf's}$ are found under squared error type loss functions. Some simulation is done and estimates graphed to better understand the estimators. Finally, estimates of the hazard rate from some data in a paper by Kaplan and Meier are constructed.
View PDF Chat

On a Mixture Autoregressive Model

2000-01-01
Chun Shan Wong , W. K. Li
Summary We generalize the Gaussian mixture transition distribution (GMTD) model introduced by Le and co-workers to the mixture autoregressive (MAR) model for the modelling of non-linear time series. The models … Summary We generalize the Gaussian mixture transition distribution (GMTD) model introduced by Le and co-workers to the mixture autoregressive (MAR) model for the modelling of non-linear time series. The models consist of a mixture of K stationary or non-stationary AR components. The advantages of the MAR model over the GMTD model include a more full range of shape changing predictive distributions and the ability to handle cycles and conditional heteroscedasticity in the time series. The stationarity conditions and autocorrelation function are derived. The estimation is easily done via a simple EM algorithm and the model selection problem is addressed. The shape changing feature of the conditional distributions makes these models capable of modelling time series with multimodal conditional distributions and with heteroscedasticity. The models are applied to two real data sets and compared with other competing models. The MAR models appear to capture features of the data better than other competing models do.

Exchangeability and related topics

1985-01-01
David Aldous

Generalized Gamma measures and shot-noise Cox processes

1999-12-01
Anders Brix
A parametric family of completely random measures, which includes gamma random measures, positive stable random measures as well as inverse Gaussian measures, is defined. In order to develop models for … A parametric family of completely random measures, which includes gamma random measures, positive stable random measures as well as inverse Gaussian measures, is defined. In order to develop models for clustered point patterns with dependencies between points, the family is used in a shot-noise construction as intensity measures for Cox processes. The resulting Cox processes are of Poisson cluster process type and include Poisson processes and ordinary Neyman-Scott processes. We show characteristics of the completely random measures, illustrated by simulations, and derive moment and mixing properties for the shot-noise random measures. Finally statistical inference for shot-noise Cox processes is considered and some results on nearest-neighbour Markov properties are given.

Markov chain Monte Carlo in approximate Dirichlet and beta two-parameter process hierarchical models

2000-06-01
Hemant Ishwaran , Mahmoud Zarepour
Journal Article Markov chain Monte Carlo in approximate Dirichlet and beta two-parameter process hierarchical models Get access H Ishwaran, H Ishwaran Search for other works by this author on: Oxford … Journal Article Markov chain Monte Carlo in approximate Dirichlet and beta two-parameter process hierarchical models Get access H Ishwaran, H Ishwaran Search for other works by this author on: Oxford Academic Google Scholar M Zarepour M Zarepour Search for other works by this author on: Oxford Academic Google Scholar Biometrika, Volume 87, Issue 2, June 2000, Pages 371–390, https://doi.org/10.1093/biomet/87.2.371 Published: 01 June 2000

Exchangeable Gibbs partitions and Stirling triangles

2006-09-11
Alexander Gnedin , Jim Pitman
View PDF Chat

Controlling the Reinforcement in Bayesian Non-Parametric Mixture Models

2007-08-07
Antonio Lijoi , Ramsés H. Mena , Igor Prünster
Summary The paper deals with the problem of determining the number of components in a mixture model. We take a Bayesian non-parametric approach and adopt a hierarchical model with a … Summary The paper deals with the problem of determining the number of components in a mixture model. We take a Bayesian non-parametric approach and adopt a hierarchical model with a suitable non-parametric prior for the latent structure. A commonly used model for such a problem is the mixture of Dirichlet process model. Here, we replace the Dirichlet process with a more general non-parametric prior obtained from a generalized gamma process. The basic feature of this model is that it yields a partition structure for the latent variables which is of Gibbs type. This relates to the well-known (exchangeable) product partition models. If compared with the usual mixture of Dirichlet process model the advantage of the generalization that we are examining relies on the availability of an additional parameter σ belonging to the interval (0,1): it is shown that such a parameter greatly influences the clustering behaviour of the model. A value of σ that is close to 1 generates a large number of clusters, most of which are of small size. Then, a reinforcement mechanism which is driven by σ acts on the mass allocation by penalizing clusters of small size and favouring those few groups containing a large number of elements. These features turn out to be very useful in the context of mixture modelling. Since it is difficult to specify a priori the reinforcement rate, it is reasonable to specify a prior for σ. Hence, the strength of the reinforcement mechanism is controlled by the data.
View PDF Chat

Bayesian mixture of autoregressive models

2008-06-11
John W. Lau , Mike K. P. So

Exercises in probability. A guided tour from measure theory to random processes, via conditioning.

2003-01-01
L. Chaumont , Marc Yor
1. Measure theory and probability 2. Independence and conditioning 3. Gaussian variables 4. Distributional computations 5. Convergence of random variables 6. Random processes. 1. Measure theory and probability 2. Independence and conditioning 3. Gaussian variables 4. Distributional computations 5. Convergence of random variables 6. Random processes.

Une extension multidimensionnelle de la loi de l'arc sinus

1989-01-01
Martin T. Barlow , Jim Pitman , Marc Yor
View PDF Chat

A note on generating random variables with log-concave densities

2012-02-21
Luc Devroye

Computing Nonparametric Hierarchical Models

1998-01-01
Michael Escobar , Mike West

Two Coalescents Derived from the Ranges of Stable Subordinators

2000-01-01
Jean Bertoin , Jim Pitman
Let $M_\alpha$ be the closure of the range of a stable subordinator of index $\alpha\in ]0,1[$. There are two natural constructions of the $M_{\alpha}$'s simultaneously for all $\alpha\in ]0,1[$, so … Let $M_\alpha$ be the closure of the range of a stable subordinator of index $\alpha\in ]0,1[$. There are two natural constructions of the $M_{\alpha}$'s simultaneously for all $\alpha\in ]0,1[$, so that $M_{\alpha}\subseteq M_{\beta}$ for $0 \lt \alpha \lt \beta \lt 1$: one based on the intersection of independent regenerative sets and one based on Bochner's subordination. We compare the corresponding two coalescent processes defined by the lengths of complementary intervals of $[0,1]\backslash M_{1-\rho}$ for $0 \lt \rho \lt 1$. In particular, we identify the coalescent based on the subordination scheme with the coalescent recently introduced by Bolthausen and Sznitman.
View PDF Chat

Partition Structures Derived from Brownian Motion and Stable Subordinators

1997-03-01
Jim Pitman
Explicit formulae are obtained for the distribution of various random partitions of a positive integer n, both ordered and unordered, derived from the zero set M of a Brownian motion … Explicit formulae are obtained for the distribution of various random partitions of a positive integer n, both ordered and unordered, derived from the zero set M of a Brownian motion by the following scheme: pick n points uniformly at random from [0, 1 ], and classify them by whether they fall in the same or different component intervals of the complement of M. Corresponding results are obtained for M the range of a stable subordinator and for bridges defined by conditioning on 1 E M. These formulae are related to discrete renewal theory by a general method of discretizing a subordinator using the points of an independent homogeneous Poisson process.
View PDF Chat

Modelling Heterogeneity With and Without the Dirichlet Process

2001-06-01
Peter J. Green , Sylvia Richardson
We investigate the relationships between Dirichlet process (DP) based models and allocation models for a variable number of components, based on exchangeable distributions. It is shown that the DP partition … We investigate the relationships between Dirichlet process (DP) based models and allocation models for a variable number of components, based on exchangeable distributions. It is shown that the DP partition distribution is a limiting case of a Dirichlet–multinomial allocation model. Comparisons of posterior performance of DP and allocation models are made in the Bayesian paradigm and illustrated in the context of univariate mixture models. It is shown in particular that the unbalancedness of the allocation distribution, present in the prior DP model, persists a posteriori . Exploiting the model connections, a new MCMC sampler for general DP based models is introduced, which uses split/merge moves in a reversible jump framework. Performance of this new sampler relative to that of some traditional samplers for DP processes is then explored.
View PDF Chat

Estimating Normal Means with a Dirichlet Process Prior

1994-03-01
Michael Escobar
Abstract In this article, the Dirichlet process prior is used to provide a nonparametric Bayesian estimate of a vector of normal means. In the past there have been computational difficulties … Abstract In this article, the Dirichlet process prior is used to provide a nonparametric Bayesian estimate of a vector of normal means. In the past there have been computational difficulties with this model. This article solves the computational difficulties by developing a "Gibbs sampler" algorithm. The estimator developed in this article is then compared to parametric empirical Bayes estimators (PEB) and nonparametric empirical Bayes estimators (NPEB) in a Monte Carlo study. The Monte Carlo study demonstrates that in some conditions the PEB is better than the NPEB and in other conditions the NPEB is better than the PEB. The Monte Carlo study also shows that the estimator developed in this article produces estimates that are about as good as the PEB when the PEB is better and produces estimates that are as good as the NPEB estimator when that method is better.

Survival models for heterogeneous populations derived from stable distributions

1986-01-01
Philip Hougaard
Journal Article Survival models for heterogeneous populations derived from stable distributions Get access PHILIP HOUGAARD PHILIP HOUGAARD Biostatistical Department, Novo Research InstituteNovo Allé, DK-2880 Bagsvœrd, Denmark Search for other works … Journal Article Survival models for heterogeneous populations derived from stable distributions Get access PHILIP HOUGAARD PHILIP HOUGAARD Biostatistical Department, Novo Research InstituteNovo Allé, DK-2880 Bagsvœrd, Denmark Search for other works by this author on: Oxford Academic Google Scholar Biometrika, Volume 73, Issue 2, August 1986, Pages 387–396, https://doi.org/10.1093/biomet/73.2.387 Published: 01 August 1986 Article history Received: 01 October 1984 Revision received: 01 December 1985 Published: 01 August 1986

A Monte Carlo Markov chain algorithm for a class of mixture time series models

2009-08-28
John W. Lau , Mike K. P. So

An Introduction to the Theory of Point Processes

2007-12-28
D. J. Daley , D. Vere-Jones
View PDF Chat

Record Indices and Age-Ordered Frequencies in Exchangeable Gibbs Partitions

2007-01-01
Robert Griffiths , Dario Spanò
The frequencies of an exchangeable Gibbs random partition of the integers (Gnedin and Pitman 2005) are considered in their age-order, i.e. their size-biased order. We study their dependence on the … The frequencies of an exchangeable Gibbs random partition of the integers (Gnedin and Pitman 2005) are considered in their age-order, i.e. their size-biased order. We study their dependence on the sequence of record indices (i.e. the least elements) of the blocks of the partition. In particular we show that, conditionally on the record indices, the distribution of the age-ordered frequencies has a left-neutral stick-breaking structure. Such a property in fact characterizes the Gibbs family among all exchangeable partitions and leads to further interesting results on: (i) the conditional Mellin transform of the $k$-th oldest frequency given the $k$-th record index, and (ii) the conditional distribution of the first $k$ normalized frequencies, given their sum and the $k$-th record index; the latter turns out to be a mixture of Dirichlet distributions. Many of the mentioned representations are extensions of Griffiths and Lessard (2005) results on Ewens' partitions.
View PDF Chat

On a logistic mixture autoregressive model

2001-10-01
Corine Sau Man Wong , Wanxin Li
Journal Article On a logistic mixture autoregressive model Get access C. S. Wong, C. S. Wong Search for other works by this author on: Oxford Academic Google Scholar W. K. … Journal Article On a logistic mixture autoregressive model Get access C. S. Wong, C. S. Wong Search for other works by this author on: Oxford Academic Google Scholar W. K. Li W. K. Li Search for other works by this author on: Oxford Academic Google Scholar Biometrika, Volume 88, Issue 3, 1 October 2001, Pages 833–846, https://doi.org/10.1093/biomet/88.3.833 Published: 01 October 2001

Spinal partitions and invariance under re-rooting of continuum random trees

2009-07-01
Bénédicte Haas , Jim Pitman , Matthias Winkel
We develop some theory of spinal decompositions of discrete and continuous fragmentation trees. Specifically, we consider a coarse and a fine spinal integer partition derived from spinal tree decompositions. We … We develop some theory of spinal decompositions of discrete and continuous fragmentation trees. Specifically, we consider a coarse and a fine spinal integer partition derived from spinal tree decompositions. We prove that for a two-parameter Poisson–Dirichlet family of continuous fragmentation trees, including the stable trees of Duquesne and Le Gall, the fine partition is obtained from the coarse one by shattering each of its parts independently, according to the same law. As a second application of spinal decompositions, we prove that among the continuous fragmentation trees, stable trees are the only ones whose distribution is invariant under uniform re-rooting.
View PDF Chat

On simulation and properties of the stable law

2014-03-11
Luc Devroye , Lancelot F. James