Small-Sample Comparison of the Gamma Kernel and the Orthogonal Series Methods of Density Estimation

Introduction

Estimation of a probability density function is an important area of nonparametric statistical inference that has received much attention in recent decades. The kernel method is widely used in nonparametric estimation of the probability density function of an absolutely continuous distribution with support on the whole real line. However, for a distribution with support on a subset of the real line, the kernel density estimator with fixed symmetric kernels encounters bias at the boundaries of the support, which is known as the boundary bias issue. This is due to smoothing data near the boundary points by the fixed symmetric kernel that leads to allocating probability density to outside of the distribution’s support (see Silverman, 1986).

There are many applications, such as reliability, insurance and life testing, dealing with non-negative data and estimating the probability density function of distributions with support on the non-negative real line is the object of interest. Using the kernel estimator with fixed symmetric kernels in these cases results in the boundary bias issue at the origin. A number of methods have been proposed to avoid the boundary bias issue at the origin. A simple remedy is to replace symmetric kernels by asymmetric kernels which never assign density to negative values. The Gamma kernels proposed by Chen (2000) are the effective asymmetric kernels to estimate the probability density function of distributions on the non-negative real line.

Orthogonal series estimators form another class of nonparametric probability density estimators, which go back to Cencov (1964). In this approach, as reviewed in Efromovich (2010), the target probability function is expanded in terms of a sequence of orthogonal basis functions. After selecting a suitable sequence of orthogonal basis functions, the observed data are used to estimated the coefficients of the expansion in order to obtain the orthogonal series density estimator. Similar to kernel estimators, under some mild conditions the orthogonal series estimators have appealing large sample properties. Moreover, the boundary issue can be avoided by using orthogonal density estimators with suitable basis functions.

Although small sample properties of asymmetric kernel estimators with the Gamma kernels and orthogonal series estimators are well-studied separately, but to the best of our knowledge, there have been no reports of comparing their performance in estimating the probability density function of distributions on the non-negative real line. In this paper‎, a simulation study is conducted to compare the small-sample performance of the Gamma kernel estimators and orthogonal series estimators for a set of distributions on the positive real line.



Material and methods

Following Malec and Schienle (2014), we consider six parameter settings for the generalized F distribution to obtain probability density functions with different shapes, near-origin behaviors and tail decays (Figure 2). Based on 5000 simulations from any of these density functions with sample sizes , we estimate the target density function using the type I and II Gamma kernel estimators and the orthogonal series estimators with Hermite and Laguerre basis functions and compute the mean integrated squared error (MISE).

The bandwidth parameter in the Gamma kernel estimators and the cutoff and smoothing parameters in the orthogonal series estimators are significantly affect the performance of the estimators. We use optimal bandwidths for the Gamma kernel estimators and optimal cutoff and smoothing parameters of the orthogonal series estimators to avoid variations due to uncertainty of tuning parameters. To obtain the optimal tuning parameters for each target density, we compute and minimize the MISE with respect to the tuning parameters based on additional 5000 simulations from the true density function.

Results and discussion

For each density function, the optimal tuning parameters and the MISE’s of the estimators are reported (Table 2). As expected from the large sample properties, increasing the sample size improves the performance of all estimators. The performances of estimators vary from cases to cases and there no considered estimator is the best in all cases. In all cases except one, the type II Gamma kernel estimator is superior to the type I Gamma kernel estimator, which is in agreement with Chen (2000) suggestion of preferring type II to type I Gamma kernel estimator. However, in one case the type I Gamma kernel estimator is better than all other estimators. In cases where the shape and near-origin behavior of the target density is similar to the Hermit or Laguerre basis functions, the corresponding orthogonal series estimator outperforms all the other competing estimators.

Conclusion

The following conclusions were drawn from this research to choose among the considered estimators.

If the basis functions of the orthogonal series estimator are chosen to have similar shape and near-origin behavior to the target density function, then the corresponding orthogonal series estimator can outperform the Gamma kernel estimators.
If there is no prior knowledge about the shape and near-origin behavior of the target density function and the sample size is relatively large (n=400), then the type II Gamma kernel estimator can outperform the orthogonal series estimators.