رویکرد جدید برای اطلاع فیشر

نویسنده
دانشگاه کاشان
چکیده
اطلاع فیشر معیاری برای اندازه گیری اطلاعات نهفته در متغیر تصادفی راجع به پارامتر مجهول است. اطلاع متقابل وابستگی بین دو متغیر و آنتروپی نسبی تفاوت بین دو توزیع احتمال را نشان می دهد. در این مقاله اطلاع متقابل و آنتروپی نسبی برای اطلاع فیشر تعمیم داده می شود و خواص یکنوایی اطلاع فیشر مورد بررسی قرار می گیرد. سپس مفاهیمی چون همبستگی اطلاع و ضریب همبستگی اطلاع معرفی می شوند. در پایان نشان داده می شود که آنتروپی دیفرانسیل شانون که متغیر تصادفی را به صورت کمی اندازه گیری می کند و اطلاع فیشر شرطی برای تعیین احتمال خطای برآورد مورد استفاده قرار می گیرند.
کلیدواژه‌ها

عنوان مقاله English

New Approach to Fisher Information

نویسنده English

Mehdi Shams
University of Kashan
چکیده English

Fisher information is a measure of the information inside a random variable about an unknown parameter. Mutual information shows the dependence between two variables and relative entropy shows the difference between the two probability distributions. In this paper, Fisher information is generalized for mutual information and relative entropy and the monotonicity properties of Fisher information are examined. Then concepts such as information correlation and information correlation coefficient are introduced. Finally, it is shown that Shannon differential entropy, which measures the behavior of a random variable, and conditional Fisher information are used to determine the probability of estimation error.

کلیدواژه‌ها English

Shannon differential entropy
Fisher information
Markov chains
CramerRao inequality
Basu, D. ( 1955), On Statistics Independent of a Complete Sufficient Statistic, Sankhya, 15, 377–380.

Bercher, J. F. (2013), Some properties of generalized Fisher information in the context of nonextensive thermostatistics, Physica A: Statistical Mechanics and its

Applications, 392, 3140–3154.

Bobkov, S. G., Chistyakov, G. P. and Gotze, F. (2014), Fisher information and the central limit theorem, Probab. Theory Relat. Fields, 159, 1–59.

Boekee, D. E. (1974), Generalized Fisher information with application to estimation problems, IFAC Proceedings, 10, 75–82.

Bouchon, B. and Pessoa, F. (1985), Logarithmic information of degree q linked with an extension of Fisher’s information, Kybernetika, 21, 346–359.

Cianchi, A., Lutwak, E., Yang, D. and Zhang, G. (2013), A unified approach to Cramér–Rao inequalities, IEEE Trans. Inf. Theory, 60, 643–650.

Cohen, M. (1968), The Fisher Information and Convexity, IEEE Trans. Inf. Theory, 14, 591–592.

Cover, T. and Thomas, J. (2006), Elements of Information Theory, John Wiley and Sons, Inc.: Hoboken, NJ, USA.

Fisher, R. (1925), Theory of Statistical Estimation, Proc. Camb. Philos. Soc. 22, 700–725.

Flego, S., Olivares, F., Plastino, A. and Casas, M. (2011), Extreme Fisher Information, Non-Equilibrium Thermodynamics and Reciprocity Relations, Entropy, 13, 184–194.

Frieden, B.R. (2004), Science from Fisher Information: A Unification, Cambridge University Press: Cambridge, UK.

Frieden, B.R. and Gatenby, R.A. (2013), Principle of maximum Fisher information from Hardy’s axioms applied to statistical systems, Phys. Rev. E, 88, 1–6.

Frieden, B.R. and Petri, M. (2012), Motion-dependent levels of order in a relativistic universe, Phys. Rev. E, 86, 1–5.

Frieden, B.R., Plastino, A., Plastino, A.R. and Soffer, B.H. (1999), Fisher-Based Thermodynamics: Its Legendre Transform and Concavity Properties, Phys. Rev. E, 60, 48–53.

Frieden, B.R., Plastino, A., Plastino, A.R. and Soffer, B.H. (2002), Non-equilibrium thermodynamics and Fisher information: An illustrative example, Phys. Lett. A, 304, 73–78.

Guo, D. (2009), Relative Entropy and Score Function: New Information-Estimation Relationships through Arbitrary Additive Perturbation, In Proceedings of the IEEE International Symposium on Information Theory, Seoul, Korea, 28 June–3 July 2009; 814–818.

Hirata, M., Nemoto, A. and Yoshida, H. (2012), An integral representation of the relative entropy, Entropy, 14, 1469–1477.

Hogg, R.V. and Craig, A.T. (1995), Introduction to Mathematical Statistics, Prentice Hall: Upper Saddle River, NJ, USA.

Kullback, S. (1968), Information Theory and Statistics, Dover Publications Inc.: Mineola, NY, USA.

Kullback, S. and Leibler, R.A. (1951), On Information and Sufficiency, Ann. Math. Stat., 22, 79–86.

Langley, R. S. (2013), Probability Functionals for Self-Consistent and Invariant Inference: Entropy and Fisher Information, IEEE Trans. Inf. Theory, 59, 4397–4407.

Lehmann, E. L. and Casella, G. (1998), Theory of Point Estimation, 2nd Edition, Springer-Verlage, New York.

Luo, S. (2002), Maximum Shannon entropy, minimum Fisher information, and an elementary game, Found. Phys, 32, 1757–1772.

Lv, S. (2017), General Fisher information matrices of a random vector, Advances in Applied Mathematics, 89, 18–40.

Ly, A., Marsman, M., Verhagen, J., Grasman, R. P. and Wagenmakers, E.J. (2002), A tutorial on Fisher information, Journal of Mathematical Psychology, 80, 40–55.

Otto, F. and Villani, C. (2000), Generalization of an Inequality by Talagrand and Links with the Logarithmic Sobolev Inequality, J. Funct. Anal., 173, 361–400.

Park, C. and Basu, A. (2003), The generalized Kullback-Leibler divergence and robust inference, Journal of Statistical Computation and Simulation, 73, 311–332.

Pascanu, R. and Bengio, Y. (2014), Revisiting Natural Gradient for Deep Networks, Cornell University Library: Ithaca, NY, USA, 1–18.

Plastino, A.R. and Plastino, A. (1996), Symmetries of the Fokke-rPlanck equation and the Fisher-Frieden arrow of time, Phys. Rev. E, 54, 4423–4426.

SánchezMoreno, P., Zarzo, A. and Dehesa, J.S. (2012), Jensen divergence based on Fisher’s information, J. Phys. A: Math. Theor, 45, 125–305.

Shannon, C. A. (1948), Mathematical Theory of Communication, Bell Syst. Tech. J., 27, 379–423.

Tsallis, (1998), C. Generalized entropy-based criterion for consistent testing, Phys. Rev. , 58, 1442–1445.

Venkatesan, R.C. and Plastino, A. (2014), Legendre transform structure and extremal properties of the relative Fisher information, Phys. Lett. A, 378, 1341–1345.

Zamir, R. A. ( 1998), Proof of the Fisher Information Inequality Via a Data Processing Argument, IEEE Trans. Inf. Theory, 44, 1246–1250.

Zegers, P., Fuentes, A. and Alarcon, C. (2013), Relative Entropy Derivative Bounds, Entropy, 15, 2861–2873.