Information Theoretic Learning Renyi's Entropy and Kernel Perspectiv
This book presents the first cohesive treatment of Information Theoretic Learning (ITL) algorithms to adapt linear or nonlinear learning machines both in supervised or unsupervised paradigms. ITL is a framework where the conventional concepts of second or
- PDF / 11,440,798 Bytes
- 538 Pages / 439.37 x 666.142 pts Page_size
- 71 Downloads / 217 Views
For other titles published in this series, go to http://www.springer.com/series/3816
Jos´e C. Principe
Information Theoretic Learning Renyi’s Entropy and Kernel Perspectives
123
Jos´e C. Principe University of Florida Dept. Electrical Engineering & Biomedical Engineering Gainesville FL 32611 NEB 451, Bldg. 33 USA
Series Editors Michael Jordan Division of Computer Science and Department of Statistics University of California, Berkeley Berkeley, CA 94720 USA
Bernhard Sch¨olkopf Max Planck Institute for Biological Cybernetics Spemannstrasse 38 72076 T¨ubingen Germany
Robert Nowak Department of Electrical and Computer Engineering University of Wisconsin-Madison 3627 Engineering Hall 1415 Engineering Drive Madison, WI 53706
ISSN 1613-9011 ISBN 978-1-4419-1569-6 e-ISBN 978-1-4419-1570-2 DOI 10.1007/978-1-4419-1570-2 Springer New York Dordrecht Heidelberg London Library of Congress Control Number: 2010924811 c Springer Science+Business Media, LLC 2010 All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)
To the women in my life Natercia Joana Leonor Mariana Isabel
V
Preface
This book is an outgrowth of ten years of research at the University of Florida Computational NeuroEngineering Laboratory (CNEL) in the general area of statistical signal processing and machine learning. One of the goals of writing the book is exactly to bridge the two fields that share so many common problems and techniques but are not yet effectively collaborating. Unlike other books that cover the state of the art in a given field, this book cuts across engineering (signal processing) and statistics (machine learning) with a common theme: learning seen from the point of view of information theory with an emphasis on Renyi’s definition of information. The basic approach is to utilize the information theory descriptors of entropy and divergence as nonparametric cost functions for the design of adaptive systems in unsupervised or supervised training modes. Hence the title: Information-Theoretic Learning (ITL). In the course of these studies, we discovered that the main idea enabling a synergistic view as well as algorithmic implementations, does not involve the conventional central moments of the data (mean and covariance). Rather, the core concept is the α-norm of the PDF, in part
Data Loading...
