Independent Component Analysis Theory and Applications
Independent Component Analysis (ICA) is a signal-processing method to extract independent sources given only observed data that are mixtures of the unknown sources. Recently, blind source separation by ICA has received considerable attention because of it
- PDF / 19,002,226 Bytes
- 218 Pages / 437.907 x 666.581 pts Page_size
- 1 Downloads / 303 Views
INDEPENDENT COMPONENT ANALYSIS THEORY AND APPLICATIONS
by
TE-WON LEE
Computational Neurobiology Laboratory The Salk Institute, La Jolla, California
SPRINGER-SCIENCE+BUSINESS MEDIA, B.v.
A C.I.P. Catalogue record for this book is available from the Library of Congress.
ISBN 978-1-4419-5056-7 ISBN 978-1-4757-2851-4 (eBook) DOI 10.1007/978-1-4757-2851-4
Printed on acid-free paper
Ali Rights Reserved
© 1998 Springer Science+Business Media Dordrecht Originally published by Kluwer Academic Publishers in 1998 Softcover reprint of the hardcover 1st edition 1998 No part of the material protected by this copyright notice may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording or by any information storage and retrieval system, without written permission from the copyright owner.
This book is dedicated to my parents Jeong-Bog Lee and Sun-Ja Kang, my sister Soon-Hie and my brother Yu-Won.
Contents
Abstract Preface Acknowledgments
xi xiii xvii
List of Tables
xix xxiii
Abbreviations and Symbols
xxv
List of Figures
Introd uction
Part I
Independent Component Analysis: Theory
1. BASICS 1.1 1.2 1.3
1.4
1.5
1.6
xxix
Overview Bayesian Probability Theory Information Theory 1.3.1 Differential Entropy 1.3.2 Maximum Entropy
5 5 6 7 10 11
Artificial Neural Networks 1.4.1 Neural networks using unsupervised learning rules 1.4.2 The Principle of Maximum Entropy Preservation
13 14 18
1.5.1 1.5.2 1.5.3
Higher-Order Statistics Moments Cumulants Cross-cumulants
21 21 23 23
Summary
24
2. INDEPENDENT COMPONENT ANALYSIS
27
2.1
Overview
27
2.2
Problem statement and assumptions
29
2.3
The Poverty of PCA
31
vii
viii
ICA THEORY AND APPLICATIONS
2.4
The Information Maximization Approach to ICA
35
2.5
Derivation of the Infomax Learning Rule for ICA
37
2.6
A simple but general ICA learning rule 2.6.1 Deriving the extended infomax learning rule to separate sub- and superGaussian sources 2.6.2 Switching between nonlinearities 2.6.3 The hyperbolic-Cauchy density model
42
2.7
Simulations 2.7.1 10 Mixed Sound Sources 2.7.2 20 Mixed Sound Sources
49 49 52
2.8
Convergence properties in blind source separation 2.8.1 An intuition for the natural gradient 2.8.2 Robustness to parameter mismatch
56 56 58
2.9
Discussions 2.9.1 Comparison to other algorithms and architectures 2.9.2 Applications to real world problems 2.9.3 Biological plausibility 2.9.4 Limitations and future research 2.9.5 Conclusions
62 62 62 63 63 64
3. A UNIFYING INFORMATION-THEORETIC FRAMEWORK FOR ICA
67
4.
43 47 47
3.1
Overview
67
3.2
Information Maximization
68
3.3
Negentropy Maximization
3.4
Maximum Likelihood Estimation
69 72 73 76 78 80
3.5
Higher-order moments and cumulants
3.6 3.7
Nonlinear PCA Bussgang Algorithms
3.8
Conclusion
BLIND SEPARATION OF TIME-DELAYED AND CONVOLVED SOURCES
83
4.1
Overview
83
4.2
Problem statement and assumptions
85
4.3
Feedback Architecture 4.3.1 Learning Rules 4.3.2 Simulations
86 87 89
4.4
Feedforw
Data Loading...
