User Tools

Site Tools


machine_learning

Machine Learning

概念

流派

派别 起源 擅长算法 代表人物
符号主义 (Symbolists) 逻辑学、哲学 逆演绎算法 (Inverse deduction) , Decision Tree Tom Mitchell, Steve Muggleton, Ross Quinlan
联结主义 (Connectionists)神经科学反向传播算法(Backpropagation) Yann LeCun, Geoff Hinton, Yoshua Bengio
进化主义(Evolutionaries)进化生物学基因编程(Genetic programming) John Koza, John Holland, Hod Lipson
贝叶斯派(Bayesians)统计学概率推理(Probabilistic inference) David Heckerman, Judea Pearl, Michael Jordan
Analogizer心理学核机器(Kernel machines), KNN Peter Hart, Vladimir Vapnik, Douglas Hofstadter

Problems

Supervised learning

Classification

predicating categories

Regression Analysis

Regression Analysis: predicating values

  • Linear Regression 线性回归
  • Logistic Regression 逻辑回归
  • Ordinary Least Squares Regression (OLSR) 普通最小二乘回归
  • Stepwise Regression 逐步回归
  • Ridge Regression: L2 regularization
  • Lasso Regression: Least Absolute Shrinkage and Selection Operator, L1 regularization
  • Locally Estimated Scatterplot Smoothing (LOESS) 本地散点平滑估计
  • Multivariate Adaptive Regression Splines (MARS) 多元自适应回归样条
  • Regularization
    • Ridge Regression 岭回归
    • Least Absolute Shrinkage and Selection Operator (LASSO) 最小绝对收缩与选择算子
    • GLASSO
    • Elastic Net 弹性网络
    • Least Angle Regression (LARS) 最小角回归

Unsupervised learning

Clustering

Discovering structures

  • Partitioning Methods/Centroid-based clustering
    • K-Means k-means clustering
      • K-means++
      • Fuzzy c-means
    • k-Medoids k-medians clustering
      • Partitioning Around Medoids (PAM) algorithm
      • CLARA (Clustering LARge Applications)
        • CLARANS (Clustering Large Applications based upon RANdomized Search)
  • Hierarchical clustering/Connectivity-based clustering (also called hierarchical cluster analysis or HCA)
    • agglomerative clustering
      • AGNES (AGglomerative NESting)
      • SLINK
      • CLINK[
      • Chameleon (): Multiphase Hierarchical Clustering Using Dynamic Modeling
    • DIANA (DIvisive ANAlysis)
      • BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies): Multiphase Hierarchical Clustering Using Clustering Feature Trees
    • Probabilistic Hierarchical Clustering
  • Density-based clustering
    • DBSCAN: Density-Based Clustering Based on Connected Regions with High Density
      • EnDBSCAN
    • OPTICS: Ordering Points to Identify the Clustering Structure
    • DENCLUE: Clustering Based on Density Distribution Functions
    • EM clustering
    • Mean-shift
  • Grid-Based Methods
    • STING: STatistical INformation Grid
    • CLIQUE (CLustering In QUEst): An Apriori-like Subspace Clustering Method
  • 图团体检测(Graph Community Detection)
  • mixture models

Anomaly detection

finding unusual data points

latent variable models

  • Method of moments
  • Blind signal separation techniques
    • Principal component analysis
    • Independent component analysis
    • Non-negative matrix factorization
    • Singular value decomposition

Neural networks

  • Hebbian learning
  • Generative adversarial networks

Semi-Supervised Learning

Structured Predication

Reinforcement Learning

Learning to rank

Active Learning

Association rules

* Apriori * Eclat algorithm * FP-Tree

Anomaly detection

  • Anomaly detection
    • K-nearest_neighbors_classification
    • Local outlier factor

Feature engineering

  • Feature engineering

Online learning

  • Online learning

Dimensionality Reduction

  • Canonical correlation analysis
  • Factor analysis
  • Flexible Discriminant Analysis (FDA) 灵活判别分析
  • Independent component analysis
  • Linear Discriminant Analysis (LDA) 线性判别分析
  • Multidimensional Scaling (MDS) 多维尺度变换
  • Mixture Discriminant Analysis (MDA) 混合判别分析
  • Non-negative matrix factorization (NMF)
  • Partial Least Squares Regression (PLSR) 偏最小二乘回归
  • Principal Component Analysis (PCA) 主成分分析
  • Principal Component Regression (PCR) 主成分回归
  • Quadratic Discriminant Analysis (QDA) 二次判别分析
  • Sammon Mapping Sammon 映射
  • T-distributed stochastic neighbor embedding

Theory

损失函数(Cost Function)

图谱

会议,期刊

Tools

  • mlpack is a C++ machine learning library.
  • PLearn is a C++ library aimed at research and development in the field of statistical machine learning algorithms. Its originality is to allow to easily express, directly in C++ in a straightforward manner, complex non-linear functions to be optimized.
  • Waffles- C++ Machine Learning。
  • Torch7 provides a Matlab-like environment for state-of-the-art machine learning algorithms. It is easy to use and provides a very efficient implementation
  • SHARK is a modular C++ library for the design and optimization of adaptive systems. It provides methods for linear and nonlinear optimization, in particular evolutionary and gradient-based algorithms, kernel-based learning algorithms and neural networks, and various other machine learning techniques. SHARK serves as a toolbox to support real world applications as well as research in different domains of computational intelligence and machine learning. The sources are compatible with the following platforms: Windows, Solaris, MacOS X, and Linux.
  • Dlib-ml is an open source library, targetedat both engineers and research scientists, which aims to provide a similarly rich environment fordeveloping machine learning software in the C++ language.
  • Eblearn is an object-oriented C++ library that implements various machine learning models, including energy-based learning, gradient-based learning for machine composed of multiple heterogeneous modules. In particular, the library provides a complete set of tools for building, training, and running convolutional networks.
  • Edward:融合了贝叶斯、深度学习和概率编程
  • Machine Learning Open Source Software :Journal of Machine Learning Research: http://jmlr.csail.mit.edu/mloss/.
  • ELF: ensemble learning framework。特点:c++,监督学习,使用了intel的IPP和MKL,training speed 和accuracy是主要目标。http://elf-project.sourceforge.net/

Code

Books

Mathematics

  • Calculus. Strang, tiny
  • Calculus. Spivak, Principles of Mathematical Analysis a.k.a “Baby Rudin”, heavy book
  • Discrete Math (ALADM above + a good book on Algorithms, Cormen will do [*]
  • Linear Algebra (First work through Strang's book, then Axler's)
  • Probability (see Bradford's very comprehensive reccomendations) and
  • Statistics (I would recommend Devore and Peck for the total beginner but it is a damn expensive book. So hit a library or get a bootlegged copy to see if it suits you before buying a copy, see brad's list for advanced stuff.)
  • Learning About Statistical Learning
  • Introduction to Algorithms by Charles E. Leiserson
  • Introduction to Linear Algebra, Fourth Edition by Gilbert Strang
  • Introduction to Probability, 2nd Edition by Dimitri P. Bertsekas
  • A Course in Probability Theory, Revised Edition, Second Edition by Kai Lai Chung
  • First Look at Rigorous Probability Theory by Jeffrey S. Rosenthal
  • All of Statistics: A Concise Course in Statistical Inference (Springer Texts in Statistics) by Larry Wasserman
  • Machine Learning (Mcgraw-Hill International Edit) by Tom M. Mitchell
  • The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition (Springer Series in Statistics) by Robert Tibshirani et al.
  • Introduction to Stochastic Search and Optimization by James C. Spall
  • Introduction to Analysis by Maxwell Rosenlicht
  • How to Prove It: A Structured Approach by Daniel J. Velleman
  • Learning about Machine Learning from Pin Dancing by Ravi
  • Velleman's “How to Prove It”(which has more of a math-ey feel)
  • Gries and Schneider's “A Logical Approach to Discrete Math”
  • Casella, G. and Berger, R.L. (2001). “Statistical Inference” Duxbury Press.
  • Ferguson, T. (1996). “A Course in Large Sample Theory” Chapman & Hall/CRC. You'll need to learn something about asymptotics at some point, and a good starting place is:
  • Lehmann, E. (2004). “Elements of Large-Sample Theory” Springer.
  • Gelman, A. et al. (2003). “Bayesian Data Analysis” Chapman & Hall/CRC.
  • Robert, C. and Casella, G. (2005). “Monte Carlo Statistical Methods” Springer.
  • Grimmett, G. and Stirzaker, D. (2001). “Probability and Random Processes” Oxford.
  • Pollard, D. (2001). “A User's Guide to Measure Theoretic Probability” Cambridge.
  • Durrett, R. (2005). “Probability: Theory and Examples” Duxbury.
  • Bertsimas, D. and Tsitsiklis, J. (1997). “Introduction to Linear Optimization” Athena.
  • Boyd, S. and Vandenberghe, L. (2004). “Convex Optimization” Cambridge.
  • Golub, G., and Van Loan, C. (1996). “Matrix Computations” Johns Hopkins.
  • Cover, T. and Thomas, J. “Elements of Information Theory” Wiley.
  • Kreyszig, E. (1989). “Introductory Functional Analysis with Applications” Wiley.

Science popularizing

  • The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World 译本:终极算法:机器学习和人工智能如何重塑世界
  • Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die 译本:大数据预测
  • The Signal and the Noise: Why So Many Predictions Fail–but Some Don't
  • Naked Statistics: Stripping the Dread from the Data
  • The Drunkard's Walk: How Randomness Rules Our Lives

Machine Learning

Neural Network

  • Neural Network Design Hagan Demuth and Beale,
  • Neural Networks, A Comprehensive Foundation (2nd edition) - By Haykin (there is a newer edition out but I don't know anything about that, this is the one I used)
  • Neural Networks for Pattern Recognition ( Bishop)

Courses

2016

2015

Others

  • Andrew Ng's online Machine Learning course
  • Mining Massive Datasets
  • Recommender Systems
  • Machine Learning Summer School

Links

Tutorials

machine_learning.txt · Last modified: 2019/04/22 13:45 by x