Reinforcement learning and approximate dynamic programming,

Learning control,

Robotics and intelligent vehicles,

Machine learning and data mining

(1) Principle Investigator (PI): Program for New Centry Excellent Talent in University (NCET-10-0901): "Intelligent Learning Systems and Applications" , 2011-2013

(2) Principle Investigator (PI): National Natural Science Foundation of China (NSFC: 61075072): "Hierarchical reinforcement learning and its applications in motion planning of virtual humans" , 2011-2013

(3) Principle Investigator (PI):  National Natural Science Foundation of China (NSFC: 60774076): "Kernel-based reinforcement learning and approximate dynamic programming" , 2008-2010

(4) Principle Investigator (PI):  National Natural Science Foundation of China (NSFC: 60303012): "Adaptive intrusion detection based on reinforcement learning", 2004-2006

(5) Principle Investigator (PI): Natural Science Foundation of Hunan Province "Distributed reinforcement learning with applications on autonomic computing", 2008-2010

(6) Principle Investigator (PI): The Fork Ying Tong Youth Teacher Foundation "Research on new mechanisms on network computing", 2008-2011

(7) Co-Investigator (Co-I): Key project of National Natural Science Foundation of China (NSFC: 90820302): Research on Scientific Problems for Intelligent Driving of High-way Vehicles, 2009-2012

(8) Co-Investigator (Co-I): National Natural Science Foundation of China (NSFC: 60234030): Navigation and control of mobile robots in unknown environments. 2003-2006

(1) FastAHC:  Learning control with RLS-TD(lamda) and adaptive heuristic critic, matlab code with a cart-pole example [download]

   For detailed discussions, please refer to [Xu, et al., 2002] Efficient reinforcement learning using recursive least-squares methods. Jounral of AI Research, 2002, 16, 259-292

(2) KLSPI: Learning control with kernel-based least-squares policy iteration (KLSPI), matlab code with an inverted pendulum example

[download]

    For detailed discussions, please refer to [Xu, et al., 2007] Kernel-based least squares policy iteration for reinforcement learning. IEEE Transactions on Neural Networks, 2007, 18(4) 973-992

(3) Visual Saliency Based on Scale-Space Analysis in the Frequency Domain.

    For detailed discussions, please refer to Jian Li, Martin D. Levine, Xiangjing An, Xin Xu and Hangen He. Visual Saliency Based on Scale-Space Analysis in the Frequency Domain. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(4): 996-1010. (regular paper)

More software on RL and Pattern Recognition research will appear soon...