The fusion of multiple features is important for recognizing actions, since a single feature based representation is not enough to capture imaging variations (view-point, illumination etc.) and attributes of individuals (size, age, gender etc.). We propose to use two types of features: The first feature is the quantized vocabulary of local spatio-temporal (ST) volumes (or cuboids) that are centered around 3D interest points in the video. The second feature is a quantized vocabulary of spin-images, which is aimed at capturing the 3D shape of the actor by considering actions as 3D objects. To optimally combine these features, we develop a mathematical framework that treats different features as nodes in a graph, where weighted edges between the nodes represent the strength of the relationship between entities. The graph is then embedded into a k-dimensional space, subject to the criteria that similar nodes have Euclidian coordinates which are closer to each other. This is achieved by converting this constraint into a minimization problem whose solution is the eigenvectors of the graph Laplacian matrix. The embedding into a common space allows the discovery of relationships among features by using Euclidian distances. The performance of the proposed framework is tested on publicly available data sets. The results demonstrate that fusion of multiple features help in achieving improved performance.

Generating Multiple Features: