Statistical Machine Learning:

Fundamental Limits and Applications of Reinforcement Learning (Single and Multi-Agent):


Reinforcement Learning is about taking suitable action to maximize reward in a particular situation. It is employed by various software and machines to find the best possible behavior or path it should take in a specific situation. Finding near-optimal algorithms for taking actions in reinforcement learning in different settings, and their applications in real world problems is the focus of our research. The different type of problems that we have considered include (i) Combinatorial bandits problem where the regret bounds are provided with low space-time complexity algorithm, and the approach has been applied to social networking applications. (ii) State in reinforcement learning is known in many applications after a delay, due to communication and other bottlenecks, and efficient algorithms are essential. (iii) Efficient algorithms for multiple competing agents to maximize their rewards are essential since most applications have competing agents. These works have multiple applications in social networks, transportation, cloud computing, video streaming. The figure alongside depicts the improved performance of proposed strategy, DeepPool, for ride-sharing. The number of customers accepted are higher for same number of vehicles used and ride-sharing improves the costs, travel times, and number of customers served. 

Fundamental Limits on Subspace Clustering and Data Completion:


tensor_matrix.jpg

Tensors are generalizations of vectors and matrices; a vector is a first-order tensor and a matrix is a second-order tensor. Most of the data around us are better represented with multiple orders to capture the correlations across different attributes. For example, a color image can be considered as a third-order tensor, two of the dimensions (rows and columns) being spatial, and the third being spectral (color), while a color video sequence can be considered as an order four tensor, time being the fourth dimension besides spatial and spectral. Similarly, a colored 3-D MRI image across time can be considered as an order five tensor. Exploiting additional structure leads to better embedding algorithms for subspace analysis and the elements needed for data completion (as shown alongside). 

Applications of Exploiting Data Structure in Matrix/Tensor for Better Sampling, Fingerprinting, and Estimation:


ajay.jpg

map_area.jpg

Exploiting structure in the data across multiple dimensions can lead to efficient techniques for efficient sampling and fingerprinting. The figure alongside shows different network service atributes dependence on time, spatial locations and data service quality measurements thus motivating tensor structure. Further, the map shows the received signal strength in indoor has spatial dependence and the signals from different APs are correlated and thus adaptive sampling can reduce fingerprints needed for localization.