CIS & MINDS Seminar - Martin Takac

<p>Recorded Seminar:</p><p><a href="https://wse.zoom.us/rec/share/Av2aEUREJfyXzdEx_C1vCoTLZgknRpHUCpHgYqp7IA... Zoom Meeting:</p><p><a href="https://wse.zoom.us/j/93822965644?pwd=dDNHYVZGY096QU9Dem45STBsQWQ2dz09">... Takac, PhD</b><b> </b></p><p>Associate Professor</p><p>Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)</p><p><b>“FLECS: A Federated Learning Second-Order Framework viaCompression and Sketching”</b></p><p><b>Abstract:  </b><span>Inspired by the recent work FedNL(Safaryan et al, FedNL: Making Newton-Type Methods Applicable to FederatedLearning), we propose a new communication efficient second-order framework forFederated learning, namely FLECS. The proposed method reduces the high-memoryrequirements of FedNL by the usage of an L-SR1 type update for the Hessianapproximation which is stored on the central server. A low dimensional `sketch'of the Hessian is all that is needed by each device to generate an update, sothat memory costs as well as number of Hessian-vector products for the agentare low. Biased and unbiased compressions are utilized to make communicationcosts also low. Convergence guarantees for FLECS are provided in both thestrongly convex, and nonconvex cases, and local linear convergence is alsoestablished under strong convexity. Numerical experiments confirm the practicalbenefits of this new FLECS algorithm.</span></p><p><b> </b></p><p></p><p><b>Biography:  </b>Martin Takac is an Associate Professor at Mohamed bin ZayedUniversity of Artificial Intelligence (MBZUAI), UAE. Before joining MBZUAI, hewas an Associate Professor in the Department of Industrial and SystemsEngineering at Lehigh University, where he has been employed since 2014. Hereceived his B.S. (2008) and M.S. (2010) degrees in Mathematics from ComeniusUniversity, Slovakia, and Ph.D. (2014) degree in Mathematics from TheUniversity of Edinburgh, United Kingdom. His current research interests includethe design and analysis of algorithms for machine learning, applications of ML,optimization, HPC. Martin received funding from various U.S. National ScienceFoundation programs, including through a TRIPODS Institute grant awarded to himand his collaborators at Lehigh, Northwestern, and Boston University. He servedas an Associate Editor for Mathematical Programming Computation, Journal ofOptimization Theory and Applications, and Optimization Methods and Software andis an area chair at machine learning conferences like ICML, NeurIPS, andAISTATS.  </p>