Title
Coded Computing: How to Utilize Fast Workers in Heterogenous Systems?
Abstract
The widespread use of machine learning and big data has led to a surge in demand for computational power. To efficiently train learning models, computation tasks are often offloaded to distributed servers, or workers, which can compute in parallel to speed up the process. However, the workers may not be trustworthy, which raises data privacy concerns. Moreover, they may suffer from straggling or outages, causing delays or even complete failures. Researchers have adopted coding techniques originally designed for error correction to address these challenges to ensure data privacy and enhance system resilience against slow or unresponsive workers.
This seminar will be divided into two parts. The first part will introduce the concept of coded computing and highlight the challenge of dealing with heterogeneous worker speeds, particularly in edge computing scenarios where no prior knowledge of worker speeds is available. The selection of workers is formulated as a multi-armed bandit problem, solved by a reinforcement learning method called Thompson sampling. In the second part, the complementary scenario of multi-cloud computing will be considered, where there is complete knowledge of worker speeds. The fundamental tradeoff between storage and computing under security constraints is analyzed via information theory, and the impact of heterogeneity is characterized via majorization theory. The talk will then conclude by discussing future directions in this emerging field. Attendees will gain insights into the latest advances in coded computing and its potential for improving the efficiency and resilience of distributed learning systems.
Biographies
Albert Sung is an Associate Professor and the Associate Head of Undergraduate Programmes in the Department of Electrical Engineering at City University of Hong Kong. He holds a B.Eng, M.Phil, and Ph.D, all in Information Engineering, from the Chinese University of Hong Kong. His research interests span broad areas of communications, coding, and computing, with a specific focus on resource allocation for mobile networks, code constructions for distributed storage systems, and the design and analysis of evolutionary algorithms. In recognition of his contribution to the field, he has been included in the list of top 2% most highly cited scientists complied by Stanford University since 2020. He also serves on the editorial boards of the ETRI Journal and Electronics Letters.
Jiajun Chen is a Ph.D. student in the Department of Electrical Engineering at City University of Hong Kong. She received the B.S. degree from Sichuan University, Chengdu, China, in 2018. She has a journal paper published in IEEE Transactions on Information Theory (TIT) in 2023. Her research interests include coded distributed computing, information theory, and resource allocation.
Jonathan handles all external affairs include business development, patents write up and public relations. He is frequently interviewed by media and is considered a pioneer in 3D printing products.
After graduating from OUHK, Krutz obtained an M.Sc. in Engineering Management from CityU. He is now completing his second master degree, M.Sc. in Biomedical Engineering, at CUHK. Krutz has a wide range of working experience. He has been with Siemens, VTech, and PCCW.
Hugo Leung Wai-yin, who graduated from his four-year programme in 2015, won the Best Paper Award for his ‘intelligent pill-dispenser’ design at the Institute of Electrical and Electronics Engineering’s International Conference on Consumer Electronics – China 2015.
The pill-dispenser alerts patients via sound and LED flashes to pre-set dosage and time intervals. Unlike units currently on the market, Hugo’s design connects to any mobile phone globally. In explaining how it works, he said: ‘There are three layers in the portable pillbox. The lowest level is a controller with various devices which can be connected to mobile phones in remote locations. Patients are alerted by a sound alarm and flashes. Should they fail to follow their prescribed regime, data can be sent via SMS to relatives and friends for follow up.’ The pill-dispenser has four medicine slots, plus a back-up with a LED alert, topped by a 500ml water bottle. It took Hugo three months of research and coding to complete his design, but he feels it was worth all his time and effort.
Hugo’s public examination results were disappointing and he was at a loss about his future before enrolling at the OUHK, which he now realizes was a major turning point in his life. He is grateful for the OUHK’s learning environment, its industry links and the positive guidance and encouragement from his teachers. The University is now exploring the commercial potential of his design with a pharmaceutical company. He hopes that this will benefit the elderly and chronically ill, as well as the society at large.
Soon after completing his studies, Hugo joined an automation technology company as an assistant engineer. He is responsible for the design and development of automation devices. The target is to minimize human labor and increase the quality of products. He is developing products which are used in various sections, including healthcare, manufacturing and consumer electronics.
| Course Code | Title | Credits | |
|---|---|---|---|
| COMP S321F | Advanced Database and Data Warehousing | 5 | |
| COMP S333F | Advanced Programming and AI Algorithms | 5 | |
| COMP S351F | Software Project Management | 5 | |
| COMP S362F | Concurrent and Network Programming | 5 | |
| COMP S363F | Distributed Systems and Parallel Computing | 5 | |
| COMP S382F | Data Mining and Analytics | 5 | |
| COMP S390F | Creative Programming for Games | 5 | |
| COMP S492F | Machine Learning | 5 | |
| ELEC S305F | Computer Networking | 5 | |
| ELEC S348F | IOT Security | 5 | |
| ELEC S371F | Digital Forensics | 5 | |
| ELEC S431F | Blockchain Technologies | 5 | |
| ELEC S425F | Computer and Network Security | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| ELEC S201F | Basic Electronics | 5 | |
| IT S290F | Human Computer Interaction & User Experience Design | 5 | |
| STAT S251F | Statistical Data Analysis | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| COMPS333F | Advanced Programming and AI Algorithms | 5 | |
| COMPS362F | Concurrent and Network Programming | 5 | |
| COMPS363F | Distributed Systems and Parallel Computing | 5 | |
| COMPS380F | Web Applications: Design and Development | 5 | |
| COMPS381F | Server-side Technologies and Cloud Computing | 5 | |
| COMPS382F | Data Mining and Analytics | 5 | |
| COMPS390F | Creative Programming for Games | 5 | |
| COMPS413F | Application Design and Development for Mobile Devices | 5 | |
| COMPS492F | Machine Learning | 5 | |
| ELECS305F | Computer Networking | 5 | |
| ELECS363F | Advanced Computer Design | 5 | |
| ELECS425F | Computer and Network Security | 5 |
