Engineer Intern Summer

*

Ayush Sarkar

University of Illinois at Urbana-Champaign

Email: *******@********.***

LinkedIn Profile: https://www.linkedin.com/in/ayush-sarkar-033071126 Phone: 650-***-**** (Mobile)

• CAREER SUMMARY

Pursuing Ph.D. degree in Computer Science at the University of Illinois, Urbana-Champaign, with a focus on artificial intelligence, distributed systems, multimedia, networking, security, and smart contracts in blockchain. Passionate about solving real-world problems in various industry verticals involving AI/ML, AR/VR applications on distributed systems from edge to cloud.

• LANGUAGES, SYSTEMS, TOOLS

Python, C/C++, Java, Scala, JavaScript, JSON, SQL, MATLAB, CUDA Programming, x86 Assembly; Ubuntu Linux, Windows; Docker, Git, MongoDB, MySQL, Artificial Intelligence (core concepts and applications: search, classification, reasoning, machine learning, games, planning, robotics, computer vision, and natural language processing), multimedia systems, Tesseract, Pytesseract, Keras, TensorFlow, OpenCV, Blockchain smart contracts.

• EXPERIENCE

Department of Computer Science, University of Illinois at Urbana Champaign, IL CS Department Fellow, Ph.D. Program, August 2022 – Present

Working on Deep Generative Models for AI under the supervision of Professor Svetlana Lazebnik. Applying attention mechanisms to diffusion models, with the aim of improving model architecture by trading off diversity for fidelity. Additionally, investigating neural radiance fields, looking at angles through which attention models can improve performance and quality of volumetric rendition functions.

Coordinated Science Laboratory (CSL), University of Illinois at Urbana Champaign, IL Graduate Research Assistant, August 2021 – May 2022

Worked on an NSF project involving augmented 360 videos for situational awareness in firefighting under the supervision of Prof. Klara Nahrstedt, Director of CSL and Professor of CS Department. Implementation involved stitching 2D videos followed by developing tools for action and object detection of 2D and 360 stitched videos. Overall process involved streaming the views with detected objects to VR/AR devices connected to the helmets of firefighters, improving situational awareness in training environments. Wrote a paper for the CPS-ER 2022 Workshop (Cyber Physical Systems for Emergency Response, co-located with CPS-IOT Week 2022) - “A 360-Degree Video Analytics Service for In-Classroom Firefighter Training”.

Involved in an NSF project “miVirtualSeat”, whose goal was to enable hybrid teleconferencing, comprising local and remote participants wearing augmented-reality headsets and engaging in a natural-feeling in-person meeting. Interacting with a diverse team of researchers to develop ways to detect, track, and localize distributed physical and virtual 360 avatars and objects in real time to ensure a good quality-of-experience (QoE) between physical and virtual participants. o Intel Corporation, Santa Clara, CA

Deep Learning Software Engineer Intern, May 2021-August 2021 Enhanced BRGEMM (Batch-Reduce General Matrix-Matrix Multiplication) operations for 1D dilated Convolution Neural Networks

(1D_CNNs) by integrating into ONNX runtime as an operator for applications like NLP, speech recognition, etc. Achieved 2x performance improvement of BRGEMM over SGEMM (Single precision General Matrix Multiply) for Intel processor graphics. o Department of Computer Science, University of Illinois at Urbana Champaign, IL Graduate Teaching Assistant, January 2021-May 2021 Worked as a Teaching Assistant for the Artificial Intelligence course (CS440/ECE448) at UIUC. In addition to AI concepts, the lectures included applications like games, robotics, computer vision, and natural language processing. o Coordinated Science Laboratory (CSL), University of Illinois at Urbana Champaign, IL Graduate Research Assistant, August 2020 – December 2020 Worked on an NSF project called Clowder under the supervision of Prof. Klara Nahrstedt. Clowder is a cloud-based scalable, distributed data management system where data from disparate scientific instruments and other sources is stored, shared, organized, and analyzed for scientific research and discoveries. Implemented hierarchical role-based access control techniques for a single user and groups of users.

o Coordinated Science Laboratory (CSL), University of Illinois at Urbana Champaign, IL Summer Intern, June 2020 – July 2020

Worked on the Clowder project under the supervision of Prof. Klara Nahrstedt. Designed access control techniques for a single user and groups of users performing forensic analysis and provenance over various trails of actions executed in Clowder. Implementation included Docker, Git, Java, Python, MongoDB (as Clowder data repository), a flexible user interface and analytical tools. 2

o Summer Research Projects

May 2019 – August 2019

Implemented a software ecosystem using deep learning within blockchain smart contracts. Performed research for using the ecosystem within the healthcare domain, where deep learning could be applied to component medical images, signatures, handwritten text, information in medical forms, and documents, etc., for digital identity management in smart contracts. Presented

“Deep Chain Learning by Collusions over Networks” at the Engineering Research Fair and International Research Symposium, University of Illinois at Urbana-Champaign. The presentation included techniques unifying deep learning and blockchain smart contracts for various industry verticals like healthcare, auto-insurance, supply chain and digital identity management. May 2018 – August 2018

Implemented an auto insurance claim management system to foster acceptance of digital attestations within a legal framework, enabling trust in the security of blockchain smart contracts integrated with deep learning. The implementation involved a) the decomposition of insurance forms, claims and identity documents into component text, signatures, and images, followed by using JSON for (attribute, value) pairs to represent all components; b) writing JSON queries in blockchain for propagating the main and/or component objects to all parties for content recognition using deep learning followed by sharing of results; and c) demonstrating an increased efficiency and quality of service (QoS) using a decentralized process for analyzing and validating sensitive and legal documents.

May 2017 – August 2017

Performed research and prototype development based on the principles of Deep Learning and Blockchain; investigated possibilities for unifying neural networks and blockchain into a new AI technique called “Deep-chain Learning,” along with use cases for applying this concept in DAOs (Decentralized Autonomous Organizations). Presented topics on Convolutional Neural Nets at a Meetup event at DataWorks Summit, 2018 in San Jose, CA.

o Other Research Projects

May 2014 – August 2015

Performed research and development on Sensor data analytics from IoT devices, collaborating with an engineering mentor from Teradata Corporation. Implemented a smart home data analytics system using 2M records from 54 sensors within Intel-Berkeley Research Lab. Came up with a new model called CALSTATDN, which iterated over a sequence of computing stages based on a Calculus

(CAL) based model, a Statistics (STAT) based model and database normalization (DN) to reduce processing time of machine learning by several orders of magnitude. Applied this new iterative machine learning model using Java and K-Means clustering on the sensor data sets and showed a significant reduction in time for processing and analysis. This project won the first place at the District Science Fair 2015. A US patent was issued on January 8, 2019 (Patent# 10176435). January 2013 – May 2014

Performed research and worked on a project, “An Information Supply Chain for Genome Analysis and Cancer Cure”. Implemented a method for analyzing re-sequenced cancer genomes using decision tree learning within a graph database. Demonstrated how decision tree pruning can increase the precision of discovering regions in chromosomes with large mutation counts. Collaborated with an engineering mentor at Silicon Graphics International (SGI). Presented this project at AMPLAB, U.C. Berkeley. This project won an award in the Computer Science Category at the Alameda County Science and Engineering Fair 2014. May 2012 – January 2013

Performed independent research and worked on a project, “Vertical axis wind turbine using helical blades with serrated edges”. Built a vertical axis wind turbine comprising helical shaped blades with serrated edges that exhibited unique airfoil characteristics. Experimented with various wind speeds and blade designs and proved that this innovative design led to an improved efficiency of the turbine. This work won an award in the Environmental Science category at the Alameda County Science and Engineering Fair 2013. A US patent was issued on December 1, 2015 (Patent# 9200615).

• PUBLICATIONS/PRESENTATIONS

o May 2022: The 1st Workshop on Cyber Physical Systems for Emergency Response (CPS-ER 2022 Workshop, Co-located with CPS-IOT Week 2022) – “A 360-Degree Video Analytics Service for In-Classroom Firefighter Training”. o November 2021: 23rd IEEE International Symposium on Multimedia (Best Student Paper Award) - “L3BOU: Low Latency, Low Bandwidth, Optimized Super-Resolution Backhaul for 360-Degree Video Streaming”. o May 2020: BS Thesis, University of Illinois at Urbana-Champaign – “Deep Chain Learning by Collusions over Network with Improved Blockchain Security”.

o September 2019: Engineering Research Fair and International Research Symposium, University of Illinois at Urbana-Champaign, IL

– “Deep Chain Learning by Collusions over Networks”. o June 2018: Big Data Science Meetup, DataWorks Summit, San Jose, CA – “Convolutional Neural Networks - Theories and Practices”. o June 2017: (Co-Presenter) Big Data Science Meetup, DataWorks and Hadoop Summit, San Jose, CA – “Deep-chain Learning”. o 2014-2016: Big Data Science Meetups, San Francisco Bay Area, CA

“The Big Picture of Cancer Genomics”

“Big Data, Hadoop, NoSQL Graph databases and Cancer Genomics”

“CALSTATDN – A New Model for Sensor Data Analytics and Internet of Things”. 3

o September 2015: 3rd Annual Global Big Data Conference, Santa Clara, CA - “Combining Techniques of Calculus, Statistics and Data Normalization in CALSTATDN model”.

o US Patents:

“Method and Apparatus for combining techniques of Calculus, Statistics and Data Normalization in Machine Learning for analyzing large volumes of data” (Patent# 10176435, issued January 8, 2019)

“Vertical axis wind turbine using helical blades with serrated edges” (Patent# 9200615, issued December 1, 2015).

• EDUCATION

University of Illinois at Urbana-Champaign, IL

o Ph.D., Computer Science (August 2022 – Present)

Working on Deep Generative Models for AI, focusing on architecture, performance and QoS (Quality of Service) under the supervision of Professor Svetlana Lazebnik, CS Department. o M.S, Computer Science, May 2022. GPA 3.95

Key Projects Accomplished:

- Developed a novel three-tier software architecture called L3BOU, which reduces cloud-edge bandwidth in the backhaul network and lowers average end-to- end latency for 360 video streaming applications. L3BOU accomplishes this by utilizing down-scaled MPEG-DASH-encoded 360 video data, known as Ultra Low Resolution (ULR) data, that the L3BOU edge applies distributed super-resolution (SR) techniques on, providing a high-quality video to the client. L3BOU is able to reduce the cloud-edge backhaul bandwidth by up to a factor of 24, and the optimized super-resolution processing of ULR data provides a 10-fold latency reduction in super resolution upscaling at the edge. This work received the Best Student Paper Award at the IEEE ISM 2021 conference - “L3BOU: Low Latency, Low Bandwidth, Optimized Super-Resolution Backhaul for 360-Degree Video Streaming”.

- As a part of the advanced course on the theory of reinforcement learning (RL) focusing on sample complexity analysis, learned about Markov Decision Process (MDP), finite-sample analysis of online and offline RL with a tabular representation, finite- sample analysis of online and offline RL with function approximation, state abstraction theory, importance sampling and policy gradient. The course project involved acceleration of imitation learning with predictive models. Online imitation learning, which interleaves policy evaluation and policy optimization provides performance guarantees. The investigation was on further acceleration of convergence rate of online limitation learning for making it more sample efficient.

- Implemented a three-tier pipeline architecture that leverages the advantages of sequential pattern mining for probabilistic log processing. The transactions occurring in a database management system were modeled as Bayesian networks adhering to the Markov assumption. The system employed probabilistic graph processing techniques for extracting actionable knowledge by executing sequential pattern mining algorithms on normalized logs. The implementation provides an online query-based system where a user queries about a particular event in an event log and the system outputs the log lines estimated to have occurred in the same transaction.

- Applied the concept of transfer learning to construct a classifier that classifies financial tweets as “positive” or “negative” to evaluate stock market movements through sentiment analysis. One thousand financial tweets were collected for 50 different stocks through Twitter’s API. A pre-trained language model trained on the Wikitext-103 dataset was fine-tuned on a sample of the Sentiment-140 dataset to gain a better understanding of the informal language used in tweets. The encoder of the fine- tuned model was then used for the new sentiment classifier. This sentiment analysis project was coded using Python and used a MongoDB database to store the tweets.

o B.S. with Honors, Computer Engineering, May 2020. GPA 3.56

Senior thesis (Advisor Prof. Klara Nahrstedt, CS department, UIUC): Deep Chain Learning by Collusions over Network with Improved Blockchain Security

The thesis introduces a concept called “Deep Chain Learning,” which provides a secured technique for integrating deep learning within smart contracts in a blockchain in a decentralized architecture. A user-authentication mechanism is used for accessing different object component(s) (images, signatures, handwritten text, information in forms, and documents) by each party in a blockchain to execute deep learning, triggered by the smart contracts. Three use cases involving an auto insurance and healthcare application were executed for performance evaluations. The distribution of the inference tasks among multiple parties led to an almost linear reduction of the elapsed times for identifications and verifications of the component images and improved the overall quality of service (QoS), when compared to a centralized, sequential mode of execution. Implemented a 4-node blockchain with Linux/Windows servers, and used Anaconda and relevant packages like flask, requests, numpy, keras (with backend TensorFlow), json, re, time, tesseract, and pytesseract.

• EXTRACURRICULAR ACTIVITIES

o 2019-Present: Active Member of IEEE and ACM professional organizations for research, publications, conferences, and meetups. o 2017-Present: Co-Organizer of Big Data Science Meetup events, sponsored by various companies in San Francisco Bay Area, CA. 4

• AWARDS / HONORS

o 2022: Member, Tau Beta Pi - The Engineering Honor Society o 2022: Member, Phi Kappa Phi Honor Society

o 2021: Best Student Paper Award, 23rd IEEE International Symposium on Multimedia (IEEE ISM 2021). o 2020: B.S. with Honors, Computer Engineering, University of Illinois at Urbana Champaign. o 2017-2018: Dean’s list, College of Engineering, University of Illinois at Urbana Champaign. o 2016: College Board AP Scholar with Honor.

o 2013-2015: 1st at High School Science Fairs and District Science Fair, Fremont, CA. o 2013-2014: Category Awards in Computer Science and Environmental Science at Alameda County Science and Engineering Fair, CA. o 2014-2015: Two Presidential Volunteer Service Silver awards.

Contact this candidate