Lei Ma

Associate Professor

The University of Tokyo, Japan

University of Alberta, Canada

Email: ma.lei [at] acm [dot] org

Prospective Students:

My research group is continuously recruiting. Currently, our lab have multiple scholarship & RA positions for Ph.D., MSc. candidates, and postdocs with competitive salary support, at both The University of Tokyo, Japan and University of Alberta, Canada. If you have a strong background in the topics relevant (but not limited) to my research with strong self-motivation & strong will to achieve, I strongly encourage you to apply and join us. Note that to inquiry about opportunities to join our research group, please send emails only to contact@momentum-lab.com to inquiry and apply!
For students already with Todai, or students who can come to Todai Hongo Campus area, please simply drop me an email to schedule a meet up at the Campus, to discuss the possibilities.

NEWS

(03/2024) Our paper Active Code Learning: Benchmarking Sample-Efficient Training of Code Models is accepted at IEEE Transactions on Software Engineering (TSE).
(02/2024) Our paper PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinement is accepted at ACM CHI 2024. This paper proposes PromptCharm, a very early mixed-initiative and interactive system that facilitates text-to-image creation through multi-modal prompt engineering and refinement. It helps designers in prompting, automatically refines and optimizes the the initial prompt, and further supports in exploring and selecting different image styles within a large database with iterative refinments.
(01/2024) Our paper ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planning is accepted at ICRA 2024. LLMs are advantageous in offering the potential to enhance the generalizability as task-agnostic planners and facilitate flexible interaction between human instructors and planning systems. However, task plans generated by LLMs often lack feasibility and correctness. To address this challenge, in this exploratory work, we designed a novel framework that improves LLM-based planning through an iterative self-refinement process towards appraoching better fesibility and trustworthiness.
(01/2024) Our papers LRR: Language-Driven Resamplable Continuous Representation against Adversarial Tracking Attacks and Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalization are accepted at ICLR 2024.
(12/2023) Our paper MultiTest: Physical-Aware Object Insertion for Testing Multi-sensor Fusion Perception Systems is accepted at ICSE 2024. This is our exploratory work that proposes a testing framework for muti-sensor system by considering the multi-modality characteristics of such systems.
(10/2023) Our paper LUNA: A Model-Based Universal Analysis Framework for Large Language Models is available on Arxiv. This paper introduces the inaugural mode-based, general-purpose analysis framework designed to scrutinize, comprehend, and rationalize the behaviors and risks associated with inferences made by Large Language Models (LLMs) following a human-centered manner. LUNA is crafted with high extensibility in mind, and our ongoing efforts are geared towards adapting LUNA for various domain-specific tasks and ensuring its compliance with trustworthiness standards. We would regularly updating more extensive findings. Should you have an interest in LUNA, for both researchers or industry practitioners, please do not hesitate to contact us. We welcome collaboration and are keen to explore the potential of LUNA together.
(08/2023) Our paper Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models is available on Arxiv. This paper investigates blackbox techniques to understand the confidence of inference by LLMs. We found that even under the blackbox setting, we can still gain intuition and understanding of the level of confidence in an LLM's inference at industrial scale. We will continuously update and release more comprehensive results.
(08/2023) Our paper Is Model Attention Aligned with Human Attention? An Empirical Study on Large Language Models for Code Generation is available on Arxiv. This is an study for better understanding and constructing human-centered process of Large Language Models for Code Generation.
(08/2023) Our paper Towards Building AI-CPS with NVIDIA Isaac Sim: An Industrial Benchmark and Case Study for Robotics Manipulation is available on Arxiv. This is our exploratory study towards sim to real, via efficiently building AI-CPS with Nvidia Isaac Sim as an early step.
(08/2023) Three papers are accepted by ESEC/FSE 2023.
(08/2023) Three papers are accepted by ASE 2023.
(06/2023) Our paper SIEGE: A Semantics-Guided Safety Enhancement Framework for AI-Enabled Cyber-Physical Systems is accepted at TSE 2023 . This paper performs an early explorative study for the safety enhancement of AI-CPS via ensembling approach, which is also a continuous series of work towards investigating safety assurance after our previous work on AI-CPS benchmarking (ICSE 2022), Falsifai (TSE 2022), etc.
(06/2023) Our paper Faire: Repairing Fairness of Neural Networks via Neuron Condition Synthesis is accepted at ACM TOSEM 2023 .
(04/2023) Our work ArchRepair on the Repair of Deep Neural Network and the recent research activities of JST eAI project has been reported by 日本経済新聞 (Nikkei News) , also see here.
(04/2023) On April 2023, I joined computer science department of The University of Tokyo, Japan as Associate Professor.
(03/2023) Our paper Evading DeepFake Detectors via Adversarial Statistical Consistency is accepted at CVPR 2023 .
(02/2023) Our paper PatchCensor: Patch Robustness Certification for Transformers via Exhaustive Testing is accepted at ACM Transactions on Software Engineering and Methodology (TOSEM 2023) .
(02/2023) Our paper ArchRepair: Block-Level Architecture-Oriented Repairing for Deep Neural Networks is accepted at ACM Transactions on Software Engineering and Methodology (TOSEM 2023) .
(01/2023) Our papers on human-centered interactive data debugging DeepLens and model debugging DeepSeer are accepted at CHI 2023 .
(01/2023) Our paper Neural Episodic Control with State Abstraction is accepted at ICLR 2023 as a spotlight paper .
(12/2022) Our paper GraphSearchNet: Enhancing GNNs via Capturing Global Dependencies for Semantic Code Search is accepted at IEEE Transactions on Software Engineering (TSE 2023) .
(12/2022) Four papers accepted at ICSE 2023 (two on research track, one on SEIP track, one NIER track).
(11/2022) Our paper DeepGemini: Verifying Dependency Fairness for Deep Neural Network is accepted at AAAI 2023 .
(07/2022) Join IEEE CIS Neural Networks Committee on Secure Learning as a Task Force Member.
(07/2022) Our paper Decompiling x86 Deep Neural Network Executables is accepted at The 32nd USENIX Security Symposium (USENIX Security '23 Summer)
(07/2022) Our paper FalsifAI: Falsification of AI-Enabled Hybrid Control Systems Guided by Time-Aware Coverage Criteria is accepted at IEEE Transactions on Software Engineering (TSE)
(06/2022) Our paper DARTSRepair: Core-Failure-Set Guided DARTS for Network Robustness to Common Corruptions is accepted at Pattern Recognition 2022
(06/2022) Our paper xFuzz: Machine Learning Guided Cross-Contract Fuzzing is accepted at IEEE Transactions on Dependable and Secure Computing, TDSC 2022
(03/2022) Our paper Countering Malicious DeepFakes: Survey, Battleground, and Horizon is accepted at International Journal of Computer Vision, IJCV 2022. We present a comprehensive survey covering the latest DeepFake generation and detection methods, as well as the battleground landscape.
(01/2022) Our paper When Cyber-Physical Systems Meet AI: A Benchmark, an Evaluation, and a Way Forward is accepted at ICSE SEIP 2022, to the best of our knowledge, this is the very first systematic benchmark construction and study of AI-enabled Cyber Physical Systems (AI-CPS). We make our dataset and benchmark publically available, and hope this would inspire lots of future new reserach and investigation along this exciting direction!
(01/2022) Our paper An Empirical Study on Data Distribution-Aware Test Selection for Deep Learning Enhancement is accepted at ACM Transactions on Software Engineering and Methodology (TOSEM 2022).
(12/2021) Our paper An Empirical Study of the Impact of Hyperparameter Tuning and Model Optimization on DNN is accepted at ACM Transactions on Software Engineering and Methodology (TOSEM 2021).
(11/2021) Our paper Towards Robustness of Deep Program Processing Models -- Detection, Estimation and Enhancement is accepted at ACM Transactions on Software Engineering and Methodology (TOSEM 2021).
(09/2021) Our paper NPC: Neuron Path Coverage via Characterizing Decision Logic of Deep Neural Networks is accepted at ACM Transactions on Software Engineering and Methodology (TOSEM 2021).
(08/2021) Our paper DeepRepair: Style-Guided Repairing for Deep Neural Networks in the Real-World Operational Environment is accepted at IEEE Transactions on Reliability 2021.
(07/2021) Our paper Learning to Adversarially Blur Visual Object Tracking is accepted at ICCV 2021.
(07/2021) Join IJCAI 2022 as a PC Member.
(07/2021) Our paper Predictive Perturbation-aware Filtering against Adversarial Attack via Multi-domain Learning is accepted at ACM MM 2021.
(07/2021) Our paper An Empirical Study of GUI Widget Detection for Industrial Mobile Games is accepted at FSE 2021 Industry Track.
(06/2021) Join ICLR 2022 as a reviewer, AISec 2021 and ESEC/FSE 2022 as a PC member. Please consider submitting your high quality papers.
(05/2021) Our paper Automatic RNN Repair via Model-based Analysis is accepted at ICML 2021.
(04/2021) Our paper Effective Hybrid System Falsification Using Monte Carlo Tree Search Guided by QB-Robustness is accepted at CAV 2021.
(03/2021) Our survey paper Countering Malicious DeepFakes: Survey, Battleground, and Horizon that makes comprehensive coverage regarding malicious DeepFakes is available on Arxiv.
(12/2020) Media coverage: Our work DeepRhythm that uses heartbeat detection to identify deepfake videos is reported in English (Link1, Link2), Japanese, and Chinese!
(12/2020) Our paper Automatic Web Testing using Curiosity-Driven Reinforcement Learning is accepted at ICSE 2021.
(12/2020) Our paper A Search-Based Testing Framework for Deep Neural Networks of Source Code Embedding is accepted at ICST 2021.
(12/2020) Join ICML 2021 and CVPR 2021 as reviewers.
(12/2020) Our papers EfficientDeRain: Learning Pixel-wise Dilation Filtering for High-Efficiency Single-Image Deraining and Decision-Guided Weighted Automata Extraction from Recurrent Neural Networks are accepted at AAAI 2021.
(11/2020) Join ICSE 2022 as PC Member.
(11/2020) Our paper Oracle-Supported Dynamic Exploit Generation for Smart Contracts after two years' countinous effort is finally accepted at IEEE Transactions on Dependable and Secure Computing (TDSC)!
(11/2020) Join DeepTest 2021 as PC Member.
(11/2020) Join GECCO 2021 as PC Member.
(10/2020) Join ASE 2021 as PC Member.
(10/2020) Join ICPC 2021 as PC Member.
(10/2020) Join IJCAI 2021 as PC Member.
(09/2020) Our paper Watch out! Motion is Blurring the Vision of Your Deep Neural Networks is accepted at NeurIPS 2020. This paper performs intensitive investigation on the impact of potential camera blurring effects to the decision of a DNN!
(09/2020) Join AAAI 2021 as PC Member.
(08/2020) Our empirical study on Autonomous Driving Systems and its ML Components is accepted at Industry Track of ESEC/FSE 2020.
(07/2020) Our papers MARBLE: Model-Based Robustness Analysis of Stateful Deep Learning Systems and Cats Are Not Fish: Deep Learning Testing Calls for Out-Of-Distribution Awareness are accepted at ASE 2020.
(07/2020) Two papers are accepted at ICSME 2020, NIER Track.
(07/2020) Our DeepFake Series works DeepRhythm, DeepSonar, and Amora are accepted at ACM MM 2020.
(07/2020) Our paper SPARK: Spatial-aware Online Incremental Attack Against Visual Tracking is accepted at ECCV 2020. This paper proposes a novel and efficient attack to mislead online trackers along an incorrect or specified trajectory!
(06/2020) Join ESEC/FSE 2021 as PC Member.
(06/2020) Join NeurIPS’20 as a reviewer.
(05/2020) Our paper FakeSpotter is accepted at IJCAI 2020, a previous verison is reported as AI vs AI: ‘FakeSpotter’ Studies Neurons to Bust DeepFakes, by the Synced AI Industry Review!
(05/2020) Our paper DeepSonar: Towards Effective and Robust Detection of AI-Synthesized Fake Voices is available on CoRR arXiv!
(04/2020) Join AISec’20, ASE'20, ICSME'20 as PC member.
(03/2020) Give tutorials regarding Testing and Analysis of Stateful Deep Learning Systems at IVADO Training Program on Software Engineering for Machine Learning Applications, at Montreal, Canada, March 2-6 2020.
(02/2020) Give Invited talk Automatic Video Game Testing via Evolutionary Deep Reinforcement Learning at Polytechnique Montreal, Canada, Feb.6, 2020
(02/2020) Give Invited talk Quality and Reliability Assurance for Deep Learning Systems at School of Computer Science, McGill University, Canada, Feb.5, 2020
(02/2020) Our paper Towards Byzantine-resilient Learning in Decentralized Systems is available on CoRR arXiv!
(02/2020) Our paper ABBA: Saliency-Regularized Motion-Based Adversarial Blur Attack is available on CoRR arXiv!
(01/2020) Our paper FakeLocator: Robust Localization of GAN-Based Face Manipulations via Semantic Segmentation Networks with Bells and Whistles is available on CoRR arXiv!
(12/2019) I am honored to be invited to Dagstuhl Seminar 20091:SE4ML - Software Engineering for AI-ML-based Systems to be held on February 23 – 28, 2020
(12/2019) I am honored to be invited to give lectures at IVADO Training Program on Software Engineering for Machine Learning Applications, to be held at Montreal, Canada, March 2-6 2020.
(12/2019) Our paper Amora: Black-box Adversarial Morphing Attack is available on CoRR arXiv!
(12/2019) Our paper Towards Characterizing Adversarial Defects of Deep Learning Software from the Lens of Uncertainty is accepted at ICSE'2020, This work is the very early work that discovers the new uncommon adversarial examples that represent the challenging vulnerability of deep learning software, we are now working to seek new ways to counteract these issues!
(11/2019) Our survey paper Machine Learning Testing: Survey, Landscapes and Horizons, on the emerging direction of testing and anlaysis of machine learning is accepted at IEEE Transactions on Software Engineering (TSE), a previous early draft is available at CoRR arXiv, we are working on the camera-ready version and would make it available soon!
(11/2019) Our research framework DeepStudio: a Framework of Securing Deep Learning Systems receives the 2019 CCF Research Prototype Competition Award (Second Prize).
(11/2019) Our papers Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning and Generating Adversarial Examples for Holding Robustness of Source Code Processing Models are accepted at AAAI 2020.
(10/2019) Our paper Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning receives the ACM SIGSOFT Distinguished Paper Award in ASE'19.
(10/2019) Media coverage: AI vs AI: ‘FakeSpotter’ Studies Neurons to Bust DeepFakes, our work FakeSpotter is reported by the media of AI Industry Review!

(10/2019) Our paper Model-based Testing of Apache ZooKeeper: Fundamental API Usage and Watchers is finally accepted at Software Testing, Verification AND Reliability (STVR) Journal after two years continous effort !

(09/2019) Our papers DeepMutation++, DeepHunter, and DeepStellar are accepted and will be demonstrated at ASE 2019 as tool papers !

(09/2019) Our papers Wuji and Deep Learning Platform/Framework Compatibility Study are accepted at ASE 2019 as full research papers !

(08/2019) Our paper CDA: Characterising Deprecated Android APIs is accepted at Empirical Software Engineering (EMSE) Journal !

(07/2019) Our paper An Empirical Study of Common Challenges in Developing Deep Learning Applications is accepted at ISSRE'19 !

(07/2019) Two papers: Safe Inputs Generation for Black-box Systems and MobiDroid: A Performance-Sensitive Malware Detection System on Mobile Platform are accepted at ICECCS 2019!

(06/2019) Our survey paper Machine Learning Testing: Survey, Landscapes and Horizons is available on CoRR arXiv!

(05/2019) Our paper DiffChaser: Detecting Disagreements for Deep Neural Networks is accepted at IJCAI'19 !

(05/2019) Our paper DeepStellar: Model-Based Quantitative Analysis of Stateful Deep Learning Systems is accepted at FSE'19 !

(05/2019) I am invited to join AISec'19 Program Committee collocated with CSS'19 this year, please considier to submit your high quality papers!

(05/2019) I am invited to visit and give a talk on Testing Deep Learning System Towards Intelligence of Things at Google Waterloo on June 5!

(05/2019) Our paper DeepHunter: A Coverage-Guided Fuzz Testing Framework for Deep Neural Networks is accepted at ISSTA'19 !

(04/2019) I am happy to be invited to join and give a talk on NII Shonan Meeting on Software Engineering for Machine Learning Systems (SEMLS) which will be held on October this year.

(04/2019) I am invited by Machine Learning and Software Engineering Community of Japan (MLSE) to give tutorial on "Software Testing of Deep Learning System: from the Past to the Future"

(04/2019) I am co-organizing the first MLST Workshop Collocated with ICST!

(03/2019) I am happy to be invited to join and give a talk on The Software Engineering for Machine Learning Applications (SEMLA) International Symposium, Polytechnique Montreal which will be held on May this year.

(02/2019) I am co-organizing the first AI4Mobile Workshop Collocated with SANER!

(12/2018) Our paper VULTRON: Catching Vulnerable Smart Contracts Once and for All ICSE'19 as an NIER Track Paper!

(12/2018) Our paper DeepCT: Tomographic Combinatorial Testing for Deep Learning Systems SANER'19 as an ERA Track Paper!

(09/2018) Our paper Practical Fault Attack on Deep Neural Networks is accepted at CCS'18 as a Poster Paper!

(09/2018) Our paper Metamorphic Relation Based Adversarial Attacks on Differentiable Neural Computer is available on CoRR arXiv. This is the very first work towards in-depth analysis on the robustness of Differtentiable Neural Computer!

(08/2018) Our paper DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems receive the ACM SIGSOFT Distinguished Paper Award in ASE'18.

(08/2018) Our paper DeepMutation: Mutation Testing of Deep Learning Systems is accepted at ISSRE'18

(07/2018) Our paper DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems is accepted at ASE'18

(06/2018) Our paper Combinatorial Testing for Deep Learning Systems is available on CoRR arXiv. This is the first combinatorial testing for Deep Neural Networks!

(06/2018) Our paper Practical Fault Attack on Deep Neural Networks is available on CoRR arXiv. This is the first practical attacks on Deep Neural Networks using laser technology!

(05/2018) Our paper Characterising Deprecated Android APIs won the FOSS Impact Paper Award in MSR'18.

(05/2018) Our paper DeepMutation: Mutation Testing of Deep Learning Systems is available on CoRR arXiv. This is the first mutation testing technique for Deep Neural Networks!

(03/2018) Our paper DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems is available on CoRR arXiv. This paper proposes a set of testing critiera with multi-granuality for Deep Neural Networks!

(01/2018) Our paper Characterising Deprecated Android APIs has been accepted to appear in MSR'18.

(03/2017) Our paper Model-based API Testing of Apache ZooKeeper won the IEEE Best Presentation Award at ICST'17.

(07/2016) Our paper Runtime monitoring for concurrent systems has been accepted to appear in RV'16.

(09/2016) Our paper An empirical study on the effects of code visibility on program testability has been accepted to appear in Springer Software Quliaty Journal (SQJ).

(03/2016) Our paper Efficient and Precise Dynamic Slicing for Client-Side JavaScript Programs has been nominated for the IEEE Best Candidate Paper Award in SANER'16.

(11/2015) Our paper GRT: Program-analysis-guided Random Testing won the ACM SIGSOFT Distinguished Paper Award at ASE'15.

(11/2015) Our paper Incremental Web Application Testing using Page Object won the IEEE Best Paper Award at HotWeb'15.

Pulications

Associate Professor

The University of Tokyo, Japan

University of Alberta, Canada

Email: ma.lei [at] acm [dot] org

Prospective Students:

About Me