AI ACI Worldwide

Introduction:
ACI Worldwide is a leader in real-time payments, delivering quick and robust software solutions to meet the demand of today’s swift payment systems. At ACI, we have adopted data-driven solutions to provide swift payment solutions, but we also care about swifter software releases and speeding up time to market.

A software engineer spends 40% of the programming effort writing unit-test cases to make software robust and reliable. This effort could be potentially minimised using AI-based solutions. The Data Science team at ACI Worldwide is committed to driving this massive effort-saving solution that can automatically write unit-test cases to productionise software solutions faster.

Project Background:
Unit testing refers to testing individual components in source code, such as classes and their provided methods. The purpose is to validate that each software component performs as designed. This is a manual process. If automated, the effort required for this process can be significantly reduced.

Key Challenges/Opportunities:
This problem statement can be defined as a special application of Natural Language Processing techniques where instead of natural languages, we are analysing Java codebases. The critical problem here is that while tools, implementation examples and software solutions are readily available for natural language processing, technology is a relatively new field. Therefore, there are limited resources to refer to. Moreover, there is less research on unit test generation using artificial intelligence.

The opportunity is to be ahead of the game in applying AI to reduce manual effort in writing unit tests and let our engineers instead focus on the development side.

How the project was executed:
Research is the key to accomplishing this project. Automated unit-test case generation has been conceptualised in three phases of development:
Phase 1: Pilot Research and Design Phase
• Feasibility Analysis
• Codebase Analysis using NLP to identify opportunities
• Problem-solving Approach Research
o Syntax-based Approach and algorithm development
o Machine Learning-based approach
• Identifying KPIs and metrics

Phase 2: Implementation Phase
• Abstract Syntax Tree Search Algorithm Development
• Transformers for Code Generation
• Testing the Results and Feedback Analysis
• Cost-Benefit Analysis

Phase 3: Roll-out Phase
• Production Deployment
• Bitbucket integration

Description of AI Solution:
The first step of building this AI solution was to detect the elements of Java code, like identifying classes and methods, class instantiations, Java keywords, conditional and return statements. Before applying on our proprietary codebases, an initial scanning was performed on Java GitHub repositories. One of the impressive results from this study was the word collocations obtained by spreading the word embedding generation approach using word2vec school of algorithms on the Java codebase, where it is seen that public, static, string, and override occur together, whereas string is closer to new, which indicates the usage of new String syntax over other object creations, whereas try, catch, throw, logger, occur together. This reinstates applying natural language processing techniques to programming languages and human language. Besides, the investigation also looked at what kind of Java keywords occurred to prioritise the types of unit test case generation.

We had begun at LSTMs and Transformers for causal text generation in the research phase. We designed the system to take an input called the focal method, essentially the Java function for which a unit test case is being written. This focal method is the seed for the causal text generation; however, as text generation is open-ended, we moved to study a masked text generation approach.

In masked text generation approach, our first step was to analyse the patterns in test cases and categorise them. This approach is based on the idea that patterns in programming languages are finite and deterministic while that in natural  languages, patterns are infinite and partially deterministic. Using the pre-identified patterns in the Java codebase, a suitable masked test case template is selected. Based on the information extracted from each given focal method, one or more unit test cases are created by predicting the masks. The information extraction is aided by parsing Java code into Abstract Syntax Tree or AST, which is then traversed to get relevant information. Additionally, packages, classes for java files and imports are also generated automatically.

Challenges:
Programming languages have repetitive patterns, but the logic embedded is complex. This is the main challenge of this project. The state-of-the-art NLP models cannot understand the meaning of a piece of code until it is provided with a tag or a label. While logic is essential, we only undertook a syntactic approach in this study, while a logic-based method will be implemented in the upcoming phases. The bottleneck again is the data. While there are datasets available for text to code generation and models for the same, that pre-trained model would finally need to be fine-tuned on the proprietary codebase since a general model will not be able to pick up the patterns here.

Business / Organisational Impact
Having the edge over time to market is essential in today’s competitive market. Saving manual effort in writing use-cases will give developers valuable time to work on other tasks involved in software development. From an engineer’s point of view, this project is estimated to save about 80 days’ worth of work for one engineering team in one year, precious time that can be utilised for other essential tasks. This project saves time, but it also greatly assists in achieving robustness, as credibility and reliability are cornerstones of any software product. This solution helps push up code coverage and enhance the confidence that engineering teams can have in their work and clients in the product.