Recently, artificial intelligence has received serious development. In some areas, neural networks can already compete with humans. For example, Midjourney can already produce high quality images and ChatGPT can write or analyse fragments of code.
Callisto Team is pioneering the adoption of neural networks in security area to automate the process of bulk code reviews.
The task
We decided to analyse real DeFi exploits in living DAPPs deployed on Ethereum chain. Since Monday, April 3 me and Yuriy manually reviewed the codes of dozens of DeFis however with no significant results.
The problem with finding an exploitable vulnerability in a DeFi right now is that there are too much DeFis and 90% of them are clones of one another and there can be no custom code. If you are reviewing the code that doesn’t contain any vulnerabilities — it doesn’t matter if you are a security expert or not, you will not find any vulnerabilities.
The analysis of known DeFi hacks showed that in most cases the exploitable vulnerability is a well-known mistake (such as replay attack or a code developer forgot to add “onlyOwner” modifier to governance functions).
After that we came to the conclusion that in order to improve the security of the industry of smart-contracts we need a “security crawler” — a program that will continuously analyse the codes of newly created contracts and check them for well-known vulnerabilities. If the crawler would detect a vulnerability in a newly created contract — this can be fixed before this contract will receive significant amount of funds letting hackers to drain it later.
AI powered security crawler
The latest research showed that ChatGPT is capable of detecting vulnerabilities and exploits in a code. At least well-known vulnerabilities have a very high chance to be detected.
The latest version of ChatGPT allows us to submit 20–30 requests per hour.
So, the plan is to create a program that will take the code of a verified contract from Etherscan or similar explorer on another chain and then feed 20 of them per hour to ChatGPT. The contracts that will be flagged as “potentially insecure” will be manually reviewed by our security auditors.
We are planning to scan all existing contracts on major chains to find any existing vulnerabilities. After that the crawler will stay online to monitor the newly created contracts.
If this practice will prove efficient we can expand the functionality of the crawler to analyse the bytecodes of the contracts instead of just verified contracts.