July 17, 2024
Check Your Good Contract Code with ChatGPT and Diligence Fuzzing

ChatGPT is phenomenal at discovering bugs with out requiring any significant extra context past the supply code. This makes it a superb first place to start out your testing journey.

One of many first belongings you be taught as a sensible contract coder is that contracts are extraordinarily unforgiving in the case of vulnerabilities. Contracts are immutable by default. On the identical time, they’re able to dealing with extraordinary quantities of cash. This makes safety (and sensible contract testing) arguably the most important concern for any staff of web3 builders. 

However auditing contracts—the everyday final of line protection towards bugs—has traditionally been time-consuming and costly. Luckily, there was an explosion in highly effective instruments that make safety and sensible contract testing cheaper, less complicated, and sooner.

On this article, we are going to discover testing sensible contracts utilizing two of those instruments: ChatGPT by OpenAI and Diligence Fuzzing by ConsenSys.

Testing an Ethereum Good Contract with ChatGPT and Diligence Fuzzing

Let’s stroll via a sensible contract for an Ethereum venture that options weak ERC-20 code and see what the 2 instruments can do for us. 

Diligence Fuzzing is a device from ConsenSys that implements fuzzing for web3. Fuzzing is a dynamic testing method the place random (or semi-random) inputs referred to as “fuzz” are generated and injected into code. Fuzzing might help reveal bugs and vulnerabilities that weren’t caught by conventional testing strategies.

And naturally, ChatGPT is a big language mannequin (LLM) AI. Until you’ve been residing below a rock, you’re most likely already conscious of ChatGPT and a few of its capabilities. It’s probably the most thrilling applied sciences to have come out within the final decade. Not solely can it enable you to with testing (as we’ll see on this article), but additionally with sensible contract growth, writing assessments, and extra.

Let’s get began!

Step 1: Set up Python, pip, npm, and Node.js

In an effort to set up the instruments required to run Diligence Fuzzing, you first want the next in your native machine: the newest variations of Python and pip and the newest variations of Node.js and npm

Examine that you’ve got all 4 operating by utilizing these instructions:

$ node -v
$ npm -v
$ python –-version
$ pip --version

Step 2: Set up the Susceptible Contract Repository

Along with creating Diligence Fuzzing and the Scribble annotation language, ConsenSys has made public a set of repositories containing weak contracts. To maintain it easy, you’ll be utilizing considered one of these.

You’ll use a repo that comprises a weak implementation of the ERC-20 customary (the usual for cryptocurrencies and fungible tokens).

Clone this repository utilizing the next command:

$ git clone https://github.com/ConsenSys/scribble-exercise-1.git contracts-test

This repository has a folder referred to as contracts. Inside this folder, you’ll find a file referred to as vulnerableERC20.sol. As talked about earlier, this file comprises a defective implementation of the ERC-20 customary.

That is the contract you’re going to check utilizing ChatGPT and later, Diligence Fuzzing.

Step 3: Create an OpenAI Account

ChatGPT occurs to be extraordinarily proficient at all kinds of duties—together with web3 growth. (See my earlier article on creating a sensible contract with ChatGPT.) 

You may ask ChatGPT to unpack a posh blockchain idea, write sensible contracts that implement one thing you had in thoughts (as an example, a decentralized lottery), and even clarify a bit of code to you and discover potential vulnerabilities.

As you’ll have already guessed, it’s that final functionality we’re taken with for this text.

In an effort to entry ChatGPT, it’s good to create an OpenAI account. When you’ve created an account, access ChatGPT’s UI.

Speaking to ChatGPT is so simple as speaking to an individual on a messaging app. If that is your first time utilizing the device, I strongly advocate you check out some instance prompts (for instance, “Clarify quantum computing in easy phrases”) to get snug.

Step 4: Discover Vulnerabilities within the Contract 

The important thing to getting proper solutions from ChatGPT is being extraordinarily clear and detailed about your request. Paste the next immediate into the UI:

Following is a brilliant contract I've written in Solidity. As a primary step in testing its security, I need you to look at the code, and let me know of all the safety vulnerabilities that you simply discover.

```
pragma solidity ^0.6.0;

contract VulnerableToken 
  uint256 non-public _totalSupply;
  mapping (tackle => uint256) non-public _balances;
  mapping (tackle => mapping (tackle => uint256)) non-public _allowances;

  constructor() public 
    _totalSupply = 1000000;
    _balances[msg.sender] = 1000000;
  

  perform totalSupply() exterior view returns (uint256) 
    return _totalSupply;
  

  perform balanceOf(tackle _owner) exterior view returns (uint256) 
    return _balances[_owner];
  

  perform allowance(tackle _owner, tackle _spender) exterior view returns (uint256) 
    return _allowances[_owner][_spender];
  

  perform switch(tackle _to, uint256 _value) exterior returns (bool) 
    tackle from = msg.sender;
    require(_value <= _balances[from]);


    uint256 newBalanceFrom = _balances[from] - _value;
    uint256 newBalanceTo = _balances[_to] + _value;
    _balances[from] = newBalanceFrom;
    _balances[_to] = newBalanceTo;

    emit Switch(msg.sender, _to, _value);
    return true;
  

  perform approve(tackle _spender, uint256 _value) exterior returns (bool) 
    tackle proprietor = msg.sender;
    _allowances[owner][_spender] = _value;
    emit Approval(proprietor, _spender, _value);
    return true;
  

  perform transferFrom(tackle _from, tackle _to, uint256 _value) exterior returns (bool) 
    uint256 allowed = _allowances[_from][msg.sender];
    require(_value <= allowed);
    require(_value <= _balances[_from]);
    _balances[_from] -= _value;
    _balances[_to] += _value;
    _allowances[_from][msg.sender] -= _value;
    emit Switch(_from, _to, _value);
    return true;
  

  occasion Switch(tackle listed _from, tackle listed _to, uint256 _value);
  occasion Approval(tackle listed _owner, tackle listed _spender, uint256 _value);

```

ChatGPT ought to provide you with a response that appears one thing like this:

Upon reviewing the code, I've recognized a number of safety vulnerabilities and 
issues within the supplied sensible contract:

1. Lack of Entry Management: The contract doesn't implement any entry management 
mechanism, permitting anybody to name the `switch`, `approve`, and `transferFrom` 
features. Which means that anybody can switch tokens on behalf of one other 
tackle and modify allowances.

2. Integer Underflow and Overflow: The code makes use of fundamental arithmetic operations 
with none checks for potential underflow or overflow. For instance, within the 
`switch` and `transferFrom` features, the subtraction and addition of token 
balances can result in unintended conduct if the ensuing stability exceeds the 
most worth that may be saved in a `uint256`. You need to embody checks to 
stop these points.

3. Lack of Require Statements for Enter Validation: The contract lacks require 
statements to validate inputs in numerous features. As an example, within the 
`switch` and `transferFrom` features, there is no such thing as a validation to make sure 
that the recipient tackle (`_to`) just isn't the zero tackle. Moreover, 
the `approve` perform doesn't validate that the spender tackle (`_spender`) 
just isn't the zero tackle.

4. Potential Reentrancy Assaults: The contract is inclined to reentrancy 
assaults, the place a malicious contract may repeatedly name the `transferFrom` 
perform to empty the token stability of one other tackle. It is suggested to 
use the "Checks-Results-Interactions" sample or implement a reentrancy guard 
to mitigate this vulnerability.

5. Lack of Occasions for Essential State Adjustments: Whereas the contract emits 
`Switch` and `Approval` occasions, it's lacking occasions for important state 
adjustments, comparable to when the full provide is modified or token balances are 
up to date. Emitting occasions for these occasions helps present transparency and 
permits simpler monitoring of contract actions.

6. Mounted Token Provide: The contract initializes with a set token provide 
of 1,000,000 tokens, and there aren't any provisions for minting or burning 
tokens. Should you intend to have a set provide, this is probably not a vulnerability. 
Nonetheless, if you happen to plan to have a dynamic token provide, you will have to 
implement minting and burning features whereas contemplating correct entry 
management.

To reinforce the safety of your sensible contract, take into account addressing these 
vulnerabilities and implementing extra safety measures based mostly in your 
particular necessities.

ChatGPT offers you fairly a couple of suggestions! Whereas they’re largely generic, they’ll nonetheless enable you to uncover some key points. 

Beware, although—in a couple of instances, such because the tip on Entry Management, it’s plain fallacious. And imagine it or not, ChatGPT truly finally ends up lacking probably the most obtrusive error in our contract— one thing that has to do with transferring tokens to ourselves. That is an especially nuanced error that ChatGPT would solely be capable to choose up given extra prompting. (Be aware: with sufficient prompting, I used to be capable of get ChatGPT to search out the error.)

The important thing lesson right here is that ChatGPT can help in debugging, however solely as a place to begin. At the very least in the intervening time, it’s finest at figuring out errors which are syntactical, apparent, or widespread. This may be most useful for starting or intermediate builders—or possibly for drained and overworked senior ones! For extra nuanced instances, you’ll want to show to extra superior instruments.

Step 5: Signal Up for Diligence Fuzzing

The train above reveals that, at the very least on the time of writing (June 2023), ChatGPT can do so much—however just isn’t an entire substitute for a well-versed web3 developer, particularly one who can leverage state-of-the-art testing instruments.

On this step, you’re going to be taught to make use of one such device: Diligence Fuzzing.

As talked about above, fuzzing is a well-established testing method that includes sending random, typically malformed information to a specific program in an try to set off sudden behaviors or crashes. This methodology has proved exceptionally good at detecting safety points that conventional testers typically miss.

Diligence Fuzzing implements conventional fuzzing in a web3 atmosphere. The device has been created to detect important vulnerabilities and unthought-of points utilizing easy annotations in Scribble.

In an effort to use this device, it’s good to first sign up for a free account with Diligence Fuzzing. The Free Tier subscription is greater than sufficient for this tutorial, but when you end up doing numerous important testing, take into account transferring to a paid tier.

When you’ve subscribed, it’s best to see a dashboard that appears one thing like this.

Step 6: Create a Diligence Fuzzing API Key

In an effort to entry Diligence Fuzzing’s capabilities, it’s good to create an API key here.

You may identify your key something you need. Hold this key secret and protected. You’ll want it later.

Step 7: Set up Required Packages and Libraries

First, you’re going to put in Truffle, Scribble, and the Fuzzing CLI in your native machine. Run the next instructions:

$ cd contracts-test
$ npm set up
$ npm set up -g truffle eth-scribble ganache
$ pip3 set up diligence-fuzzing

Step 8: Create a Fuzzing Marketing campaign

Navigate to the contracts-test folder you cloned earlier. Open this repository in your favourite code editor (for instance, VS Code) and open the file referred to as .fuzz.yml. You need to see a line that claims key which has an empty string related to it. Change the latter together with your API key.

For Diligence Fuzzing to work accurately, it’s good to annotate our contract with guidelines you suppose must be adopted. As talked about earlier, the annotation language right here is Scribble.

For this tutorial, you don’t have to know find out how to write Scribble your self (although it’s best to take a couple of moments to get conversant in it for future tasks). The repository you downloaded comes (by default) with an annotated contract within the contracts/answer folder. You’ll be utilizing this as a base to annotate the weak contract.

Change the contents of vulnerableERC20.sol with the next:

pragma solidity ^0.6.0;

contract VulnerableToken 
  uint256 non-public _totalSupply;
  mapping (tackle => uint256) non-public _balances;
  mapping (tackle => mapping (tackle => uint256)) non-public _allowances;

  constructor() public 
    _totalSupply = 1000000;
    _balances[msg.sender] = 1000000;
  

  perform totalSupply() exterior view returns (uint256) 
    return _totalSupply;
  

  perform balanceOf(tackle _owner) exterior view returns (uint256) 
    return _balances[_owner];
  

  perform allowance(tackle _owner, tackle _spender) exterior view returns (uint256) 
    return _allowances[_owner][_spender];
  

  /// #if_succeeds msg.sender != _to ==> _balances[_to] == previous(_balances[_to]) + _value;
  /// #if_succeeds msg.sender != _to ==> _balances[msg.sender] == previous(_balances[msg.sender]) - _value;
  /// #if_succeeds msg.sender == _to ==> _balances[msg.sender] == previous(_balances[_to]);
  /// #if_succeeds previous(_balances[msg.sender]) >= _value;
  perform switch(tackle _to, uint256 _value) exterior returns (bool) 
    tackle from = msg.sender;
    require(_value <= _balances[from]);

    // _balances[from] -= _value;
    // _balances[_to] += _value;

    uint256 newBalanceFrom = _balances[from] - _value;
    uint256 newBalanceTo = _balances[_to] + _value;
    _balances[from] = newBalanceFrom;
    _balances[_to] = newBalanceTo;

    emit Switch(msg.sender, _to, _value);
    return true;
  

  perform approve(tackle _spender, uint256 _value) exterior returns (bool) 
    tackle proprietor = msg.sender;
    _allowances[owner][_spender] = _value;
    emit Approval(proprietor, _spender, _value);
    return true;
  

  perform transferFrom(tackle _from, tackle _to, uint256 _value) exterior returns (bool) 
    uint256 allowed = _allowances[_from][msg.sender];
    require(_value <= allowed);
    require(_value <= _balances[_from]);
    _balances[_from] -= _value;
    _balances[_to] += _value;
    _allowances[_from][msg.sender] -= _value;
    emit Switch(_from, _to, _value);
    return true;
  

  occasion Switch(tackle listed _from, tackle listed _to, uint256 _value);
  occasion Approval(tackle listed _owner, tackle listed _spender, uint256 _value);

The feedback in traces 25-28 symbolize annotations written in Scribble. Even if you happen to’ve by no means been uncovered to this language earlier than, the contents must be fairly self-explanatory.

You’re all set. Begin fuzzing by operating the next command:

make fuzz

This could immediate your terminal to run a sequence of instructions.

Discover that the output additionally comprises a URL that gives extra details about your marketing campaign.

This marketing campaign takes roughly 5 minutes to run. However as soon as it’s carried out, you get an in depth report about its findings, together with vulnerabilities.

Discover that below the Property part, the report states that one consumer assertion has failed. Upon nearer inspection, you see that it’s this one:

/// #if_succeeds msg.sender == _to ==> _balances[msg.sender] == previous(_balances[_to]);

You may take a look at the transaction particulars to search out out extra.

In essence, it’s telling you there’s a vulnerability when the sender and the receiver are the identical! 

Diligence Fuzzing does a fantastic job at a particular process—it routinely creates all kinds of inputs to generate unit assessments and system assessments that discover bugs and vulnerabilities in your sensible contract. 

Create a ChatGPT and Diligence Plugins

You might take this even additional by combining the facility of the 2 testing instruments utilizing a ChatGPT plugin. This plugin may:

  • Execute the code to initialize a Diligence Fuzzing marketing campaign towards the contract

  • Parse the outcomes and use them to do one thing fascinating: instantly rewrite the code, provide you with solutions for enhancements, routinely rewrite and retest the code till there aren’t any discovered defects, and extra

You should utilize this starting point to create the plugin—or within the spirit of this text, even ask ChatGPT to put in writing the plugin for/with you!

Conclusion

On this tutorial, you explored two very highly effective instruments for sensible contract testing and safety audits.

ChatGPT is phenomenal at discovering bugs with out requiring any significant extra context past the supply code. This makes it a superb first place to start out your testing journey.

Nonetheless, as you’ve seen, ChatGPT tends to go off observe and infrequently produces fallacious solutions. In its place for a whole audit, it has an extended technique to go. That is the place a device like Diligence Fuzzing can shine. It could require a little bit extra guide intervention, but it surely’s far more complete when it comes to protection.

Have a extremely nice day!