Defects - Fixing AI code using local models
This article was written without AI help. The code however was 100% generated by AI.
In my previous article I showed how machine-assisted code translation was able to translate a python-based tool to Go. Of course, an AI stating "I'm done and everything tested fine!" is to be looked at a bit skeptical. Why? Because it confabulates at unexpected times, but more importantly, ...
Seeing is believing
AI has no clue about the subject matter
Computer generated code does not have any domain expertise built in. My model doesn't know what a valid Wireguard configuration file is and the tests it wrote do not involve external systems, such as actual servers trying to "nmcli con import type wireguard file suscpdx1dc10001.conf".
The real world is different from an optimal gradient descent.The generated code had two issues. And I created one, too.
- A trailing "," in the AllowedIPs field - a classic programmer error, off-by-one. A logic Fix: Ensured AllowedIPs are correctly joined without trailing commas, ensuring the configurations are valid for the WireGuard tool.
- The models assumption that PersistentKeepAlive applies to the Interface section. Fix: Removed PersistentKeepalive from the [Interface] (server) section of the configuration while retaining it for the [Peer] sections.
Both of these bugs were pretty simple to discover and fix. Finding these was simple: The import of the generated files on the destination systems failed immediately.
Once the build was complete, all there was to do was to run Ansible to deploy the generated files. The script deployed perfectly fine, except on one server the wg-admin interface wasn't coming up. This wasn't a code issue, though. This mistake was mine. The UDP listen port I configured was used by another Wireguard interface I created previously, using a different overlay network. Once that was fixed, all nodes communicated on all interfaces.
The moral of the story is: AI makes mistakes and so do I.
Do you want to learn from my mistakes? I provide strategic and technical AI consulting. You can book an initial meeting with me right here: https://calendly.com/tggai