As seen in the example above, ChatGPT replicated the typical software engineer’s behaviour: coming up with the solutions before writing the tests. Also, when we tried a more iterative approach, ChatGPT came up with the final solution very quickly ignoring the red-green-blue cycle.
The test-driven approach might have failed, but we refused to give up. We tried to write only the unit tests and let ChatGPT create the production code. We started with the straightforward task of creating a converter that converts Roman numerals into Integers. The result was great, with all tests passing, and the code was quite good, close to what an average engineer would produce.
On the other hand, do we really care about the code quality if everything is working fine?
Here is a thought experiment…
We let ChatGPT create whatever good or crappy code it generates as long as all of our test passes. If any bug occurs, we will need to write new tests, and then we let ChatGPT re-create the code. If all tests pass, the bug is fixed.
However, for more complex scenarios i.e. multi-threading, the above approach will probably fail. In such cases, we could still leverage ChatGPT to help us understand and improve upon the buggy code parts with a Refactor-By-Demand approach.
It sounds wrong to us. But, it might be our fear of losing our jobs that make this a daring thesis. Our experience with developing applications tells us that software engineering is much more than simply writing code on a green field.