Are there tools yet that put the compiler in the loop? Maybe even a check-compile-test loop that bails on the first failure and then tries to refine it in the background based on what failed?
Yes. ChatGPT Code Interpreter mode does exactly this - it writes code, runs it through a Python interpreter, then if there are any errors it rewrites the code and runs it until it works.
It's a bit of fools-errand because they train on information which is no longer valid and will get stuck if you don't inform them. For instance GPT cannot write a connection to an openai endpoint because the API was upgraded to 1.0 and broke compatibility with all the code it learned from
