TDD & Gen AI: A Perfect Pairing

Bouke's Background and Role at CINQ

Bouke Nijhuis introduced himself and his journey from a Java developer to the CTO of CINQ. As he explained, “I started out as a Java developer at CINQ about 10 years ago. Actually, in 10 days, it will be exactly 10 years ago.” CINQ is a consultancy company based in Amsterdam with specializations in Data, DevOps, and Development. Bouke shared that in his role as CTO, he oversees the development unit, focusing on Java, Kotlin, Angular, and React technologies. He also enjoys international speaking engagements, having started in 2019, which allows him to share his knowledge and engage in discussions with the tech community.

Introduction to TDD

Test-Driven Development (TDD) is a software development approach where tests are written before the code itself. Bouke shared his initial experience with TDD, stating, “I learned about TDD, I would say about 15 years ago, and I think it’s pretty hard to do it the official way. I think I’m doing a little bit in the middle.” Despite not being a diehard TDD practitioner, Bouke appreciates TDD for helping him develop faster and better. The benefits of TDD include ensuring code correctness, promoting better design, and providing a safety net during refactoring.

AI Coding Assistants in Bouke's Workflow

Bouke discussed the various AI coding assistants he uses, such as GitHub Copilot, JetBrains AI Assistant, and ChatGPT. These tools integrate seamlessly into his daily workflow, helping him generate code and identify bugs. Bouke emphasized the value of these assistants: “I use them to just generate code for me. I use the, if I just press enter, it does a proposal, I like that, but I like even better that you can chat with it, and then you can reason about the code and ask it to generate something and to improve upon it.” This interaction is akin to having a virtual pair programmer, making AI assistants an invaluable part of his development process.

Generating Tests with AI

The process of generating tests using AI assistants involves providing existing code and asking the AI to create corresponding test cases. Bouke highlighted the advantages and limitations of this approach: “I ask it to generate test cases, but that’s only when I already have an implementation.” While AI-generated tests can cover happy paths and some edge cases, it’s crucial to manually review these tests to ensure they are comprehensive and correct. This manual validation helps catch any inaccuracies or omissions that the AI might overlook.

Generating Code from Tests

A particularly intriguing concept discussed in the episode was generating code from human-written tests using AI. Bouke’s research in this area led to his talk “TDD and Gen AI: A Perfect Pairing.” He explained the motivation behind this idea: “Can you also do it the other way around? You as a human, can you write a test? And then can you ask the AI to come up with an implementation?” This exploration resulted in Bouke creating a tool to automate this process, allowing developers to write tests and have AI generate the corresponding code, which can then be validated against the tests.

Practical Implementation and Demos

Bouke provided a step-by-step description of how his tool works. Starting with simple problems like odd/even tests and prime number generation, the tool automates the process of generating code from tests. Bouke introduced a Maven plugin to handle more complex frameworks like Spring Boot, demonstrating its capabilities with a Hello World example and a more intricate age calculation endpoint. This progression from simple to complex scenarios showcases the tool’s versatility and practical applications in real-world development.

Challenges and Solutions

Using AI to generate code presents several challenges, such as non-compiling code and incorrect implementations. Bouke addressed these issues with a feedback loop mechanism: “If you would summarize it, I use it like a virtual pair programmer.” This loop involves providing the AI with test results, prompting it to refine the generated code until all tests pass. Comprehensive testing, including performance and security tests, ensures the robustness of the final implementation, highlighting the importance of detailed and thorough testing.

The Future of AI and TDD

Bouke envisions a future where tests become the primary artifact, and code generation is fully automated. He shared his thoughts: “If we trust the generated code from the AI, and that means if we trust our test, then the test will obviously be more important. We don’t care about the implementation anymore.” This shift could significantly impact software development practices, emphasizing the quality and comprehensiveness of tests. Ethical considerations and the reliability of AI-generated code will play a crucial role in this evolution, ensuring that AI tools are used responsibly and effectively.

Bouke's Experiences with Different AI Models

Throughout his research, Bouke experimented with various AI models, both local and cloud-based, such as Llama 3, ChatGPT, and Claude 3.5. He shared his insights into their performance and capabilities: “The Llama 3 models are the best one for my use case. A few months back, I switched to Llama 3 and nowadays I’m using Llama 3.1 and that works really well.” This comparison highlights the strengths and weaknesses of different models, helping developers choose the most suitable tools for their needs.

Summary

In this episode, Bouke Nijhuis shared his journey from a Java developer to the CTO of CINQ, his experiences with TDD, and the role of AI in enhancing development workflows. Key takeaways for developers include the importance of TDD, integrating AI tools, and staying updated with AI advancements. Bouke’s vision of a future where tests drive code generation underscores the potential of AI-assisted development. Developers are encouraged to experiment with AI tools and adopt TDD practices for better software development, ensuring they remain at the forefront of technological innovation.