5 Lessons Learned Using GitHub Copilot

At CloudFuel, we make it our mission to explore and harness the latest tools to make your cloud migrations journeys even better. That’s why Marnik Van Hileghem, one of our DevOps Engineers, has been trying out generative (coding) assistants in the last few months. In this blog, we’ll share five tips for working with tools like GitHub Copilot as a developer. Let’s dive in!

Lesson #1: Generated Code Looks Good – And That’s a Bad Thing

Let’s start with the most important lesson. Tools like GitHub Copilot are very capable at writing good-looking code, but that makes it even more challenging to spot mistakes when they inevitably occur. It can be tempting to use these tools to generate code for languages or libraries you’re not entirely familiar with, but you’ll need an expert’s eye to review the output and avoid bugs or vulnerabilities.

For example, check out the code snippet from our colleague Marnik below. Back in April, he asked ChatGPT to generate some initially promising deployment code, but it didn’t respond correctly. Can you spot the mistake?

In this case, ChatGPT conjured up a resource out of thin air, making the deployment code unusable. It tried to turn on container insights for AKS, but this simply doesn’t exist. While this is a rather basic example, keep in mind that generated code can contain more subtle errors. As we’ll discuss below, GitHub Copilot is less likely to ‘hallucinate’ resources since it has been specifically trained on millions of repositories to generate code, and it can use your project’s context because it is integrated with Visual Studio Code.

Lesson #2: Use the Right Tool for the Right Job

Choosing the right tool for the task is crucial. Let’s discuss what GitHub Copilot, Copilot Chat, and ChatGPT are best at, and how they differ in their approach.

GitHub Copilot is an advanced autocomplete tool for your code. It will offer suggestions, but because you can’t hold a conversation with it, you’ll have to make do with its limited set of options (which you can cycle through by pressing Tab). Since Microsoft owns GitHub, Copilot is seamlessly integrated with Visual Studio and Visual Studio Code. This means that it can use the context of the files you currently have open, which can be both an advantage and disadvantage. GitHub Copilot is perfect for boilerplate and recurring code snippets, but it’s not quite the intelligent pair programmer you might expect for general purpose coding.

 

A pair programmer is someone you can ask for feedback and cooperate with, which is why people often turn to ChatGPT as well when it comes to generating code. Unlike GitHub Copilot, ChatGPT offers a conversational interface, allowing you to iterate on its output by giving feedback and asking for corrections. Its chat history is also a significant advantage, but as we’ll discuss later, this does have its limits.

Because ChatGPT is a standalone tool, it is not integrated with your code base, so it only knows what you feed it. At the same time, this lack of context makes it ideally suited for a wide range of code-related tasks: you can use it to write project documentation, (re)structure documents, compose emails to project stakeholders, create tests, and more. For example, this conversation shows how it can help you create and invoice table for PostgreSQL.

Finally, Copilot Chat is the best of both worlds. It strikes a balance between ChatGPT and GitHub Copilot by letting you ask questions in natural language, both in-line and in a chat window. It serves as an excellent learning tool for understanding code, writing comments, and debugging. You can also use it to generate project documentation automatically in the markdown format.

Lesson #3: Don’t Be Afraid to Rinse and Repeat

An important part of holding a conversation is remembering what was said earlier. Conversational tools can iterate their output based on your feedback, but keep in mind that they have a limited ‘memory’, which is referred to as a context window. That’s why we recommend keeping your code snippets small. Don’t ask it to generate an entire application in one go but start with an outline and go through what you need piece by piece.

An even better idea is to use Copilot Chat, since it mitigates this constraint entirely by accounting for your currently opened files. With the upcoming Copilot Enterprise, it will even be able to keep the entire organisation’s codebase into account and provide detailed suggestions. For more technical details, check out GitHub’s original announcement and the improvements that were made as part of Copilot Enterprise.

Lesson #4: Don’t Be Afraid to Take Over from Your Copilot

One of the most underrated skills when using a conversational AI assistant is knowing when to cut your losses if the output is not according to your expectations. Reflect critically on if it is faster to explain to your assistant how to fix the problem, or to simply fix the problem yourself.

The pro tip, however, is to then share and review your definitive version with your assistant instead of simply moving on to the next part. Questions like “I have tweaked your code slightly to make it more performant, what do you think of this updated version?” will significantly improve the quality of later answers. Both Copilot Chat and ChatGPT also allow you to give immediate feedback on the quality of responses through a thumbs up or down.

Lesson #5: Embrace Experimentation

Generative coding assistants are new, and rapidly evolving as the technology matures. For example, GitHub Copilot feel like an amazing timesaver at the moment, but the new Copilot Chat is already miles ahead in terms of usability, and it will only improve as time goes on. Don’t hesitate to explore, experiment, and find what best matches your technology stack and preferred languages. Every tool has its strengths and weaknesses, and only by experimenting can you uncover the perfect fit for your specific needs. They can be powerful, but you need to figure out how to use them correctly.

Want to know more about how the latest technologies can help you take your journey to the cloud to the next level? Contact us today, and we’ll show you what we’re working on and how these can help your next project.

Disclaimer: Our assessment is based on the current state of the generative coding market, which is evolving rapidly. At the time of writing, we used Copilot v1.78, the public preview of Copilot Chat, and ChatGPT Plus based on GPT 4.