In-Depth

Taking My New GitHub Copilot 'AI Pair Programmer' for a Spin

• By David Ramel
• 07/07/2022

I tried to get GitHub Copilot to write this article for me, but it couldn't do it. It can do lots of other things. Some are pretty cool.

If you haven't heard of GitHub Copilot, it's a cutting-edge coding assistant tool similar to IntelliSense/IntelliCode that can do a lot more than suggest how to complete your lines of code. It's based on advanced OpenAI Codex technology that allows natural language interaction. It just recently became generally available for $10 per month (or $100 per year). It's also available in a 60-day trial, which I used.

I'm no coder, but I figured I could install it and try a few things. Here's how that went.

Setup, Authentication, Pairing ... and Mistakes
One reason I'm not a coder is that I'm impatient and impulsive and don't follow directions. I just dive in and try things until something works. (If you're like that -- probably not a coder, then -- this article can serve as a lesson in what not to do in order to get started.)

So I didn't follow the instructions outlined in "Getting started with GitHub Copilot in Visual Studio Code" documentation that says to go to the VS Code Marketplace and search for the extension and install it. I didn't read that documentation. I just installed the extension from within the editor.

Upon trying to use it, I got a notification to sign in to access GitHub Copilot.

I didn't really expect that to work, but it appeared to.

I was notified "The extension 'GitHub Copilot' wants to sign in using GitHub," but the process timed out, generating this message: "You have not yet finished authorizing this extension to use GitHub. Would you like to keep trying?"

I managed to work my way to an authorization screen.

To finish authenticating, I had to navigate to GitHub and paste in the one-time code generated from the authorization page.

That took be to GitHub again where I entered the code.

I was finally able to authorize GitHub for VS Code (with GitHub reminding me that I live in the Montana town of Libby).

And I was notified that I was all set.

Except I wasn't. Upon trying to use the tool, I had to allow the extension to open up the authentication URI.

Then I was notified my experience wasn't fully configured and I needed to complete my setup.

I went to Settings and reached an "Enable GitHub Copilot" screen.

The 60-Day Trial
After which I reached the 60-day trial page.

And confirmed my payment details.

Some preferences were next, where I could indicate if I wanted GitHub to allow or block suggestions matching public code, and whether I agreed to allow GitHub to use my code snippets for product improvements. I obligingly agreed to the latter, though I can absolutely guarantee no one wants to use my code snippets to improve their product.

And was then presented with yet another step: pairing with my pair programmer.

After being notified (with falling, colored confetti, no less) that I had successfully paired, I got a message to install the extension. I had done that first thing, remember, so the documentation informed me in that case I just had to restart the editor.

Finally Up and Running
I did that, and my pair programmer and I were finally up and running! I had to jump through some extra hoops on account of not following setup directions, but it still felt like that even the official process took a lot of steps.

Anyway, I created a JavaScript file and asked my pair programmer to generate a function to add two numbers together. You do that by typing in a request in a commented-out line (preceded with //) and clicking Enter. So the JavaScript file was empty except for this line:

//Generate a function to add two numbers together

Clicking Enter caused the tool to suggest some grayed-out code that could be accepted by clicking Tab. That made Copilot churn for a couple seconds (the little GitHub Copilot icon in the bottom right spins).

Copilot activated the line suggestion -- turning the grayed-out text into properly formatted and colored code -- and then stood by awaiting more instructions, as indicated by a red tilde.

Repeatedly clicking Enter brought in the whole function, complete with a helpful comment that the end of the function had been reached.

Upon that initial success, I got frisky and decided to ask Copilot to compute the value of pi (π). As I started typing "Compute pi ..." I got a suggestion to use the Leibniz formula for that pi computation. When I started typing "Compute the value of pi ..." Copilot suggested "... to the Nth digit."

You can supposedly cycle through the suggestions with the Windows key combinations (with Mac equivalents) of Alt + ] (forward) or Alt + ] (backward). I couldn't get the suggestions to cycle through various options, nor could I get Copilot to compute the value of pi at all, in any shape or form, even when accepting suggestions. Update: The day after this article was published I tried the pi calculation again and it worked perfectly. Go figure. I used the same exact command but this time that resulted in this:

//Compute the value of pi
//The value of pi is 3.141592653589793

I tried to generate a function to compute the area of a circle and discovered that if I just kept clicking Tab, Copilot just kept generating comments explaining aspects of the equation, eventually resorting to repeating itself until I clicked Enter and completed the function code.

I asked Copilot to spin up a new HTTP server and it did this:

var http = require('http'); //import http module from node js

I tried a series of "Create a ..." commands to see what Copilot offered up as suggestions. The suggestions for different comments are shown below, with the suggestions enclosed in parentheses.

Create a new (copilot object)
Create a function (that takes a string and returns the string in reversed order)
Create a table (of contents for the copilot test)

Regarding the latter, all that happened after hitting Tab to accept the suggestion is that Copilot generated this comment:

//Create a table of contents for the copilot test. This will be used to generate the copilot test. The table of contents will be generated from the copilot test.

When I kept pressing Tab it just started to repeat comments.

I tried the // Express server on port 3000 command from official documentation and found out that Copilot also started to repeat itself when I just kept clicking Enter/Tab combinations. It first generated more than 75 lines of code that looked like it may be a complete and correct program. But when I kept clicking Enter/Tab combinations, it eventually started to repeat itself in the Routes section of the code.

Explain this Code
I also tried the "explain this code" functionality provided in the companion GitHub Copilot Labs (preview) extension. As its name indicates, this experimental tool can take a chunk of code and explain in natural language what it does.

I didn't have to jump through any hoops to install that tool; I did it quickly and easily from within the editor -- no context switching or restarts required.

In the sidebar, it provides two options, Explain and Language Translation.

My first attempt at Explain (highlighting some code and clicking the Ask Copilot button) produced an error message: "Error generating Copilot response, please try again. Changes in the prompt or stop sequence might fix this."

I don't know what "changes in the prompt" or the "stop sequence" even mean, but reloading VS Code and signing back in to use Copilot worked.

I grabbed a chunk of code off the Mozilla site and asked Copilot what it meant.

It dutifully reported what each line does ("creates a variable ..."), but it didn't tell me what the code does -- its overall effect or purpose -- as I expected. But that might happen with other code chunks. Or it might be too much to ask at this point. Or I was expecting too much from Explain (I didn't read the full documentation), misunderstanding its purpose.

I asked Copilot to explain 88 lines of code generated by the Express server command detailed above and all it did was print out 82 numbered lines that mostly just duplicated the source code lines. Again, we're talking about experimental tech in a preview tool here.

So that's a clueless noob's initial experience with GitHub Copilot. Mileage surely varies, and real developers have been amazed at what the tool can help them do.

For example, the project's web site features this quote from Mike Krieger, co-founder of Instagram: "This is the single most mind-blowing application of machine learning I've ever seen."

I'd like to hear from some real developers to see if they think the same thing. Please weigh in with your experience in the comments section.