Setting up Jfrog Artifactory as a go proxy

In many environments, it isn’t really feasible to take build-time or even run-time dependencies on public repositories such as npmjs, nuget, or maven central,generally due to concerns about security, performance, availability, or all of the above.

Historically the story for go has been slightly different than for these other platforms, preferring to fetch dependencies directly from the source - often GitHub - rather than create a centralised registry of go packages. For the last few versions however, the default behaviour of go get, go build and friends has been to use a publicly hosted proxy proxy.golang.org when fetching dependencies.

From an end user point of view, the golang proxy is functionally similar to those centralised registries and many organisations will want to create a local proxy using JFrog Artifactory, SonaType Nexus, or similar products built into CI/CD platforms such as GitHub Packages or the Gitlab Package Registry.

I managed to get this working using Artifactory, but a couple of the steps were non-obvious, so I thought I’d write them down here for future reference. I am using the Artifactory Free Tier on AWS, but I don’t suppose this makes much difference.

In summary, you have to create a remote repository pointing to the golang proxy, a virtual repository pointing to your remote repository, and conigure your client correctly. More detailed instructions follow.

A sample application

I created a small application with a couple of external dependencies, which you can clone from https://github.com/gavincampbell-dev-example-repos/gcgreeter if you want to play along. It is “heavily inspired” by the example in the go tutorial.

Step 1 Create a remote repository pointing to proxy.golang.org

If you don’t have a “project” created in Artifactory you’ll have to create one for the subsequent steps to work.

The first step is to create the remote repository, which will point at the golang proxy. Everything here is left at its default value, the only items I set were the repository key and the remote url.

Settings for the remote repository

Settings for the remote repository

This is where I ran into the first gotcha, the “Set Me Up” dialog looks as if it will let you use go get to fetch from this repository, but this turns out not to be the case.

“Set Me Up”. And no, that’s not a working API key.

“Set Me Up”. And no, that’s not a working API key.

Following these steps will lead to the following error:


[liveuser@localhost-live gcgreeter]$ go get

go: github.com/fsnotify/fsnotify@v1.5.1: reading https://gavin%40gavincampbell.
dev:xxxxx@washbrook.jfrog.io/artifactory/api/go/gcd-goproxy/github.com/fsnotify/fsnotify/@v/v1.
5.1.mod:400 Bad Request

It turns out, quite a long way down the instructions page, that

Artifactory only supports resolution of Go packages from virtual Go repositories. To resolve Go from other local or remote Go repositories, you need to aggregate them in a virtual Go repository.

Step 2: Create a virtual repo pointing to your remote repo

This is the configuration screen for the virtual repository.

Setting up the virtual repository in Artifactory

Setting up the virtual repository in Artifactory

Again, everything here is the default, I just supplied the repository key. There’s another gotcha here though, my gcd-goproxyrepo is selected, but it hasn’t been included in the list of repositories backing the virtual repository.

The symptom of this is a 404 error trying to retrieve packages

$ go get
go: github.com/fsnotify/fsnotify@v1.5.1: reading https://gavin%40gavincampbell.dev:xxxxx@washbrook.jfrog.io/artifactory/api/go/gcd-golang-virtual/github.com/fsnotify/fsnotify/@v/v1.5.1.mod: 404 Not Found

This wasn’t obvious to me, but you have to move the name of your remote repository into the “Included” column for this to work.

Setting up the virtual repository in Artifactory, second attempt

Setting up the virtual repository in Artifactory, second attempt

If you have any private go repositories set up in artifactory, you can add them here too so that all your requests go to the same endpoint.

Step 3: Using the virtual repository

To test this, I made a few changes in the /etc/hosts file:

$ cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
0.0.0.0     proxy.golang.org
0.0.0.0     github.com
0.0.0.0     sum.golang.org
0.0.0.0     index.golang.org

This will make requests to all of these urls fail- if you are following along at home you might want to clone the repo before you do this.

Next, I emptied the module cache to make sure we weren’t cheating.

$ go clean -modcache

Assuming $GOPROXY is not set (unset $GOPROXY to make sure), go will attempt to download dependencies from proxy.golang.org and this will fail:

$ go get
go: github.com/fsnotify/fsnotify@v1.5.1: Get "https://proxy.golang.org/github.com/fsnotify/fsnotify/@v/v1.5.1.mod": dial tcp 0.0.0.0:443: connect: connection refused

Finally, we can return to the “Set Me Up” dialog in Artifactory and copy the correct configuration.

“Set Me Up”. Still not a working API key.

“Set Me Up”. Still not a working API key.

Having done this, we are able to build and run the application successfully.

export GOPROXY="https://gavin@gavincampbell.dev:AKCp8k...ssg@washbrook.jfrog.io/artifactory/api/go/gcd-golang-virtual"

$ go build
go: downloading rsc.io/quote v1.5.2
go: downloading github.com/spf13/cobra v1.3.0
go: downloading rsc.io/sampler v1.3.0
go: downloading golang.org/x/text v0.3.7
go: downloading github.com/spf13/pflag v1.0.5

$ ./gcgreeter greet
Hello, world.