GitLab and go get

(updated )

Source

I was trying to run go get on a git repository hosted on a private GitLab instance. This repository was stored under a sub-group. I was working from a non-default branch. Since it is a monorepo the module/package was in a subdirectory. So a combination of four factors complicated matters for me.

Easiest Solution

These are the first few things we must clarify,

  • go get, go mod tidy, and go mod download can use https or ssh to fetch Go modules from a git server (like GitLab or GitHub)
  • go will by default fetch modules without first authenticating
  • go needs to fetch subgroups depending on the Go module's name. For example, if the module is called gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo, go will need to fetch all subgroups of mygroup.
  • go needs to fetch tags and commits from the git server
  • GitLab has designed private subgroups to not be accessible by unauthenticated users. This includes repositories, tags, commits, etc.

The easiest way to deal with this situation is a combination of,

  • go must use https not ssh to work with GitLab
  • go must use .netrc stored in the user's home directory
  • Always use either tag or commit to specify version; never use branch name
  • Always use go get because it also updates go.mod and go.sum files
  • Environment variables GOPROXY and GOPRIVATE must be configured appropriately

Big thanks to Jonathan Hall for this.

Setup

go will prefer https over ssh (from what I have observed). The next step is to create a GitLab API read-only (because go just needs to read) personal access token. Create ~/.netrc as below (change values per your environment),

$ export GITLAB_HOSTNAME='<gitlab.my-internal-domain.com>'
$ export GITLAB_USERNAME='<your_username>'
$ export GITLAB_API_TOKEN='<your_read_only_token>'
$ printf 'machine %s\n    login %s\n    password %s\n\n' $GITLAB_HOSTNAME $GITLAB_USERNAME $GITLAB_API_TOKEN >> ~/.netrc

Read more about environment variables GOPROXY and GOPRIVATE.

go get

With the above setup complete, you can start using various ways to get modules,

$ go get gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo

The above will get the latest from the default branch. If you want to specify a different tag or commit ID, append @ID (where ID is one of tag or commit),

$ go get gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo@tag
$ go get gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo@commit-id

I have not had success with using branch names. I always get error message, invalid version: disallowed version string.

Alternative But Finicky Solution

This solution tries to get you going with ssh instead of https.

Let's say the repository URL I was working with was gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo. The go.mod in that repo was

module gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo/src/mygopackage
go 1.20

I was trying go get gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo/src/mygopackage in a separate package, essentially using mygopackage as a dependency. I got this error,

go: module gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo/src/mygopackage: git ls-remote -q origin in /Users/aikchar/go/pkg/mod/cache/vcs/27f0a3bd58674de4db4e2a3e38601eecc893ec233ba6addcf6eeb2b43f240ef9: exit status 128:
    fatal: could not read Username for 'https://gitlab.my-internal-domain.com': terminal prompts disabled
Confirm the import path was entered correctly.
If this is a private repository, see https://golang.org/doc/faq#git_https for additional information.

I figured I made a mistake. I needed to add .git to my go.mod in myrepo so gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo/src/mygopackage becomes gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo.git/src/mygopackage.

After fixing go.mod in gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo.git/src/mygopackage, it was,

module gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo.git/src/mygopackage
go 1.20

I ran go get gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo.git/src/mygopackage and got this error,

go: downloading gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo.git v0.0.0-20220303202727-f334bf068bdb
go: gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo.git/src/mygopackage: gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo.git@v0.0.0-20220303202727-f334bf068bdb: verifying module: gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo.git@v0.0.0-20220303202727-f334bf068bdb: reading https://sum.golang.org/lookup/gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo.git@v0.0.0-20220303202727-f334bf068bdb: 410 Gone
    server response:
    not found: gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo.git@v0.0.0-20220303202727-f334bf068bdb: invalid version: git ls-remote -q origin in /tmp/gopath/pkg/mod/cache/vcs/b9eaa09fa03cd0f3139730aa61579fcd85cd6fd748674c4aa228cd6a57e99b31: exit status 128:
        fatal: unable to look up gitlab.my-internal-domain.com (port 9418) (Name or service not known)

I added @commit-id to go get like so: go get gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo.git/src/mygopackage@e091a5ea01d7 and got this error,

gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo.git/src/mygopackage@v0.0.0-20220531210353-e091a5ea01d7: verifying module: gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo.git/src/mygopackage@v0.0.0-20220531210353-e091a5ea01d7: reading https://sum.golang.org/lookup/gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo.git/src/mygopackage@v0.0.0-20220531210353-e091a5ea01d7: 410 Gone
    server response:
    not found: gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo.git/src/mygopackage@v0.0.0-20220531210353-e091a5ea01d7: invalid version: git ls-remote -q origin in /tmp/gopath/pkg/mod/cache/vcs/b9eaa09fa03cd0f3139730aa61579fcd85cd6fd748674c4aa228cd6a57e99b31: exit status 128:
        fatal: unable to look up gitlab.my-internal-domain.com (port 9418) (Name or service not known)

I added environment variable GOPRIVATE and ran this command: GOPRIVATE=gitlab.my-internal-domain.com go get gitlab.my-internal-domain.com/mygroup/mysubgroup/myrepo.git/src/mygopackage@e091a5ea01d7 and IT WORKED!

go: added gitlab.my-internal-domain.commygroup/mysubgroup/myrepo.git/src/mygopackage v0.0.0-20220531210353-e091a5ea01d7

Basically, to counter the four factors I mentioned at the top, I needed to do this,

  • Make sure .git is part of the module name and during import or go get. This is not always possible to do.
  • Add commit-id (NOT branch name) at the end of go get when getting from non-default branch
  • Since GitLab is private and not publically accessible, set the GOPRIVATE environment variable to the right hostname

Let's add one more factor: the GitLab server is not reachable over port 22 for ssh but instead uses a custom port. How about one more factor i.e. anonymous, non-logged in read access is not allowed; you need to login to even read a repository.

The symptom of the second new factor, no anonymous read allowed, is an error message like fatal: could not read Username for 'https://gitlab.my-internal-domain.com': terminal prompts disabled.

The fix for this factor is to use ssh instead of http or https to connect to GitLab. This way we can setup ssh keys to work seamlessly without needing a human to enter username and password to login. We will use git config's insteadOf option (thanks to Set GOPRIVATE to match your github organization).

$ git config --global url."ssh://git@gitlab.my-internal-domain.com".insteadOf "https://gitlab.my-internal-domain.com"

Remember the other factor we wanted to consider i.e. ssh is accessible on a non-standard port e.g. 2222? Modify the config to include the port,

$ git config --global url."ssh://git@gitlab.my-internal-domain.com:2222".insteadOf "https://gitlab.my-internal-domain.com"

One more thing to possibly cause issues is that regardless of the configrations discussed above, go get will use http(s) instead of ssh. I couldn't find a good solution for it. A workaround is to always use go mod tidy instead. Another less-than-optimial workaround is to modify the module name by appending .git. This has additional effects which may make this impossible in most (in my opinion) cases. This ties into the next point as well.

You may need to use both require and replace in go.mod. I had a situation where the module I wanted to pull in was named gitlab.my-internal-domain.com/group/subgroup/module and the git repository was in a sub-group like gitlab.my-internal-domain.com:2222/group/subgroup/module.git. When I tried go get or even go mod tidy (with debug flag -x) it showed that instead of pulling module.git it was pulling subgroup.git. That was a real surprise. The fix for it was to add the following lines in go.mod (v0.4.7 is the version/tag of the module I needed, change it to suit your situation),

require gitlab.my-internal-domain.com/group/subgroup/module v0.4.7
replace gitlab.my-internal-domain.com/group/subgroup/module => gitlab.my-internal-domain.com/group/subgroup/module.git v0.4.7

What the above does is when you run go mod tidy it will treat module.git as the git respository instead of assuming subgroup was the git repository. I can't find the Stack Overflow post anymore but it said that GitLab doesn't give visibility to repositories in subgroups to go get so when you have require gitlab.my-internal-domain.com/group/subgroup/module in go.mod then go get assumes submodule.git is the repository and module/ is the directory/path to the go.mod in it. But with replace and go mod tidy it all works like it should. To repeat, in this situation, always use go mod tidy because go get will not work (it didn't for me with Go 1.20.3).