Issue
I created a Freestyle Project on jenkins to deploy spring boot application using Docker Build and Publish. Moreover I created a secure private registry on same machine using rel="nofollow noreferrer">https://www.codeproject.com/Articles/1263817/How-to-Setup-Our-Own-Private-Docker-Registry and https://www.codeproject.com/Articles/1263831/How-to-secure-your-private-Docker-Registry.
I got the following error from Jenkins:
received unexpected HTTP status: 500 Internal Server Error
Build step 'Docker Build and Publish' marked build as failure
Finished: FAILURE
The registry logs are as following:
{"log":"time=\"2022-02-08T13:24:35.3479554Z\" level=error msg=\"response completed with error\" err.code=unknown err.message=\"unknown error\" go.version=go1.16.13 http.request.contenttype=\"application/vnd.docker.distribution.manifest.v2+json\" http.request.host=hub.docker http.request.id=798ba3fb-0471-4e76-b8c6-fa04b7cd5d6a http.request.method=PUT http.request.remoteaddr=\"xxx.xxx.xxx.xxx\" http.request.uri=\"/v2/docker-jenkins-integration-sample/manifests/latest\" http.request.useragent=\"docker/19.03.5 go/go1.12.12 git-commit/2ee0c57608 os/windows arch/amd64 UpstreamClient(Docker-Client/19.03.5 \\(windows\\))\" http.response.contenttype=\"application/json; charset=utf-8\" http.response.duration=\"999.5µs\" http.response.status=500 http.response.written=523 vars.name=docker-jenkins-integration-sample vars.reference=latest \n","stream":"stderr","time":"2022-02-08T13:24:35.8661829Z"}
{"log":"time=\"2022-02-08T13:24:35.3480004Z\" level=error msg=\"response completed with error\" err.code=unknown err.message=\"invalid URL on layer\" go.version=go1.16.13 http.request.contenttype=\"application/vnd.docker.distribution.manifest.v2+json\" http.request.host=hub.docker http.request.id=798ba3fb-0471-4e76-b8c6-fa04b7cd5d6a http.request.method=PUT http.request.remoteaddr=\"xxx.xxx.xxx.xxx\" http.request.uri=\"/v2/docker-jenkins-integration-sample/manifests/latest\" http.request.useragent=\"docker/19.03.5 go/go1.12.12 git-commit/2ee0c57608 os/windows arch/amd64 UpstreamClient(Docker-Client/19.03.5 \\(windows\\))\" http.response.contenttype=\"application/json; charset=utf-8\" http.response.duration=1.0451ms http.response.status=500 http.response.written=523 vars.name=docker-jenkins-integration-sample vars.reference=latest \n","stream":"stderr","time":"2022-02-08T13:24:35.8661829Z"}
{"log":"time=\"2022-02-08T13:24:35.3480222Z\" level=error msg=\"response completed with error\" err.code=\"manifest blob unknown\" err.detail=sha256:4612f6d0b889cad0ed0292fae3a0b0c8a9e49aff6dea8eb049b2386d9b07986f err.message=\"blob unknown to registry\" go.version=go1.16.13 http.request.contenttype=\"application/vnd.docker.distribution.manifest.v2+json\" http.request.host=hub.docker http.request.id=798ba3fb-0471-4e76-b8c6-fa04b7cd5d6a http.request.method=PUT http.request.remoteaddr=\"192.168.183.103:51762\" http.request.uri=\"/v2/docker-jenkins-integration-sample/manifests/latest\" http.request.useragent=\"docker/19.03.5 go/go1.12.12 git-commit/2ee0c57608 os/windows arch/amd64 UpstreamClient(Docker-Client/19.03.5 \\(windows\\))\" http.response.contenttype=\"application/json; charset=utf-8\" http.response.duration=1.0671ms http.response.status=500 http.response.written=523 vars.name=docker-jenkins-integration-sample vars.reference=latest \n","stream":"stderr","time":"2022-02-08T13:24:35.8661829Z"}
{"log":"time=\"2022-02-08T13:24:35.3481934Z\" level=error msg=\"response completed with error\" err.code=unknown err.message=\"unknown error\" go.version=go1.16.13 http.request.contenttype=\"application/vnd.docker.distribution.manifest.v2+json\" http.request.host=hub.docker http.request.id=798ba3fb-0471-4e76-b8c6-fa04b7cd5d6a http.request.method=PUT http.request.remoteaddr=\"xxx.xxx.xxx.xxx\ http.request.uri=\"/v2/docker-jenkins-integration-sample/manifests/latest\" http.request.useragent=\"docker/19.03.5 go/go1.12.12 git-commit/2ee0c57608 os/windows arch/amd64 UpstreamClient(Docker-Client/19.03.5 \\(windows\\))\" http.response.contenttype=\"application/json; charset=utf-8\" http.response.duration=1.2378ms http.response.status=500 http.response.written=523 vars.name=docker-jenkins-integration-sample vars.reference=latest \n","stream":"stderr","time":"2022-02-08T13:24:35.8661829Z"}
{"log":"time=\"2022-02-08T13:24:35.3483014Z\" level=error msg=\"response completed with error\" err.code=unknown err.message=\"invalid URL on layer\" go.version=go1.16.13 http.request.contenttype=\"application/vnd.docker.distribution.manifest.v2+json\" http.request.host=hub.docker http.request.id=798ba3fb-0471-4e76-b8c6-fa04b7cd5d6a http.request.method=PUT http.request.remoteaddr=\"xxx.xxx.xxx.xxx\" http.request.uri=\"/v2/docker-jenkins-integration-sample/manifests/latest\" http.request.useragent=\"docker/19.03.5 go/go1.12.12 git-commit/2ee0c57608 os/windows arch/amd64 UpstreamClient(Docker-Client/19.03.5 \\(windows\\))\" http.response.contenttype=\"application/json; charset=utf-8\" http.response.duration=1.2945ms http.response.status=500 http.response.written=523 vars.name=docker-jenkins-integration-sample vars.reference=latest \n","stream":"stderr","time":"2022-02-08T13:24:35.8661829Z"}
{"log":"time=\"2022-02-08T13:24:35.3483546Z\" level=error msg=\"response completed with error\" err.code=\"manifest blob unknown\" err.detail=sha256:567fd00846e9a9f44eea5925b497356dda00fe89b8335d2a3b2a8b9d84b0bb69 err.message=\"blob unknown to registry\" go.version=go1.16.13 http.request.contenttype=\"application/vnd.docker.distribution.manifest.v2+json\" http.request.host=hub.docker http.request.id=798ba3fb-0471-4e76-b8c6-fa04b7cd5d6a http.request.method=PUT http.request.remoteaddr=\"192.168.183.103:51762\" http.request.uri=\"/v2/docker-jenkins-integration-sample/manifests/latest\" http.request.useragent=\"docker/19.03.5 go/go1.12.12 git-commit/2ee0c57608 os/windows arch/amd64 UpstreamClient(Docker-Client/19.03.5 \\(windows\\))\" http.response.contenttype=\"application/json; charset=utf-8\" http.response.duration=1.3989ms http.response.status=500 http.response.written=523 vars.name=docker-jenkins-integration-sample vars.reference=latest \n","stream":"stderr","time":"2022-02-08T13:24:35.8661829Z"}
{"log":"xxx.xxx.xxx.xxx - - [08/Feb/2022:13:24:35 +0000] \"PUT /v2/docker-jenkins-integration-sample/manifests/latest HTTP/1.1\" 500 523 \"\" \"docker/19.03.5 go/go1.12.12 git-commit/2ee0c57608 os/windows arch/amd64 UpstreamClient(Docker-Client/19.03.5 \\\\(windows\\\\))\"\n","stream":"stdout","time":"2022-02-08T13:24:35.8661829Z"}
Setting like "max-concurrent-downloads": 1 is not worked. How to solve the issue?
EDIT
I created the private registry by following command on powershell in Windows Server 2019
docker run -d -p 443:443 --name hub.local -v C:/localhub/certs:/certs -v C:/localhub/registry:/var/lib/registry -e REGISTRY_HTTP_ADDR=0.0.0.0:443 -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/localhub.crt -e REGISTRY_HTTP_TLS_KEY=/certs/localhub.key registry
Jenkins setting are as below:
Solution
Given the invalid URL error and on the "windows" in the 500, I'm guessing it's a foreign layer for windows images that's getting rejected by validation. I tried figuring out the right syntax to allow that in the past and eventually gave up and turned off the validation with REGISTRY_VALIDATION_DISABLED=true. E.g. here's a command I use to start a registry:
docker run -d --restart=unless-stopped --name registry \
-e "REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY=/var/lib/registry" \
-e "REGISTRY_STORAGE_DELETE_ENABLED=true" \
-e "REGISTRY_VALIDATION_DISABLED=true" \
-v "registry-data:/var/lib/registry" \
-p "127.0.0.1:5000:5000" \
registry:2
Answered By - BMitch
Answer Checked By - Marie Seifert (JavaFixing Admin)
