How to use docker pull with VPN or http proxy or socks5 proxy

[unfinished]

Prerequisites: Assuming you have a VPS at USA node. You have completed installed Cisco Ipsec[1] and squid on VPS. In this passage,I will say http proxy and socks5 proxy, [3] is the success method, and so does the [4].In some cases, [4] requires user and password in the form of “usr:pwd”, but it is not always required in many cases, it may only use IP and port as http proxy.You can use socks5 proxy[2] as well.

1.Basic framework

Remote environment:VPS OS is centos7.

Local environment:

In my cases, my laptop is Windows 11/10 with certificate to dial up ipsec. I use hyper-v to create a virtual machine(VM for short in the following) ubuntu 18.04. The VM ubuntu has docker installed.

1.1 install cisco ipsec

Reference [1] to install cisco ipsec, and your terminal device can be macos, windows, ios, or android.

1.2 squid related

sudo yum install squid

service squid start

1.2.1 scoks5 proxy in client

In your client is Windows, you could use the following command to connect to VPS. The VPS_HOST is the domain of VPS or ip.

powershell -command “ssh -NfD 0.0.0.0:3128 user@VPS_HOST”

This made a socks5 proxy on LAN.

  • ssh: The SSH client command.
  • -N: Do not execute a remote command. This is useful for just forwarding ports.
  • -f: Go to background after authentication.
  • -D 0.0.0.0:3128: Specifies a dynamic application-level port forwarding. 0.0.0.0 tells SSH to bind the SOCKS proxy to all available network interfaces, and 3128 is the local port it will listen on.
  • user@VPS_HOST: Your SSH username and the hostname/IP of your VPS.

1.2.2 http proxy in client

You could use the following command to setup http proxy in your Windows client.

powershell -command “ssh -NfL 0.0.0.0:8080:127.0.0.1:3128 user@VPS_HOST”

  • ssh: The SSH client command.
  • -N: Do not execute a remote command.
  • -f: Go to background after authentication.
  • -L 0.0.0.0:8080:127.0.0.1:3128: Specifies a local port forward.
    • 0.0.0.0: The bind address on your local laptop. This means the HTTP proxy will be accessible from all interfaces on your laptop (including your LAN IP 192.168.1.106). If you only want it accessible from your laptop, use 127.0.0.1 instead.
    • 8080: The local port on your laptop that you will connect to. You can choose any unused port here (e.g., 8080, 8118, etc.).
    • 127.0.0.1: The destination address on the VPS. This tells SSH to connect to the Squid proxy running on the VPS’s localhost.
    • 3128: The port Squid is listening on on the VPS.
  • user@VPS_HOST: Your SSH username and the hostname/IP of your VPS.

1.3 clash for http proxy

Just allow local lan on your 7890 port, in your “clash for windows” app.

2. Trying pull image directly

Running the command:

docker pull hello-world

You’ll probably get the error message like this:

Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

At that situation, you may not access the url directly, try this:

curl https://registry-1.docker.io/v2/

You should get:

curl: (28) Failed to connect to registry-1.docker.io port 443 after 21062 ms: Couldn’t connect to server

The reason is that your internet can not connect to registry-1.docker.io.

3.Trying pull image through global proxy(cisco ipsec)

3.1 dial up Cisco ipsec in Windows. You should aceess google.com at that time. Try curl https://registry-1.docker.io/v2/ in your Windows and VM ubuntu again. If it is succeed, then you could run

docker pull hello-world

It may the easiest way to use.

4.Trying pull image through http proxy

Using docker info to check the proxy information.There should be no proxy at this time.

docker info

4.1 through your clash port 7890

Enable “Allow LAN” in “your clash for windows”‘s “general” Tab. Now you can connect lan ip with port 7890.

sudo mkdir -p /etc/systemd/system/docker.service.d

sudo nano /etc/systemd/system/docker.service.d/http-proxy.conf

And enter the following content:

[Service]

Environment=http_proxy=http://YOUR_LAN_IP:7890/

Environment=no_proxy=localhost,127.0.0.1

Environment=https_proxy=http://YOUR_LAN_IP:7890/

After that, execute the 2 command:

systemctl daemon-reload

systemctl restart docker

4.2 through your squid

In the cmd.exe of your windows, using chapter1.2.2 command to set up http proxy on port 8080

sudo mkdir -p /etc/systemd/system/docker.service.d

sudo nano /etc/systemd/system/docker.service.d/http-proxy.conf

And enter the following content:

[Service]

Environment=http_proxy=http://YOUR_LAN_IP:8080/

Environment=no_proxy=localhost,127.0.0.1

Environment=https_proxy=http://YOUR_LAN_IP:8080/

After that, execute the 2 command:

systemctl daemon-reload

systemctl restart docker

 

4.3 using bash command

If you are tired with command , you could copy the following bash command to run it, like [4] does. Set http proxy’s ip , port, domain name of your own. If you have not user and password of the proxy, the ActivePorxyVar is just like this: “ActiveProxyVar=IP:PORT”.

5.Trying pull image through socks5 proxy

In the cmd.exe of your windows, using chapter1.2.1 command to set up socks5 proxy on port 3128

sudo mkdir -p /etc/systemd/system/docker.service.d

sudo nano /etc/systemd/system/docker.service.d/socks5-proxy.conf

And enter the following content:

[Service]

Environment=http_proxy=socks://YOUR_LAN_IP:3128/

Environment=no_proxy=localhost,127.0.0.1

Environment=https_proxy=socks://YOUR_LAN_IP:3128/

After that, execute the 2 command:

systemctl daemon-reload

systemctl restart docker

 

6.compare sock5 proxy versus http proxy

Sock5 proxy is faster than http proxy.

7.the same situation

If you use *.yml and execute the command, you may also engage the same problem. Reference step 2 to 6 to fix the problem. The command you may use once the *.yml is completed is:

docker-compose up -d

Reference:

1. https://github.com/hwdsl2/setup-ipsec-vpn

2. https://www.cnblogs.com/mq0036/p/17184494.html

3.https://stackoverflow.com/questions/51571686/ubuntu-18-04-error-response-from-daemon-get-https-registry-1-docker-io-v2/51648635#51648635

4.https://stackoverflow.com/questions/48056365/error-get-https-registry-1-docker-io-v2-net-http-request-canceled-while-b/77130871#77130871

Migrate virtual machine from ovf file to hyper-v

1.Ovf file is exported from vmware workstation 15

So we get 3 files:
(1)Ubuntu_64_bit-disk1.vmdk;
(2)Ubuntu 64 bit.mf;
(3)Ubuntu 64 bit.ovf;

2.Using qemu-img to convert ovf files to vhdx

qemu-img.exe convert D:\ubovf\Ubuntu_64_bit-disk1.vmdk -O vhdx -o subformat=dynamic D:\test_dir\dest.vhdx

3.Create a new virtual machine in hyper-v

Create a new virtual machine, attaching existing vhdx hard disk files, and select generation 1 only. The source vmdk virtual machine should be legacy bios machine, so generation 2 can not be started.
Then start and connect to the VM.

Reference:

1. https://cloudbase.it/qemu-img-windows/
2. https://learn.microsoft.com/en-us/windows-server/virtualization/hyper-v/plan/should-i-create-a-generation-1-or-2-virtual-machine-in-hyper-v

Realvnc black screen and use in vmware

I am a beginner with using realvnc. I have a headless PC which runs remote application and steam. The vncviewer is always black screen when I use realvnc instead of mstsc. Now, if I plugged monitor to the display card, the vncviewer could display content while it was connected.
1.I noticed that, if I un-checked the directX relatived opinion and un-plugged monitor, the vncviewer could show some content such as Microsoft 365 copilot and notepad, but some of the window are displayed as black screen such as CPU fans which may be using directX technology.
2. How to use vncviewer in vmware workstation?
I use virtual machine with bridgged host network or NAT, enable the vnc in the opinion and give a port which is not used. Connect to the HOST name or ip address rather than the virtual machine’s network name or ip address, I can connect the virtual machine with vncviewer.
Reference:
1.https://help.realvnc.com/hc/en-us/articles/360004012211-Display-issues-when-connecting-to-RealVNC-Server-running-on-Windows

Could not get vmci driver version: The handle is invalid.

I update the vmware workstation from 12 to 15, but there is a problem I got when I try to start the virtual machine from snapshot. It says:
“Could not get vmci driver version: The handle is invalid.

You have an incorrect version of driver ‘vmci.sys’. Try reinstalling VMware Workstation.

Module ‘DevicePowerOn’ power on failed.

Failed to start the virtual machine.”

Solution 1: find xxx.vmx to edit this line:
vmci0.present = “TRUE”
Modify it to
vmci0.present = “FALSE”
or delete this line.
It is temporary workaround that should only be used to get the VM to boot in a pinch.

Solution 2: update the vmci driver.
Open device manager, the driver of vmci may be incorrect.
If you are connect to internet, just update it. You can get vmci.sys driver installed automatically.
The driver is stored in
C:\Program Files\Common Files\VMware\Drivers\vmci\device\Win8\vmci.sys
You should installed it from there manually if you do not have internet connection.

Finally, check if it is conflict with hyper-v componet in Windows.

Summary: Updating the vmci.sys driver via Device Manager is the fundamentally correct and recommended approach.

Reference:

https://www.cnblogs.com/Jasper-changing/p/12035933.html

https://www.youtube.com/watch?v=aZk3B31Utzo