Your browser was unable to load all of the resources. They may have been blocked by your firewall, proxy or browser configuration.
Press Ctrl+F5 or Ctrl+Shift+R to have your browser try again.

QuickBuild

Docker build agent #3151

#1
scddev ·
I am trying to create a docker image for my build agent. The problem is the port reported by the agent to the server.

When starting the container using the specified port from node.properties (docker run -p 8811:8811) everything works fine.

The problem arises, when trying to start multiple containers using dynamic ports (e.g. docker run -P). The port which can be used to connect to the agent is not the same as specified in node.properties (49137->8811). I could modify node.properties after creating the container and set the outside port but that does not seem nice. Also I would loose the feature of exposing a port from the image cause I don't know beforehand which port would be available (docker run -p 49137:49137).

Are there any plans to support docker using a plugin like the Amazon EC2?
  • replies 6
  • views 3949
  • stars 1
#2
robinshen ADMIN ·
Right now, QB agent only picks up port (and various other properties such as SSL key store, tokens, etc.) from node.properties, and the EC2 agent also works this way (port needs to be defined before-hand). I am not familiar with Docker, but just wonder why do you want to start multiple QB agents on a single machine (hence I guess there will exist a port conflict)?
#3
scddev ·
There are two reasons to go that way.

At first you can define a clean image for your builds. We need several components preinstalled to build some parts so we can do that once in the image and start as many qb nodes as we like with a simple "docker run" command. Also if the prerequisites change, we only need to adapt the image and are good.

Second are integration tests which start e.g. jetty, httpd, ... and the container gives a nice way to separate everything and use the hardware by running multiple integration tests in parallel.

Here a sample Dockerfile so you can test. Just place this and the buildagent.tar.gz into a directory.


FROM ubuntu:14.04

RUN apt-get update && apt-get install -y wget

WORKDIR /opt
RUN wget -O /tmp/jdk-8u40-linux-x64.tar.gz --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u40-b26/jdk-8u40-linux-x64.tar.gz \
  && tar zxvf /tmp/jdk-8u40-linux-x64.tar.gz \
  && rm /tmp/jdk-8u40-linux-x64.tar.gz

ENV PATH /opt/jdk1.8.0_40/bin:$PATH

RUN locale-gen en_US.UTF-8
RUN echo 'LANG=en_US.UTF-8' > /etc/default/locale

COPY buildagent.tar.gz /tmp/buildagent.tar.gz
RUN tar zxvf /tmp/buildagent.tar.gz && rm /tmp/buildagent.tar.gz

WORKDIR /opt/buildagent

CMD ["./bin/agent.sh","console"]

EXPOSE 8811


Now you can build the image with


docker build -t qbagent .


And start the container using that image with


docker run -d -p 8811:8811 qbagent


To use the dynamic port mapping use


docker run -d -P qbagent

docker ps -a
CONTAINER ID        IMAGE               COMMAND                CREATED             STATUS              PORTS                     NAMES
e50c21f4c6f7        qbagent:latest      "./bin/agent.sh cons   3 seconds ago       Up 2 seconds        0.0.0.0:49153->8811/tcp   clever_goldstine
#4
robinshen ADMIN ·
Thanks for the detailed script. It seems quite simple. So the problem is that you want QB to pick up ports from docker run command, instead of reading it from node.properties? If it is true, I guess this is a quite common problem, for instance one might want to control Tomcat port via docker run option, and how does Tomcat handles that? Does it need third party softwares to add the logic of reading options from docker run arguments to achieve this flexibility?
#5
scddev ·
No, the process inside the container does not know that it is behind a NAT and also does not need to know. Its like you are using VMware and create a NAT network between host and virtual machines.

The ports are redirected with iptables.

Docker creates an internal network. When using -p or -P you can define port forwards from the hosts IP:PORT to the container local IP:PORT. The local IPs of the containers are only reachable from the docker host.

E.g.:

Docker Host
  eth0 192.168.0.1
  docker0 10.0.0.1

  container qbagent01 inside host01
    eth0 10.0.0.2

  container qbagent02 inside host01
    eth0 10.0.0.3

QB Server
  eth0 192.168.0.50



docker run --name qbagent01 -d -P qbagent
docker run --name qbagent02 -d -P qbagent

docker ps -a
CONTAINER ID        IMAGE               COMMAND                CREATED             STATUS              PORTS                     NAMES
aebd6d2387dd        qbagent:latest      "./bin/agent.sh cons   2 minutes ago       Up 2 minutes        0.0.0.0:49156->8811/tcp   qbagent02
f1886a7c53c1        qbagent:latest      "./bin/agent.sh cons   2 minutes ago       Up 2 minutes        0.0.0.0:49155->8811/tcp   qbagent01


Now you will notice that the server has only one unauthorized node which switches with every refresh from one agent to the other. Both of them try to register with 192.168.0.1:8811.

Trying to register them using REST:


curl -u admin:password "http://192.168.0.50:8810/rest/tokens/authorize?ip=192.168.0.1&port=49155"
curl -u admin:password "http://192.168.0.50:8810/rest/tokens/authorize?ip=192.168.0.1&port=49156"


This yields now two inactive nodes plus the alternating unauthorized node from 192.168.0.1:8811.
#6
robinshen ADMIN ·
Thanks for the explanation. I understand the problem now. When QB agent connects to server, it reports to server the port (read from node.properties) it is currently using, so that server can connect to agent later when a build needs to run on the agent. So if the host port is forwarded to a different port inside Docker container, it will cause issues as QB agent is not aware of the forwarded port (49155/49156). To solve this, the node.properties has to be changed via some form of scripting to use the forwarded port instead of 8811 before running agent, is this possible via some scripting working together with docker?
#7
scddev ·
Yes, thats always possible, I can create the container with a manually chosen port, modify node.properties and start the agent to match that forwarded port. I just wanted to make sure, this is the only viable solution and I did not miss an other option. Maybe thats the way I will use it for now. Thanks.
Log in to reply