Jenkins in a container

The jenkins.simulakrum.org server is now running from within a Docker container. Migration was flawless and done from scratch in less than 30 minutes. The reason was that Ubuntu would fail to restart a native jenkins service if another Docker container would use a port, albeit on a different IP. After being fed up with constant joggling between solutions for that, I decided it would be faster to simply “dockerise” Jenkins, too, and have it confined in a container for good.

Here’s a simple Dockerfile:

FROM centos:7

MAINTAINER “Vanja A” <vanja@simulakrum.org>

ENV TERM xterm

RUN yum -y install epel-release

RUN yum -y install deltarpm

RUN yum -y update

#RUN yum -y install systemd-container

RUN yum -y erase ntpdate

RUN yum -y install –enablerepo=base –disablerepo=updates ntpdate

RUN yum -y install \
wget tar lynx cronie nagios-plugins-all nrpe check-mk munin-node \
perl-XML-Simple perl-JSON git subversion \
openssl ssmtp rsyslog openvpn net-tools supervisor openldap-clients \
java-1.8.0-openjdk java-1.8.0-openjdk-devel java-1.8.0-openjdk-headless java-1.8.0-openjdk-javadoc \
xorg-x11-server-Xvfb openssh-server \
glibc.i686 glibc-devel.i686 glibc-devel.x86_64 glibc-headers.x86_64 glibc-utils.x86_64 libstdc++.so.6 nfs-utils \
sudo gcc-c++ p7zip p7zip-plugins unzip clamav \
clamav-update clamav-server clamav-data clamav-devel clamav-lib httpd mod_ssl mod_ldap

RUN cd /etc && mv localtime localtime.old && cp /usr/share/zoneinfo/Europe/Berlin /etc/localtime
RUN chmod u+s /bin/ping

ADD known_hosts /root/.ssh/known_hosts
ADD ssmtp.conf /etc/ssmtp/ssmtp.conf

RUN munin-node-configure –shell | sh -x &>/dev/null

RUN echo “LDAPVerifyServerCert Off” >> /etc/httpd/conf/httpd.conf && \
echo “TLS_REQCERT never” >> /etc/openldap/ldap.conf && \
echo “TLS_REQUEST allow” >> /etc/openldap/ldap.conf

RUN wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo
RUN rpm –import https://jenkins-ci.org/redhat/jenkins-ci.org.key
RUN yum -y install jenkins

RUN cd /usr/share/munin/plugins/ && wget https://raw.githubusercontent.com/munin-monitoring/contrib/master/plugins/jenkins/jenkins_ && \
chmod 775 /usr/share/munin/plugins/jenkins_ && cd /etc/munin/plugins && \
ln -s /usr/share/munin/plugins/jenkins_ /etc/munin/plugins/jenkins_results && \
ln -s /usr/share/munin/plugins/jenkins_ /etc/munin/plugins/jenkins_queue && \
ln -s /usr/share/munin/plugins/jenkins_ /etc/munin/plugins/jenkins_running && \
cd /etc/munin/plugin-conf.d/ && \
echo ‘[jenkins_*]’ > /etc/munin/plugin-conf.d/jenkins && \
echo ‘ env.url MyURLwenthere’ >> /etc/munin/plugin-conf.d/jenkins && \
echo ‘ env.port 8080’ >> /etc/munin/plugin-conf.d/jenkins && \
echo ‘ env.context /’ >> /etc/munin/plugin-conf.d/jenkins && \
echo ‘ env.user jenkins’ >> /etc/munin/plugin-conf.d/jenkins && \
echo ‘ env.apiToken MyAPItokenwenthere’ >> /etc/munin/plugin-conf.d/jenkins && \
chown -R munin:munin /etc/munin

RUN echo “Updating from here…”

RUN chown -R jenkins:jenkins /var/lib/jenkins
RUN cd /tmp && wget http://mirrors.jenkins-ci.org/war/latest/jenkins.war && \
rm -Rf /usr/lib/jenkins/jenkins.war && cp jenkins.war /usr/lib/jenkins/jenkins.war

ADD munin-node.conf /etc/munin/munin-node.conf
RUN chown -R munin:munin /var/lib/munin

COPY supervisord.conf /etc/supervisord.conf

EXPOSE 80 443 8080 8443 4949 22166 33166

ENTRYPOINT [“/usr/bin/supervisord”]

Fetch Sun/Oracle’s Java if you need it, too:

#!/bin/bash
/usr/bin/cd /tmp
/usr/bin/wget –no-check-certificate –no-cookies –header “Cookie: oraclelicense=accept-securebackup-cookie” http://download.oracle.com/otn-pub/java/jdk/8u45-b14/jdk-8u45-linux-x64.rpm
/usr/bin/wget –no-check-certificate –no-cookies –header “Cookie: oraclelicense=accept-securebackup-cookie” http://download.oracle.com/otn-pub/java/jdk/7u79-b15/jdk-7u79-linux-x64.rpm
/usr/bin/wget –no-check-certificate –no-cookies –header “Cookie: oraclelicense=accept-securebackup-cookie” http://download.oracle.com/otn-pub/java/jdk/6u45-b06/jdk-6u45-linux-x64-rpm.bin
/usr/bin/chmod 750 jdk-6u45-linux-x64-rpm.bin
/tmp/jdk-6u45-linux-x64-rpm.bin -x
/usr/bin/yum -y localinstall –nogpgcheck /tmp/jdk-6u45-linux-amd64.rpm
/usr/bin/rm -Rf /tmp/jdk-6u45-linux-x64-rpm.bin
/usr/bin/rm -Rf /tmp/jdk-6u45-linux-amd64.rpm
/usr/bin/rm -Rf /tmp/sun-javadb*
/usr/bin/cp -Rp /usr/java/* /opt/java/
/usr/bin/yum -y localinstall –nogpgcheck /tmp/jdk-7u79-linux-x64.rpm
/usr/bin/rm -Rf /tmp/jdk-7u79-linux-x64.rpm
/usr/bin/yum -y localinstall –nogpgcheck /tmp/jdk-8u45-linux-x64.rpm
/usr/bin/rm -Rf /tmp/jdk-8u45-linux-x64.rpm

Prepare a certs script:

#!/bin/bash
/usr/bin/keytool -import -trustcacerts -keystore /usr/lib/jvm/java/jre/lib/security/cacerts -storepass mystorepass -alias simulakrum.org -file /private/server.crt -noprompt 2&>/dev/null
/usr/bin/keytool -import -trustcacerts -keystore /usr/lib/jvm/java/jre/lib/security/cacerts -storepass mystorepass -alias openldap.simulakrum.org -file /private/openldap.simulakrum.org.crt -noprompt 2&>/dev/null
/usr/bin/keytool -import -trustcacerts -keystore /usr/lib/jvm/java/jre/lib/security/cacerts -storepass mystorepass -alias mail.simulakrum.org -file /private/mail.simulakrum.org.crt -noprompt 2&>/dev/null
/usr/bin/keytool -import -trustcacerts -keystore /usr/lib/jvm/java/jre/lib/security/cacerts -storepass mystorepass -alias imap.simulakrum.org -file /private/dovecotcert.pem -noprompt 2&>/dev/null
/opt/java/jdk1.6.0_45/bin/keytool -import -trustcacerts -keystore /opt/java/default/jre/lib/security/cacerts -storepass mystorepass -alias simulakrum.org -file /private/server.crt -noprompt 2&>/dev/null
/opt/java/jdk1.6.0_45/bin/keytool -import -trustcacerts -keystore /opt/java/default/jre/lib/security/cacerts -storepass mystorepass -alias openldap.simulakrum.org -file /private/openldap.simulakrum.org.crt -noprompt 2&>/dev/null
/opt/java/jdk1.6.0_45/bin/keytool -import -trustcacerts -keystore /opt/java/default/jre/lib/security/cacerts -storepass mystorepass -alias mail.simulakrum.org -file /private/mail.simulakrum.org.crt -noprompt 2&>/dev/null
/opt/java/jdk1.6.0_45/bin/keytool -import -trustcacerts -keystore /opt/java/default/jre/lib/security/cacerts -storepass mystorepass -alias imap.simulakrum.org -file /private/dovecotcert.pem -noprompt 2&>/dev/null

And fix the rest on container startup, adjust accordingly:

#!/bin/bash

sh /private/import_keys_into_java_keystore.sh
touch /var/log/cron && /usr/sbin/logrotate -vf /etc/logrotate.conf &>/dev/null
groupadd -g 1000 themounters
usermod -a -G themounters jenkins
/usr/bin/cp /private/sudoers /etc/sudoers
usermod -a -G wheel jenkins
echo “jenkins ALL=(root) NOPASSWD: ALL” > /etc/sudoers.d/jenkins.conf
sleep 2
#export _JAVA_OPTIONS=’-Xms64M -Xmx128m’
sleep 5
##su – jenkins -s /bin/bash -c ‘/etc/alternatives/java -Dhudson.model.UsageStatistics.disabled=true -Dcom.sun.akuma.Daemon=daemonized -Djava.awt.headless=true -DJENKINS_HOME=/var/lib/jenkins -jar /usr/lib/jenkins/jenkins.war –logfile=/var/log/jenkins/jenkins.log –webroot=/var/cache/jenkins/war –daemon –httpPort=8080 –ajp13Port=8009 –debug=5 –handlerCountMax=100 –handlerCountMaxIdle=20’ &
### No AJP after Jenkins 2.0 – Winstone change
su – jenkins -s /bin/bash -c ‘/etc/alternatives/java -Dhudson.model.UsageStatistics.disabled=true -Dcom.sun.akuma.Daemon=daemonized -Djava.awt.headless=true -DJENKINS_HOME=/var/lib/jenkins -jar /usr/lib/jenkins/jenkins.war –logfile=/var/log/jenkins/jenkins.log –webroot=/var/cache/jenkins/war –daemon –httpPort=8080 –debug=5 –handlerCountMax=100 –handlerCountMaxIdle=20’ &
sleep 10
cp /etc/freshclam.conf /etc/freshclam.conf.bak
sed -i ‘/^Example/d’ /etc/freshclam.conf
/usr/bin/freshclam -d &
cp /usr/share/clamav/template/clamd.conf /etc/clamd.d/clamd.conf
sed -i ‘/^Example/d’ /etc/clamd.d/clamd.conf
sed -i ‘s/<USER>/root/’ /etc/clamd.d/clamd.conf
sed -i ‘s/#TCPSocket 3310/TCPSocket 3310/’ /etc/clamd.d/clamd.conf
sed -i ‘s/#TCPAddr 127.0.0.1/TCPAddr 127.0.0.1/’ /etc/clamd.d/clamd.conf
/usr/sbin/clamd -c /etc/clamd.d/clamd.conf &
sleep 2
chown -R jenkins:jenkins /var/lib/jenkins &

curl –user ‘jenkinsuser:jenkinsuserpass’ –data-urlencode “script=$(</private/fix_frames.groovy)” http://MyIPonLAN:8080/scriptText

Deploy it with:

/usr/bin/docker run -d –privileged –name jenkins -h jenkins.simulakrum.org -e “container=docker” -v /sys/fs/cgroup:/sys/fs/cgroup -v /dockerplace/jenkins/private:/private -v /dockerplace/jenkins/etcmunin:/etc/munin -v /dockerplace/jenkins/etchttpd:/etc/httpd -v /dockerplace/jenkins/varloghttpd:/var/log/httpd -v /dockerplace/jenkins/varwww:/var/www -v /dockerplace/jenkins/etcssh:/etc/ssh -v /dockerplace/jenkins/root.ssh:/root/.ssh -v /dockerplace/jenkins/optjava:/opt/java -v /dockerplace/jenkins/usrjava:/usr/java -v /dockerplace/jenkins/vlj:/var/lib/jenkins -p 192.168.192.168:4949:4949 -p 192.168.192.168:22166:22166 -p 192.168.192.168:33166:33166 -p 192.168.192.168:8080:8080 -p 192.168.192.168:3310:3310 -p 192.168.192.168:22888:22888 -p 192.168.192.168:443:443 -p 192.168.192.168:80:80 $JENKINSIMAGEID

There, no more port conflicts!

Leave a Reply

Your email address will not be published. Required fields are marked *