In this tutorial, we will build a CI CD pipeline for a Python Flask project using Git, Docker, Jenkins and Ansible. Git shall be used for version control. Docker shall be used for containerization. Jenkins shall be used for continuous integration including build automation and Ansible shall be used for continuous deployment.
Git and Docker shall be the prerequisites for this project implementation. An account at hub.docker.com shall be required to host the docker images needed for this project. And finally, a GitHub account for hosting the source code of the project.
Chapters:
- Python Flask web app
- Github
- Jenkins
- Ansible
1. Python Flask web app
1.1 Source Code
app.py
from flask import Flask
app=Flask(__name__)
@app.route('/')
def webout():
return '<h1>DevOps is fun.</h1>'
app.run(host='0.0.0.0',port=7000)
pyreq.txt
Flask==2.0.3
Werkzeug==2.0.3
1.2 Dockerfile and docker-compose.yml files
Dockerfile
FROM python:3.9-slim
COPY . /app
WORKDIR /app
RUN pip install --no-cache-dir -r pyreq.txt
CMD ["python","app.py"]
docker-compose.yml
services:
flask:
image: apurwasingh/flask
container_name: flask
ports:
- "8000:7000"
Build the Docker image
docker build -t apurwasingh/flask .
Run the docker container
docker-compose up -d
Note: Port 8000 on the host OS is mapped to port 7000 on the container.Make sure that port 8000 on the local machine is available.
Ports in use can be found with following command:
On Unix like OS:
netstat -tlpn
On windows:
netstat -aon | findstr LISTENING
On macOS:
netstat -avn | grep LISTEN
Go to localhost:8000
2. Github
2.1 Key based authentication
Generate ssh key pair in local machine.
ssh-keygen -t rsa -b 4096 -C "apurwa@gmail.com"
Start ssh agent from cmd with following command:
net start ssh-agent
Note: If the agent cant be started, go to Windows Services and check for a service called OpenSSH Authentication Agent. Make sure that it is started. Once that is done, try the command again.
Add the ssh keys:
ssh-add
Add public key to Github
Go to github.com/settings/keys, create a new SSH key and paste the content of the public key id_rsa.pub.
Test connection from the host OS.
ssh git@github.com
Note: Don't use your username.
You should see a message like this:
Hi apurwa-np! You've successfully authenticated, but GitHub does not provide shell access.
Connection to github.com closed.
Now, we can push to GitHub repo using key based authentication from our host OS.
2.2 GitHub repository setup
Create a new repo at GitHub (python in this case)
Copy the SSH version of the repo’s URL
git@github.com:apurwa-np/python.git
Go to the folder containing the python project in the local machine.
Initialize git.
git init
Configure git.
git config user.email apurwa@gmail.com
git config user.name apurwa
Add remote repo
git remote add python git@github.com:apurwa-np/python.git
Note: Here, our remote repo is called python.
Add files to staging area
git add .
Commit to version database
git commit -m "first commit"
Push the source code to github
git push python master
Note: Find out the branch name with:
git branch
3. Jenkins
3.1 Jenkins container setup
Run docker containers for Jenkins and Ansible:
docker-compose -f ansible-jenkins.yml up -d
Go to localhost:8080 and login
username: admin
password: admin
3.2 Credentials configuration
Docker Hub credentials
Manage Jenkins > Credentials
Click on global > Add credentials
- Kind: Username with password
- ID: docker-hub-credentials
Enter Username and password for the Docker Hub account.
Note: If a Gmail account is used for logging into Docker Hub, password is the Gmail account's password. Docker Hub also uses a personal access token (PAT) mechanism which can be set here as password. For personal access token, go to hub.docker.com > Account Settings > Personal access tokens.
3.3 Jenkinsfile
A script file named Jenkinsfile is in the project directory as follows:
pipeline{
agent { label 'masternode' }
environment {
DOCKER_IMAGE_NAME = 'apurwasingh/flask'
DOCKER_IMAGE_TAG = 'latest'
}
stages{
stage('Checkout'){
steps {
checkout scm
}
}
stage('Build'){
steps{
script{
docker.build("${DOCKER_IMAGE_NAME}:${DOCKER_IMAGE_TAG}")
docker.withRegistry('https://index.docker.io/v1/', 'docker-hub-credentials') {docker.image("${DOCKER_IMAGE_NAME}:${DOCKER_IMAGE_TAG}").push()
}
}
}
}
stage('Deploy'){
steps{
script{
sh "docker exec -u root ansible ansible-playbook /root/deploy.yml"
}
}
}
}
}
Note: Change the value in yellow to match your Dockerhub’s image's name.
4. Ansible Playbook configuration
In the project folder, a file named deploy.yml is as follows:
---
- name: Deploy the latest Flask app
hosts: localhost
tasks:
- name: Stop the running container
docker_container:
name: flask
state: stopped
image: apurwasingh/flask
- name: Remove the stopped container
docker_container:
name: flask
state: absent
- name: Pull the latest image from docker hub
docker_image:
name: apurwasingh/flask
tag: latest
source: pull
- name: Run the container with the latest image
docker_container:
name: flask
image: apurwasingh/flask:latest
state: started
detach: yes
tty: yes
ports: [8000:7000]
restart_policy: always
Change the values in blue to match your dockerhub’s image name.
5 Jenkins pipeline
New Item > Pipeline
- Trigger: Poll SCM
Schedule: */5 * * * * (every 5 minutes) - Definition: Pipeline script from SCM
SCM: Git
Repository URL: https://github.com/apurwa-np/python.git - Branch Specifier: */master
- Script Path: Jenkinsfile
Note: Use the https version for Repository URL. SSH version will require SSH keys as credentials.
Push the changes to Github.
Done !
Now, every time a new commit is pushed to this GitHub repo, the Jenkins pipeline will be executed. Based on the script, the pipeline shall do the following:
- Checkout the latest source code from GitHub.
- Build a new docker image based on the latest source code.
- Push the image to Docker hub repository.
- Remove the running container.
- Run a new container based on the new image.