CI CD pipeline

Submitted on Mon, 01/20/2025 - 07:47

In this tutorial, we will build a CI CD pipeline for a Python Flask project using Git, Docker, Jenkins and Ansible. Git shall be used for version control. Docker shall be used for containerization. Jenkins shall be used for continuous integration including build automation and Ansible shall be used for continuous deployment.

Git and Docker shall be the prerequisites for this project implementation. An account at hub.docker.com shall be required to host the docker images needed for this project. And finally, a GitHub account for hosting the source code of the project.

Chapters:

  1. Python Flask web app
  2. Github
  3. Jenkins
  4. Ansible

 


1. Python Flask web app

1.1 Source Code

app.py
from flask import Flask
app=Flask(__name__)
@app.route('/')
def webout():
return '<h1>DevOps is fun.</h1>'
app.run(host='0.0.0.0',port=7000)

 

pyreq.txt
Flask==2.0.3
Werkzeug==2.0.3

 

1.2 Dockerfile and docker-compose.yml files

Dockerfile

FROM python:3.9-slim
COPY . /app
WORKDIR /app
RUN pip install --no-cache-dir -r pyreq.txt
CMD ["python","app.py"]

 

docker-compose.yml

services:
  flask:
image: apurwasingh/flask
container_name: flask
ports:
  - "8000:7000"

 

Build the Docker image

docker build -t apurwasingh/flask .

 

Run the docker container

docker-compose up -d

Note: Port 8000 on the host OS is mapped to port 7000 on the container.Make sure that port 8000 on the local machine is available. 

 

Ports in use can be found with following command:

On Unix like OS:

netstat -tlpn

On windows:

netstat -aon | findstr LISTENING

On macOS:

netstat -avn | grep LISTEN

 

Go to localhost:8000 


2. Github

2.1 Key based authentication

Generate ssh key pair in local machine.

ssh-keygen -t rsa -b 4096 -C "apurwa@gmail.com"

 

Start ssh agent from cmd with following command:

net start ssh-agent

Note: If the agent cant be started, go to Windows Services and check for a service called OpenSSH Authentication Agent. Make sure that it is started. Once that is done, try the command again. 

 

Add the ssh keys:

ssh-add

 

Add public key to Github

Go to github.com/settings/keys, create a new SSH key and paste the content of the public key id_rsa.pub.

 

Test connection from the host OS.

ssh git@github.com

Note: Don't use your username.

You should see a message like this:

Hi apurwa-np! You've successfully authenticated, but GitHub does not provide shell access.

Connection to github.com closed.

 

Now, we can push to GitHub repo using key based authentication from our host OS.


2.2 GitHub repository setup

Create a new repo at GitHub (python in this case)

Copy the SSH version of the repo’s URL

git@github.com:apurwa-np/python.git

Go to the folder containing the python project in the local machine.

Initialize git.

git init

 

Configure git.

git config user.email apurwa@gmail.com

git config user.name apurwa

 

Add remote repo

git remote add python git@github.com:apurwa-np/python.git

Note: Here, our remote repo is called python.

 

Add files to staging area

git add .

 

Commit to version database

git commit -m "first commit"

 

Push the source code to github

git push python master

Note: Find out the branch name with:

git branch

 


3. Jenkins

3.1 Jenkins container setup

Run docker containers for Jenkins and Ansible:

docker-compose -f ansible-jenkins.yml up -d

 

Go to localhost:8080 and login

username: admin
password: admin

 

3.2 Credentials configuration

Docker Hub credentials

Manage Jenkins > Credentials

Click on global > Add credentials

  • Kind: Username with password
  • ID: docker-hub-credentials

Enter Username and password for the Docker Hub account.

Note: If a Gmail account is used for logging into Docker Hub, password is the Gmail account's password. Docker Hub also uses a personal access token (PAT) mechanism which can be set here as password. For personal access token, go to hub.docker.com > Account Settings > Personal access tokens.

 

3.3 Jenkinsfile

A script file named Jenkinsfile is in the project directory as follows:

pipeline{
agent { label 'masternode' }

environment {
    DOCKER_IMAGE_NAME = 'apurwasingh/flask'
    DOCKER_IMAGE_TAG = 'latest'
}

stages{
    stage('Checkout'){
        steps {
            checkout scm
        }
    }
    stage('Build'){
        steps{
            script{
                docker.build("${DOCKER_IMAGE_NAME}:${DOCKER_IMAGE_TAG}")
                docker.withRegistry('https://index.docker.io/v1/', 'docker-hub-credentials') {docker.image("${DOCKER_IMAGE_NAME}:${DOCKER_IMAGE_TAG}").push()
                }
            }
        }
    }
    stage('Deploy'){
        steps{
            script{
                sh "docker exec -u root ansible ansible-playbook /root/deploy.yml"
            }
        }
    }
}

 } 

Note: Change the value in yellow to match your Dockerhub’s image's name.


4. Ansible Playbook configuration

In the project folder, a file named deploy.yml is as follows:

---
- name: Deploy the latest Flask app
  hosts: localhost
  tasks:
  - name: Stop the running container
docker_container:
name: flask
state: stopped
image: apurwasingh/flask
  - name: Remove the stopped container
docker_container:
name: flask
state: absent
  - name: Pull the latest image from docker hub
docker_image:
name: apurwasingh/flask
tag: latest
source: pull
  - name: Run the container with the latest image
docker_container:
name: flask
image: apurwasingh/flask:latest
state: started
detach: yes
tty: yes
ports: [8000:7000]
restart_policy: always

Change the values in blue to match your dockerhub’s image name.


5 Jenkins pipeline

 

New Item > Pipeline

  • Trigger: Poll SCM
    Schedule: */5 * * * * (every 5 minutes)
  • Definition: Pipeline script from SCM
    SCM: Git
    Repository URL: https://github.com/apurwa-np/python.git
  • Branch Specifier: */master
  • Script Path: Jenkinsfile

Note: Use the https version for Repository URL. SSH version will require SSH keys as credentials.

pipeline

cicd

Push the changes to Github.

 


Done !

Now, every time a new commit is pushed to this GitHub repo, the Jenkins pipeline will be executed. Based on the script, the pipeline shall do the following:

  1. Checkout the latest source code from GitHub.
  2. Build a new docker image based on the latest source code.
  3. Push the image to Docker hub repository.
  4. Remove the running container.
  5. Run a new container based on the new image.