Deploying a YouTube Clone App with DevSecOps and Jenkins Shared Library
In these project, we are cloning the YouTube API With Jenkins Shared Library Using the All the CI/CD And Security Plugins. these project Specially dedicated for Hoe's the Jenkins shared library Works.
Step 1: Launch an Ubuntu 22.04 instance for Jenkins
Log into AWS Console: Sign in to your AWS account.
Launch an Instance:
Choose "EC2" from services. Click "Launch Instance."
Choose an AMI: Select an Ubuntu image.
Choose an Instance Type: Pick "t2.large."
Key Pair: Choose an existing key pair or create a new one.
Configure Security Group:
Create a new security group. Add rules for HTTP, and HTTPS, and open all ports for learning purposes. Add Storage: Allocate at least 20 GB of storage.
[ opening all ports is not recommended for production environments; it's just for educational purposes. ]
Launch Instance: Review and launch the instance.
Access Your Instance: Use SSH to connect to your instance with the private key.
For the installation of the Jenkins follow the official Document.
You will need to go to your AWS EC2 Security Group and open Inbound Port 8080 since Jenkins works on Port 8080.
For unlocking the Jenkins use this command
Jenkins Dashboard
Step2A: Install Docker on the Jenkins machine
Run the below commands to install the docker
After the docker installation, we will create a Sonarqube container (Remember to add 9000 ports in the security group).
Run this command on your EC2 instance to create a SonarQube container:
Now copy the IP address of the ec2 instance
Enter username and password, click on login and change password
SonarQube Dashboard
Step2B: Install Trivy on Jenkins machine
Provide executable permissions and run the shell script
This will install Trivy on our Jenkins machine.
Step3A: Launch an Ubuntu instance for Splunk
Step 1: Launch Instances
Log in to your AWS console or your chosen cloud provider.
Navigate to the EC2 service and launch Ubuntu 22.04 instances. Ensure you select T2.medium as the instance type and allocate 24GB of storage to each instance.
Step 2: Install Splunk
At this point, the first machine is set up with Jenkins. You can now move to the second machine and proceed with the installation of Splunk.
Connect to your second instance using Putty or Mobaxtreme.
To download and install Splunk on your Ubuntu instance use the
wget
command, use the following command.
To Depackage the Splunk use the below command
By running this command, you ensure that Splunk Enterprise is configured to start automatically when your Ubuntu system boots, allowing you to seamlessly integrate it into your workflow.
Please note that after running this command, you should follow the on-screen prompts to accept the terms and complete the setup to 100%.
After completing the initial setup and accepting the terms, you'll be prompted to create an admin user.
Administrator Username: Choose a username for the admin account. This should be a unique and secure username.
Administrator Password: Set a strong and secure password for the admin account. It's important to choose a password that combines upper and lower-case letters, numbers, and special characters for enhanced security.
Confirm your password to ensure it matches the one you initially entered.
By creating an administrator username and password, you'll have full access to your Splunk instance, allowing you to configure and manage it effectively.
After Installing set up the Splunk Account- For reference please check official document
Step3B: Install the Splunk app for Jenkins
In Splunk Dashboard
Click on Apps --> Find more apps
Search for Jenkins in the Search bar
You will get the Splunk app for Jenkins and click on install
On the homepage of Splunk, you will see Jenkins has been added
In the Splunk web interface, go to Settings > Data Inputs.
Click on HTTP Event Collector. -> Click on Global Settings
Set All tokens to enabled
Uncheck SSL enable
Use 8088 port and click on save
Now click on New token -> Provide a Name and click on the next
Click Review & Submit
Click Start searching
Now let's copy our token again
In the Splunk web interface, go to Settings > Data Inputs. -> Click on the HTTP event collector
Now copy your token and keep it safe
Add Splunk Plugin in Jenkins
Go to Jenkins dashboard
Click on Manage Jenkins --> Plugins --> Available plugins
Search for Splunk and install it.
Again Click on Manage Jenkins --> System
Search for Splunk
Check to enable
HTTP input host as SPLUNK PUBLIC IP
HTTP token that you generated in Splunk
Jenkins IP and apply.
Now go to Putty or Mobaxtreme and In Splunk machine run this command
Restart Both Splunk and Jenkins
Let's Restart our Splunk machine
Click on Settings --> Server controls
Now restart Jenkins and log in again.
Now go to Splunk and click on the Jenkins app and you will get this output monitoring
Step4A: Integrate Slack for Notifications
Create a Slack account
Open your preferred web browser (e.g., Google Chrome, Mozilla Firefox, Safari, or Microsoft Edge).
In the address bar, type or click on the following URL to access Slack's official website: https://slack.com
On the Slack homepage, you'll find a prominent "SIGN UP WITH GOOGLE" or "Try for Free" button. Click on it
Add the Channel -
Step4B: Install the Jenkins CI app on Slack
Go to Slack and click on your name
Select Settings and Administration
Click on Manage apps
It will open a new tab
Search for Jenkins CI and click on it
Now choose your Slack channel
Click on Add Jenkins CI integration.
You will be redirected to this page
Copy the team subdomain and integration token credential ID for later use.
Team subdomain:
mrbhushan
Integration token credential ID: Create a secret text credential using
LpaDDc2n96KFOLtQ4mDqkxEQ
as the value
Install Slack Notification Plugin in Jenkins
Go to Jenkins Dashboard
Click on manage Jenkins --> Plugins --> Available plugins
Search for Slack Notification and install
Click on Manage Jenkins --> Credentials --> Global
Select kind as Secret Text
At Secret Section Provide Your Slack integration token credential ID
Id and description are optional and create
Click on Apply and save
manage Jenkins -> add secrete credentials of slack & workspace is slack subdomain
Apply & save.
Step5A: Start Job
Go to Jenkins dashboard and click on New Item.
Provide a name for the Job & click on Pipeline and click on OK.
Step5B: Create a Jenkins shared library in GitHub
Create a new repository in GitHub named Jenkins_shared_library.
Connect to your VS Code
Create a directory named Jenkins-shared-library
Create a Vars directory inside it
Now, Let's Write a Groovy script for our Pipeline
Create a cleanWorkspace.groovy file and add the below code
Create checkoutGit.groovy file and add the below code
Now push them to GitHub using the below commands from vs code
Step5C: Add Jenkins shared library to Jenkins system
Go to Jenkins Dashboard
Click on Manage Jenkins --> system
Search for Global Pipeline Libraries and click on Add
Now Provide a name that we have to call in our pipeline
Step5D: Run Pipeline
Go to Jenkins Dashboard again & select the job and add the below pipeline
Slack Notifications-
Step6: Install Plugins like JDK, Sonarqube Scanner, NodeJs
Step6A: Install Plugin
Goto Manage Jenkins →Plugins → Available Plugins →
Install below plugins
1 → Eclipse Temurin Installer (Install without restart)
2 → SonarQube Scanner (Install without restart)
3 → NodeJs Plugin (Install Without restart)
Step6B: Configure Java and Nodejs in Global Tool Configuration
Goto Manage Jenkins → Tools → Install JDK(17) and NodeJs(16)→ Click on Apply and Save
Step6C: Configure Sonar Server in Manage Jenkins
Grab the Public IP Address of your EC2 Instance, Sonarqube works on Port 9000, so <Public IP>:9000.
Goto your Sonarqube Server. Click on Administration → Security → Users → Click on Tokens and Update Token → Give it a name → and click on Generate Token ->click on update Token
copy Token
Goto Jenkins Dashboard → Manage Jenkins → Credentials → Add Secret Text. It should look like this
Now, go to Dashboard → Manage Jenkins → System and Add like the below image.
Click on Apply and Save.
The Configure System option is used in Jenkins to configure different server
Global Tool Configuration is used to configure different tools that we install using Plugins
We will install a sonar scanner in the tools.
In the Sonarqube Dashboard add a quality gate also
Administration--> Configuration-->Webhooks
Click on Create
Add details
Step6D: Add New stages to the pipeline
Go to vs code and create a file sonarqubeAnalysis.groovy & add the below code and push to Jenkins shared library GitHub Repo.
Create another file for qualityGate.groovy
Create another file for npmInstall.groovy
Push them to the GitHub Jenkins shared library
Add these stages to the pipeline now
Build now.
To see the report, you can go to Sonarqube Server and go to Projects.
Step7: Install OWASP Dependency Check Plugins
GotoDashboard → Manage Jenkins → Plugins → OWASP Dependency-Check. Click on it and install it without restart.
First, we configured the Plugin and next, we had to configure the Tool
Goto Dashboard → Manage Jenkins → Tools →
Click on Apply and Save here.
Create a file for trivyFs.groovy
Push to GitHub
Add the below stages to the Jenkins pipeline
Stage with the Dependency Check steps cannot be directly used inside a shared library.
The main reason is that pipelines loaded from shared libraries have more restrictive script security by default. So the dependencyCheck
and dependencyCheckPublisher
steps would fail with rejected signature errors.
Build now
You will see that in status, a graph will also be generated and Vulnerabilities.
Step8A: Docker Image Build and Push
We need to install the Docker tool in our system, Goto Dashboard → Manage Plugins → Available plugins → Search for Docker and install these plugins
Docker
Docker Commons
Docker Pipeline
Docker API
docker-build-step
and click on install without restart
Now, goto Dashboard → Manage Jenkins → Tools →
Add DockerHub Username and Password under Global Credentials
Step8B: Create an API key from Rapid API
Open a new tab in the browser and search for rapidapi.com
It will automatically provide your mail and select a mail to create an account
Account is created
Now in the search bar search for YouTube and select YouTube v3
Copy API and use it in the groovy file
docker build --build-arg REACT_APP_RAPID_API_KEY=<API-KEY> -t ${imageName} .
Create a shared library file for dockerBuild.groovy
Create another file for trivyImage.groovy
Push the above files to the GitHub shared library.
Add this stage to your pipeline with parameters
Build now with parameters
Step8C: Run the Docker container
Create a new file runContainer.groovy
Create Another file to remove container removeContainer.groovy
Push them to the Shared library GitHub repo
Add the below stages to the Pipeline
Build with parameters 'create'
It will start the container
Output-
Build with parameters 'delete'
It will stop and remove the Container
Step9A: Kubernetes Setup
Connect your machines to Putty or Mobaxtreme
Take-Two Ubuntu 20.04(t2.medium) instances one for k8s master and the other one for worker.
Install Kubectl on Jenkins machine also.
Step9B: Kubectl is to be installed on Jenkins
Connect your Jenkins machine
Create a shell script file kube.sh
Paste the below commands
Step9C: K8S Master-Slave setup
Part 1 ----------Master Node------------
----------Worker Node------------
Part 2 ------------Both Master & Node ------------
Part 3 --------------- Master ---------------
----------Worker Node------------
Copy the config file to Jenkins master or the local file manager and save it
copy it and save it in documents or another folder save it as secret-file.txt
Note: create a secret-file.txt in your file explorer save the config in it and use this at the kubernetes credential section.
Install Kubernetes Plugin, Once it's installed successfully.
goto manage Jenkins --> manage credentials --> Click on Jenkins global --> add credentials
Step9D: Install Helm & Monitoring K8S using Prometheus and Grafana
On Kubernetes Master install the helm
See the Helm version
Add Prometheus Helm repo
Create Prometheus namespace
Install kube-Prometheus-stack
Below is the command to install kube-Prometheus-stack. The helm repo kube-stack-Prometheus (formerly Prometheus-operator) comes with a Grafana deployment embedded.
Let's check if the Prometheus and Grafana pods are running or not
Now See the services
This confirms that Prometheus and grafana have been installed successfully using Helm.
To make Prometheus and grafana available outside the cluster, use LoadBalancer or NodePort instead of ClusterIP.
Edit Prometheus Service
kubectl edit svc stable-kube-prometheus-sta-prometheus -n prometheus
Edit Grafana Service
Verify if the service is changed to LoadBalancer and also get the Load BalancerPorts.
Access Grafana UI in the browser
Get the external IP from the above screenshot and put it in the browser
Create a Dashboard in Grafana
In Grafana, we can create various kinds of dashboards as per our needs.
How to Create Kubernetes Monitoring Dashboard?
For creating a dashboard to monitor the cluster:
Click the '+' button on the left panel and select ‘Import’.
Enter the 15661 dashboard id under Grafana.com Dashboard.
Click ‘Load’.
Select ‘Prometheus’ as the endpoint under the Prometheus data sources drop-down.
Click ‘Import’.
This will show the monitoring dashboard for all cluster nodes
How to Create Kubernetes Cluster Monitoring Dashboard?
For creating a dashboard to monitor the cluster:
Click the '+' button on the left panel and select ‘Import’.
Enter 3119 dashboard ID under Grafana.com Dashboard.
Click ‘Load’.
Select ‘Prometheus’ as the endpoint under the Prometheus data sources drop-down.
Click ‘Import’.
This will show the monitoring dashboard for all cluster nodes
Create a POD Monitoring Dashboard
For creating a dashboard to monitor the cluster:
Click the '+' button on the left panel and select ‘Import’.
Enter 6417 dashboard ID under Grafana.com Dashboard.
Click ‘Load’.
Select ‘Prometheus’ as the endpoint under the Prometheus data sources drop-down.
Click ‘Import’.
Step9E: K8S Deployment
Let's Create a Shared Jenkins library file for K8s deploy and delete
Name kubeDeploy.groovy
To delete deployment
Name kubeDelete.groovy
Let's push them to GitHub
The final stage of the Pipeline
Build Now with parameters 'create'
It will apply the deployment
stage view
Output -
Build with parameter 'delete'
It will destroy Container and Kubernetes deployment.
Dependacy Check Result -
Slack Notifications
Splunk
Grafhana -
Pipeline
Last updated