The Jenkins master-slave architecture is designed to distribute the workload of building, testing, and deploying applications across multiple machines, thereby improving the efficiency of CI/CD pipelines. The architecture consists of two main components: the Jenkins master and the Jenkins slaves (also known as agents).
Jenkins Master:
The master serves as the central control point for managing and coordinating jobs.It stores the configuration, schedules builds, and monitors the overall health of the CI/CD environment. The master is responsible for storing the job definitions, plugins, and other settings.
Jenkins Slaves/Agents:
An agent is typically a machine or container that connects to a Jenkins master and this agent that actually execute all the steps mentioned in a Job. When you create a Jenkins job, you have to assign an agent to it. Every agent has a label as a unique identifier.
When you trigger a Jenkins job from the master, the actual execution happens on the agent node that is configured in the job.
Benefits of the Jenkins Master-Slave Architecture
Scalability: As projects grow in complexity and scale, a single Jenkins master might struggle to handle the increasing workload. By using a master-slave architecture, the load can be distributed across multiple slaves, allowing the system to easily scale up or down based on demand. This ensures that build times remain manageable even as the project size expands.
Parallel Execution: With multiple slaves available, Jenkins can execute multiple jobs in parallel. This significantly reduces build and test times, leading to quicker feedback loops and faster delivery of software updates. Parallel execution is especially valuable for large projects with extensive testing requirements.
Isolation: Jenkins slaves can be set up on different machines or even in different environments, providing isolation for jobs that require distinct conditions. For instance, you might need a specific slave configured for running tests on a particular operating system or against a specific database.
High Availability: If one slave goes down, the others can continue executing tasks, minimizing downtime.
Reduced Master Load: Offloading the execution tasks to slaves helps reduce the load on the Jenkins master. This enhances the master's performance and responsiveness, ensuring that it remains available for job scheduling and configuration management.
Setting Up Jenkins Master-Slave Architecture
Launch 2 instances of Jenkins-Master and Jenkins-Slave on AWS console; I prefer ubuntu t2.micro which are free to use and fulfill our criteria.
Now login to both instances. Set up the Jenkins master with Java and Jenkins installation, and set up the slave node with Java installation only*.*
Follow https://kshitijaa.hashnode.dev/getting-started-with-jenkins for the installation guide.
Connect Master and Slave: Agents can be configured to connect to the master using a unique secret key as explained below.
By default in the Ubuntu system there is a .ssh hidden folder, (/home/ubuntu/.ssh) go inside and generate a key pair using the command ssh-keygen. After running this command it will generate 2 keys id_rsa (private key) and id_rsa.pub (public key).
Private key and public key together are called as Key-Pair.
Follow me: Login to Jenkins master server --> cd /home/ubuntu/.ssh/ --> ssh-keygen --> ls --> cat id_rsa.pub --> copy content
Login to Jenkins Slave server --> vim /home/ubuntu/.ssh/authorized_keys --> paste from master public key and save. --> verify using --> cat /home/ubuntu/.ssh/authorized_keys
Zoom to see below image that, keys are same in both the server.
BOOM! Jenkins Master-Slave are connected at Instance level!
Go to Jenkins web interface: JenkinsServerMaster_IPAddress:8080 You can configure node as following
Click on Manage Jenkins --> Set up agnet
Add Node name -->Type-Permanent Agnet --> Click on Create
See the Node name as mentioned above. Add Description, Number of executors - 2, Remote root directory /home/ununtu/jenkins , Label - Jenkins_Agent
Scroll Down and be with me, little more to go!
Usage: Use this node as much as possible, Launch method: Launch agents via SSH, Host: Public IP address of Agent , Credentials This is an important part, Add it carefully.
Credentials Part defined below. Kind: SSH Username with Private key, Scope: Global, ID:jenkins-agent, Username: ubuntu,
See Carefully Below: We have to now add Private Key
Added Credentials now. Host Key Verification Startegy: Non Verifying Verificvation Strategy, Click on Save
And Launch Instance.
Agent Connection has Established!
Zoom Below Image to see on left down corner; as heilighted Configured Agent is Present.
Conclusion
The Jenkins master-slave architecture plays a pivotal role in achieving scalability and efficiency in modern CI/CD pipelines. By distributing the workload across multiple nodes, it enables parallel execution, efficient resource utilization, and isolation for different types of jobs. As software development continues to evolve, mastering the intricacies of the Jenkins master-slave architecture empowers teams to deliver high-quality software faster and with greater reliability.