Skip to content

Commit

Permalink
group updates to public branch
Browse files Browse the repository at this point in the history
  • Loading branch information
lj020326 committed Mar 25, 2024
1 parent f8e9859 commit 67b54da
Show file tree
Hide file tree
Showing 556 changed files with 13,662 additions and 3,012 deletions.
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -33,4 +33,3 @@ Berksfile.lock

save/
.idea/
.DS_Store
103 changes: 83 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,38 @@
# Dettonville Packer Builds

This project contains the Packer build configurations for all of Dettonville Boxes. Each box builds a minimal base box for use with Vsphere.
Table of Contents:

You can also fork this repository and customize a build configuration with your own Ansible roles and playbooks to build a fully custom box using Packer.
* [Information/Notes on VM Template build automation](#informationnotes-on-vm-template-build-automation)
* [1 - VM Template Build Definition Repository](#1-vm-template-build-definition-repository)
* [1.1 - VM Template Build Configurations](#1-vm-template-build-configurations)
* [2 - VM Template Build Automation from Jenkins](#2-vm-template-build-automation-from-jenkins)
* [Jenkins Project Configurations stored in YAML file format](#jenkins-project-configurations-stored-in-yaml-file-format)
* [3 - Ansible role to setup docker jenkins control node](#3-ansible-role-to-setup-docker-jenkins-control-node)
* [Setup pipeline automation library used by all jenkins jobs](#setup-pipeline-automation-library-used-by-all-jenkins-jobs)
* [The `vars/buildVmTemplate.groovy` functionality](#the-varsbuildvmtemplategroovy-functionality)
* [Other related/supporting information](#other-relatedsupporting-information)
* [To refresh template configurations using json sourced files](#to-refresh-template-configurations-using-json-sourced-files)
* [To Refresh/Update/Repair submodules](#to-refreshupdaterepair-submodules)
* [Use script to automate / simplify refreshing/repairing submodules](#use-script-to-automate--simplify-refreshingrepairing-submodules)
* [Reference](#reference)


# Information/Notes on VM Template build automation<a name="informationnotes-on-vm-template-build-automation"></a>

This project contains the Packer build configurations for vmware templates.
Each template build configuration includes a minimal template definition for use with Vsphere.

![vm-template-build-pipelines](./docs/img/vm-template-build-pipelines.drawio.svg)

You can also fork this repository and customize a build configuration with your own Ansible roles and playbooks to build a fully custom template using Packer.

### GitHub Actions

[![Template Validation](https://github.com/lj020326/packer-templates/actions/workflows/main.yml/badge.svg)](https://github.com/lj020326/packer-templates/actions/workflows/main.yml)
[![Template Validation](https://github.com/lj020326/vm-templates/actions/workflows/main.yml/badge.svg)](https://github.com/lj020326/vm-templates/actions/workflows/main.yml)
[![License](https://img.shields.io/badge/license-GPLv3-brightgreen.svg?style=flat)](LICENSE)

## Requirements

The following software must be installed/present on your local machine before you can use Packer to build any of these Vagrant boxes:
The following software must be installed/present on your local machine before you can use Packer to build any of these Vagrant templatees:

- [Packer](http://www.packer.io/)
- [Ansible](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html)
Expand All @@ -20,52 +41,94 @@ I use docker containers for my jenkins build agents which have ansible and packe
The [specification for the docker agent container image can be found here](https://github.com/lj020326/jenkins-docker-agent/blob/master/image/jenkins-cicd-agent/Dockerfile).
The container leverages the 'jenkins/inbound-agent' image and the [cicd-build-tool image defined here](https://github.com/lj020326/jenkins-docker-agent/blob/master/image/cicd-build-tools/Dockerfile).

## Jenkins Pipeline Usage
## 1 - VM Template Build Definition Repository<a name="1-vm-template-build-definition-repository"></a>

The vm template repository can be found at:
https://github.com/lj020326/vm-templates

## 1.1 - VM Template Build Configurations<a name="1-vm-template-build-configurations"></a>

Detailed information for the [template build configuration parameters and options can be found here](./docs/configuration.md).

## 2 - VM Template Build Automation from Jenkins<a name="2-vm-template-build-automation-from-jenkins"></a>

All vm template builds are launched using jenkins pipeline jobs.

The jenkins controller docker instance is setup using the __jenkins config-as-code (jcac) plugin__ to automate the creation of all demo pipelines.

The jenkins pipeline repository can be found at:
https://github.com/lj020326/pipeline-automation-lib

All the jenkins projects and job pipelines are created from the job dsl script files in the folder at:
https://github.com/lj020326/pipeline-automation-lib/tree/main/jobs/jobdsl/templates

The job dsl script file used to create all the vm build pipeline jobs is at:
https://github.com/lj020326/pipeline-automation-lib/blob/main/jobs/jobdsl/templates/01_INFRA/init02_vm_templates.groovy


### Jenkins Project Configurations stored in YAML file format<a name="jenkins-project-configurations-stored-in-yaml-file-format"></a>

Each jenkins infrastructure project folder has the respective configuration state stored in corresponding YAML files seen in the top level folder:
https://github.com/lj020326/pipeline-automation-lib/tree/main/jobs/jobdsl/templates/01_INFRA

As an example see the configuration for the vm-template pipelines at:
https://github.com/lj020326/pipeline-automation-lib/blob/main/jobs/jobdsl/templates/01_INFRA/config.vm-template-jobs.yml

## 3 - Ansible role to setup docker jenkins control node<a name="3-ansible-role-to-setup-docker-jenkins-control-node"></a>

An ansible role is used to setup all docker stack instances.

The `docker-stack` ansible role used to stand up the docker stack [can be found here](https://github.com/lj020326/ansible-datacenter/tree/main/roles/docker-stack).

The [`docker-stack` ansible role](https://github.com/lj020326/ansible-datacenter/tree/main/roles/docker-stack) contains the [__jenkins config-as-code (jcac) yaml definition__](https://github.com/lj020326/ansible-datacenter/blob/main/roles/docker-stack/templates/jenkins_jcac/jenkins_casc.yml.j2) in template form used to setup the jenkins jcac instance.

[The jcac definition can be found here](https://github.com/lj020326/ansible-datacenter/blob/main/roles/docker-stack/templates/jenkins_jcac/jenkins_casc.yml.j2).

### Setup pipeline automation library used by all jenkins jobs<a name="setup-pipeline-automation-library-used-by-all-jenkins-jobs"></a>

### Setup pipeline automation library used by all jenkins jobs
The pipeline automation library used can be found [here](https://github.com/lj020326/pipeline-automation-lib).
[The pipeline automation library](https://github.com/lj020326/pipeline-automation-lib) defines the shared jenkins templates that are used throughout all of the jenkins ansible pipelines.

Configure the library in jenkins as seen below.
![Jenkins - Configure shared Pipeline Automation Library](./docs/screenshots/packer-templates-00-pipeline-library.png)
![Jenkins - Configure shared Pipeline Automation Library](./docs/img/screenshots/packer-templates-00-pipeline-library.png)


### Site Root folder

A root folder for the packer-templates environment can be setup similar to the following.
![Site Root](./docs/screenshots/packer-templates-0-toplevel.png)
![Site Root](./docs/img/screenshots/packer-templates-0-toplevel.png)

### VM Packer Template OS folders

Then setup folders for each VM OS defined as follows.
![Site Environments](./docs/screenshots/packer-templates-1a-os.png)
![Site Environments](./docs/img/screenshots/packer-templates-1a-os.png)


### VM Packer Template OS Version folders

Then setup folders for each VM OS Version defined as follows.
![Site Environments](./docs/screenshots/packer-templates-1a-os-versions.png)
![Site Environments](./docs/screenshots/packer-templates-1a-os-version-builds.png)
![Site Environments](./docs/img/screenshots/packer-templates-1a-os-versions.png)
![Site Environments](./docs/img/screenshots/packer-templates-1a-os-version-builds.png)

### VM Packer Template builder pipeline folders

For each template build node at the leaf level, setup a jenkins folder to run the groovy pipeline wrapper used to start the packer VM template builder process.

The folder definition for each VM template build folder should be as follows.
![Build Template Pipeline Folders](./docs/screenshots/packer-templates-1a-template-build-wrapper.png)
![Build Template Pipeline Folders](./docs/img/screenshots/packer-templates-1a-template-build-wrapper.png)

See the job console for the VM packer template build pipeline output.
![Job Console Output](./docs/screenshots/packer-templates-1a-template-build-output.png)
![Job Console Output](./docs/img/screenshots/packer-templates-1a-template-build-output.png)

The detailed [pipeline job console output](./docs/jenkins-consoleText-packer-build-ubuntu-log.md).

### Setup job to run all build template pipelines

Then setup a pipeline job at the top level folder to run the [runAllBuilds.groovy](./runAllBuilds.groovy) as seen below.
![Job Definition](./docs/screenshots/packer-templates-2a-run-all-builds-config.png)
![Job Definition](./docs/img/screenshots/packer-templates-2a-run-all-builds-config.png)

This job will launch all of the VM template build jobs defined in [runAllBuilds.groovy](./runAllBuilds.groovy) with output similar to below.
![Job Console Output](./docs/screenshots/packer-templates-2b-run-all-builds-output.png)
![Job Console Output](./docs/img/screenshots/packer-templates-2b-run-all-builds-output.png)

## To repair submodule

Expand Down Expand Up @@ -99,7 +162,7 @@ sudo -S -E bash /tmp/script_4278.sh
```shell
# find the temp dir used for the ansible-local provisioner from the packer log
$ cd /tmp/packer-provisioner-ansible-local/63b193ab-d1c4-b355-f4cf-9e9153570896
$ ansible-playbook bootstrap_vm_template.yml --vault-password-file=~/.vault_pass -c local -i vm_template.yml
$ ansible-playbook bootstrap_vm_template.yml --vault-password-file=~/.vault_pass --limit vm_template -e @./vars/vault.yml -c local -i hosts.yml
```

## Notes, Considerations & Final Thoughts...
Expand All @@ -111,10 +174,10 @@ If defining the ansible playbooks as a submodule the same way done in this repo,
* https://github.com/chef/bento
* https://github.com/burkeazbill/ubuntu-22-04-packer-fusion-workstation/blob/master/http/user-data
* https://github.com/williamsanmartin/packer-template-ubuntu/blob/main/http/user-data
* [vagrant-box-templates](https://github.com/mrlesmithjr/vagrant-box-templates)
* [vagrant-template-templates](https://github.com/mrlesmithjr/vagrant-template-templates)
* https://github.com/mwrock/packer-templates
* https://github.com/jacqinthebox/packer-templates
* https://github.com/geerlingguy/packer-boxes
* https://github.com/jacqinthetemplate/packer-templates
* https://github.com/geerlingguy/packer-templatees

## Contact

Expand Down
2 changes: 1 addition & 1 deletion ansible
4 changes: 1 addition & 3 deletions buildTemplate.groovy
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
#!/usr/bin/env groovy

Map config = [:]

buildVmTemplate(config)
buildVmTemplate()
Loading

0 comments on commit 67b54da

Please sign in to comment.