Skip to content

Home Lab

Notes from my learning sessions

Menu
Menu

Setup Ubuntu 20.04 repository mirror server

Posted on February 20, 2022February 20, 2022 by sandeep

Need: I keep bringing up Kubernetes clusters / destroying them as a part of my learning of Kubernetes. When using Kubespray, I observed the required packages were getting downloaded from the internet. Though I have a decent 300 Mbps internet connectivity, I decided to check if I can have a local repository mirror in my lab.

Searching the internet figured out it was not a complex task to set up. There were two main steps involved. Downloading the repo – around 240G (at the time of typing out this post for Ubuntu 20.04) and the second one configuring either proftpd or apache2 to frontend and serve the files/requests. ( Thanks to https://www.linuxtechi.com/setup-local-apt-repository-server-ubuntu/ ).

Note: Both the steps could be done in a single VM, but I decided to have two different VMs – just a personal choice and no specific technical reason.

Instantiate a VM in which the repository would be downloaded – 400G storage allocated.

Install apt-mirror

apt install -y apt-mirror

Edit the contents of /etc/apt/mirror.list – changed occurrences of ‘artful’ to ‘focal’, my requirement and uncommented backports repository.

####### config ##################
#
# set base_path    /var/spool/apt-mirror
#
# set mirror_path  $base_path/mirror
# set skel_path    $base_path/skel
# set var_path     $base_path/var
# set cleanscript $var_path/clean.sh
# set defaultarch  <running host architecture>
# set postmirror_script $var_path/postmirror.sh
# set run_postmirror 0
set nthreads     40
set _tilde 0
#
############# end config ##############

deb http://archive.ubuntu.com/ubuntu focal main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu focal-security main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu focal-updates main restricted universe multiverse
#deb http://archive.ubuntu.com/ubuntu focal-proposed main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu focal-backports main restricted universe multiverse

deb-src http://archive.ubuntu.com/ubuntu focal main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu focal-security main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu focal-updates main restricted universe multiverse
#deb-src http://archive.ubuntu.com/ubuntu focal-proposed main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu focal-backports main restricted universe multiverse

clean http://archive.ubuntu.com/ubuntu

Download or mirror the repository locally.

The first-time download would take some time depending on the speed of internet connectivity (around 240 G to download).

# nohup apt-mirror

The repository gets downloaded at /var/spool/apt-mirror.   It was recommended to add a corn job that would periodically execute apt-mirror and update the repository locally.  I preferred to update the same as and when required.  Subsequent downloads or executing apt-mirror does not take much time – depends on the delta updates available.

Download CNF directories

In Ubuntu 20.04 LTS, apt-mirror does not sync the CNF directory and its files, so we have to manually download and copy the folder and its files.  Not sure if there would be delta updates to these CNF directories.  Decided to modify the script from the site by adding the reference to backports repository and retaining only for amd64 architecture.

Contents of download_cnf.sh

#!/bin/bash
for p in "${1:-focal}"{,-{security,updates,backports}}\
/{main,restricted,universe,multiverse};
do >&2 echo "${p}"

wget -q -c -r -np -R "index.html*" "http://archive.ubuntu.com/ubuntu/dists/${p}/cnf/Commands-amd64.xz"
done

Executing the script results in a directory “archive.ubuntu.com” getting created with the required files. 

Configure mirror server in the lab

Add DNS entry for “mirror.datachronicles.net” in the DNS server, and instantiate a VM, Ubuntu 20.04, 400G disk.

Install apache2

apt install -y apache2

Create a document (in this case package repository) directory 

mkdir -p /var/www/html/ubuntu

Set the ownership to www-data

chown www-data:www-data /var/www/html/ubuntu

Update the site configuration (/etc/apache2/sites-enabled/datachronicles.net), As I have wild card certificate for the domain, configuring SSL too)

<VirtualHost *:80>
ServerAdmin admin@datachronicles.net
DocumentRoot /var/www/html/ubuntu
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

<VirtualHost _default_:443>
ServerAdmin admin@datachronicles.net
DocumentRoot /var/www/html/ubuntu

ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined

SSLEngine on

SSLCertificateFile /var/www/html/ubuntu/certs/datachronicles.crt
SSLCertificateKeyFile /var/www/html/ubuntu/certs/privatekey.key
SSLCertificateChainFile /var/www/html/ubuntu/certs/datachronicles-chain.crt

<FilesMatch "\.(cgi|shtml|phtml|php)$">
SSLOptions +StdEnvVars
</FilesMatch>
<Directory /usr/lib/cgi-bin>
SSLOptions +StdEnvVars
</Directory>
</VirtualHost>

Enable SSL and restart apache2

a2enmod ssl
systemctl restart apache2

Now from the other VM (where the repository was download need to update the document folder with the repository) – Created an “update.sh” script for the same.  Note : mirror gets resolved to the IP of the mirror VM – DNS services updated internally in the lab.

#!/bin/bash
apt-mirror
rsync -av /var/spool/apt-mirror/var root@mirror:/var/www/html/ubuntu/
rsync -av /var/spool/apt-mirror/skel root@mirror:/var/www/html/ubuntu/
rsync -av /var/spool/apt-mirror/mirror root@mirror:/var/www/html/ubuntu/
rm -rf /root/archive.ubuntu.com
/root/cnf.sh
scp -r archive.ubuntu.com root@mirror:/var/www/html/ubuntu/mirror/
rm -rf /root/archive.ubuntu.com

Updated the base image used with the contents of /etc/apt/sources.list referring to the local mirror.  Spin up a test VM and test

root@test:~# cat /etc/apt/sources.list
deb http://mirror.datachronicles.net/mirror/archive.ubuntu.com/ubuntu focal main restricted
deb http://mirror.datachronicles.net/mirror/archive.ubuntu.com/ubuntu focal-updates main restricted
deb http://mirror.datachronicles.net/mirror/archive.ubuntu.com/ubuntu focal universe
deb http://mirror.datachronicles.net/mirror/archive.ubuntu.com/ubuntu focal-updates universe

deb http://mirror.datachronicles.net/mirror/archive.ubuntu.com/ubuntu focal multiverse
deb http://mirror.datachronicles.net/mirror/archive.ubuntu.com/ubuntu focal-updates multiverse
deb http://mirror.datachronicles.net/mirror/archive.ubuntu.com/ubuntu focal-backports main restricted universe multiverse

deb http://mirror.datachronicles.net/mirror/archive.ubuntu.com/ubuntu focal-security main restricted
deb http://mirror.datachronicles.net/mirror/archive.ubuntu.com/ubuntu focal-security universe
deb http://mirror.datachronicles.net/mirror/archive.ubuntu.com/ubuntu focal-security multiverse


root@test:~# apt update
Hit:1 http://mirror.datachronicles.net/mirror/archive.ubuntu.com/ubuntu focal InRelease
Hit:2 http://mirror.datachronicles.net/mirror/archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:3 http://mirror.datachronicles.net/mirror/archive.ubuntu.com/ubuntu focal-backports InRelease
Hit:4 http://mirror.datachronicles.net/mirror/archive.ubuntu.com/ubuntu focal-security InRelease
Reading package lists... Done
Building dependency tree
Reading state information... Done
All packages are up to date.
root@test:~#

That’s it!!!!

Recent Posts

  • Ceph + KVM: 4. Orchestrating Ceph RBD backed VMs on KVM Hosts
  • Rabbit MQ Cluster + HAProxy + Keepalived
  • Install and configure MariaDB / Galera cluster
  • Ceph + KVM : 3. Installing KVM, cutsomized monitoring scripts
  • Ceph + KVM : 5. Service checks and CLI commands
  • Ceph + KVM : 2. Installation – Ceph Storage
  • Ceph + KVM : 1. Planning and preparing for Ceph Storage
  • Openstack Xena on Ubuntu 20.04 – Cinder
  • Preparing custom Debian 11 MATE image
  • Setup Ubuntu 20.04 repository mirror server

Archives

  • April 2025
  • March 2025
  • October 2024
  • September 2024
  • April 2022
  • March 2022
  • February 2022
  • December 2021
  • October 2021
  • September 2021
  • October 2020
  • February 2020
  • January 2020
  • December 2019
© 2025 Home Lab | Powered by Minimalist Blog WordPress Theme