En ce moment 2/3 des synchronisations échouent, car le serveur de référence de Canonical a trop de connexions simultanées.

Le message d'erreur est :
@ERROR: max connections (75) reached -- try again later
rsync error: error starting client-server protocol (code 5) at main.c(1653) [Receiver=3.1.1]
75 connexions simultanées en rsync est la limite haute.


J'ai fait une nouvelle version du script qui en cas d'échec va faire une seconde tentative, 4 minutes plus tard :

#!/bin/dash

fatal() {
  echo "$1"
  exit 1
}

warn() {
  echo "$1"
}

# Find a source mirror near you which supports rsync on
# https://launchpad.net/ubuntu/+archivemirrors
# rsync://<iso-country-code>.rsync.archive.ubuntu.com/ubuntu should always work
RSYNCSOURCE=rsync://fr.archive.ubuntu.com/ubuntu

# Define where you want the mirror-data to be on your mirror
BASEDIR=/home/ubuntu-archive/ubuntu/

if [ ! -d ${BASEDIR} ]; then
  warn "${BASEDIR} does not exist yet, trying to create it..."
  mkdir -p ${BASEDIR} || fatal "Creation of ${BASEDIR} failed."
fi

attempt="1"

# First sync with bit rate limited to 41 Mb/s: add
#  --bwlimit=5000 --progress \
# Step 1 of 2
date -R
echo "#################### SYNC STEP 1 OF 2 ####################"
echo "exclude: Packages*, Sources*, Release*, InRelease; .~tmp~"
rsync --verbose --out-format="%o %b bytes for sync %n (%l bytes)" \
  --recursive --times --links --safe-links --hard-links --perms \
  --stats \
  --exclude "/dists" \
  --exclude "Packages*" --exclude "Sources*" \
  --exclude "Release*" --exclude "InRelease" \
  --exclude ".~tmp~" \
  --exclude ".trace" \
  ${RSYNCSOURCE} ${BASEDIR} || attempt="2"

if [ "$attempt" = "2" ]; then
  echo "!!!!!!!!!!!!!!!!!!! ERROR : SYNC STEP 1 OF 2 HAS FAILED (1st attempt) !!!!!!!!!!!!!!!!!!!"
  echo "ERROR : SYNC STEP 1 OF 2 HAS FAILED (1st attempt)" >&2
  echo "-------------------------------------------------" >&2
  date -R
  echo "Second attempt after 4 minutes (240 seconds) of pause"
  sleep 240
  date -R
  echo "#################### SYNC STEP 1 OF 2 (2nd attempt) ####################"
  echo "exclude: Packages*, Sources*, Release*, InRelease; .~tmp~"
  rsync --verbose --out-format="%o %b bytes for sync %n (%l bytes)" \
    --recursive --times --links --safe-links --hard-links --perms \
    --stats \
    --exclude "/dists" \
    --exclude "Packages*" --exclude "Sources*" \
    --exclude "Release*" --exclude "InRelease" \
    --exclude ".~tmp~" \
    --exclude ".trace" \
    ${RSYNCSOURCE} ${BASEDIR} || attempt="3"
fi

if [ "$attempt" = "3" ]; then
  echo "!!!!!!!!!!!!!!!!!!! ERROR : SYNC STEP 1 OF 2 HAS FAILED (2nd attempt) !!!!!!!!!!!!!!!!!!!"
  echo "ERROR : SYNC STEP 1 OF 2 HAS FAILED (2nd attempt)" >&2
  echo "-------------------------------------------------" >&2
  date -R
  echo "Third attempt after 10 minutes (600 seconds) of pause"
  sleep 600
  date -R
  echo "#################### SYNC STEP 1 OF 2 (3rd attempt) ####################"
  echo "exclude: Packages*, Sources*, Release*, InRelease; .~tmp~"
  rsync --verbose --out-format="%o %b bytes for sync %n (%l bytes)" \
    --recursive --times --links --safe-links --hard-links --perms \
    --stats \
    --exclude "/dists" \
    --exclude "Packages*" --exclude "Sources*" \
    --exclude "Release*" --exclude "InRelease" \
    --exclude ".~tmp~" \
    --exclude ".trace" \
    ${RSYNCSOURCE} ${BASEDIR} || fatal "!!!!!!!!!!!!!!!!!!! ERROR : SYNC STEP 1 OF 2 HAS FAILED (3rd attempt) !!!!!!!!!!!!!!!!!!!"
fi

echo "#################### SYNC STEP 1 OF 2 IS OK WITH $attempt attempt ####################"
date -R


# Step 2 of 2
attempt="1"
echo "#################### SYNC STEP 2 OF 2 ####################"
echo "SYNC Packages* + Sources* + Release* + InRelease"
echo "exclude: .~tmp~"
rsync --verbose --out-format="%o %b bytes for sync %n (%l bytes)" \
  --recursive --times --links --safe-links --hard-links --perms \
  --stats --delete --delete-after \
  --exclude ".~tmp~" \
  ${RSYNCSOURCE} ${BASEDIR} || attempt="2"

if [ "$attempt" = "2" ]; then
  echo "!!!!!!!!!!!!!!!!!!! ERROR : SYNC STEP 2 OF 2 HAS FAILED (1st attempt) !!!!!!!!!!!!!!!!!!!"
  echo "ERROR : SYNC STEP 2 OF 2 HAS FAILED (1st attempt)" >&2
  echo "-------------------------------------------------" >&2
  date -R
  echo "Second attempt after 4 minutes (240 seconds) of pause"
  sleep 240
  date -R
  echo "#################### SYNC STEP 2 OF 2 (2nd attempt) ####################"
  echo "SYNC Packages* + Sources* + Release* + InRelease"
  echo "exclude: .~tmp~"
  rsync --verbose --out-format="%o %b bytes for sync %n (%l bytes)" \
    --recursive --times --links --safe-links --hard-links --perms \
    --stats --delete --delete-after \
    --exclude ".~tmp~" \
    ${RSYNCSOURCE} ${BASEDIR} || attempt="3"
fi

if [ "$attempt" = "3" ]; then
  echo "!!!!!!!!!!!!!!!!!!! ERROR : SYNC STEP 2 OF 2 HAS FAILED (2nd attempt !!!!!!!!!!!!!!!!!!!"
  echo "ERROR : SYNC STEP 2 OF 2 HAS FAILED (2nd attempt)" >&2
  echo "-------------------------------------------------" >&2
  date -R
  echo "Third attempt after 10 minutes (600 seconds) of pause"
  sleep 600
  date -R
  echo "#################### SYNC STEP 2 OF 2 (3rd attempt) ####################"
  echo "SYNC Packages* + Sources* + Release* + InRelease"
  echo "exclude: .~tmp~"
  rsync --verbose --out-format="%o %b bytes for sync %n (%l bytes)" \
    --recursive --times --links --safe-links --hard-links --perms \
    --stats --delete --delete-after \
    --exclude ".~tmp~" \
    ${RSYNCSOURCE} ${BASEDIR} || fatal "!!!!!!!!!!!!!!!!!!! ERROR : SYNC STEP 2 OF 2 HAS FAILED (3rd attempt) !!!!!!!!!!!!!!!!!!!"
fi

echo "#################### SYNC STEP 2 OF 2 IS OK WITH $attempt attempt ####################"
date -R > ${BASEDIR}/project/trace/fr.archive.ubuntu.com
date -R

Cela fonctionne bien.



Vous pouvez suivre les log des synchronisations sur https://bouyguestelecom.ubuntu.lafibre.info/log/

(c'est Canonical qui déclenche la synchronisation à chaque mise à jour)

Exemple de log :
Contact: Vivien GUEANT => Twitter https://twitter.com/lafibreinfo
LaFibre.info => https://lafibre.info/tutoriels-linux
Sun, 18 Mar 2018 12:38:02 +0100
#################### SYNC STEP 1 OF 2 ####################
exclude: Packages*, Sources*, Release*, InRelease; .~tmp~
This is an Ubuntu mirror - treat it kindly

receiving incremental file list
[...]
Number of files: 713,111 (reg: 651,374, dir: 50,121, link: 11,616)
Number of created files: 199 (reg: 199)
Number of deleted files: 0
Number of regular files transferred: 379
Total file size: 873,589,711,200 bytes
Total transferred file size: 3,343,448,792 bytes
Literal data: 3,075,746,443 bytes
Matched data: 267,702,349 bytes
File list size: 21,584,905
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 734,224
Total bytes received: 3,103,138,673

sent 734,224 bytes  received 3,103,138,673 bytes  63,997,379.32 bytes/sec
total size is 873,589,711,200  speedup is 281.45
#################### SYNC STEP 1 OF 2 IS OK, WITH 1 ATTEMPT ####################
Sun, 18 Mar 2018 12:38:50 +0100
#################### SYNC STEP 2 OF 2 ####################
SYNC Packages* + Sources* + Release* + InRelease
exclude: .~tmp~
This is an Ubuntu mirror - treat it kindly
@ERROR: max connections (75) reached -- try again later
rsync error: error starting client-server protocol (code 5) at main.c(1653) [Receiver=3.1.1]
!!!!!!!!!!!!!!!!!!! ERROR : SYNC STEP 2 OF 2 HAS FAILED (1st attempt) !!!!!!!!!!!!!!!!!!!
Sun, 18 Mar 2018 12:38:50 +0100
Second attempt after 4 minutes (240 seconds) of pause
Sun, 18 Mar 2018 12:42:50 +0100
#################### SYNC STEP 2 OF 2 (2nd attempt) ####################
SYNC Packages* + Sources* + Release* + InRelease
exclude: .~tmp~
This is an Ubuntu mirror - treat it kindly
@ERROR: max connections (75) reached -- try again later
rsync error: error starting client-server protocol (code 5) at main.c(1653) [Receiver=3.1.1]
!!!!!!!!!!!!!!!!!!! ERROR : SYNC STEP 2 OF 2 HAS FAILED (2nd attempt !!!!!!!!!!!!!!!!!!!
Sun, 18 Mar 2018 12:42:50 +0100
Third attempt after 10 minutes (600 seconds) of pause
Sun, 18 Mar 2018 12:52:50 +0100
#################### SYNC STEP 2 OF 2 (3rd attempt) ####################
SYNC Packages* + Sources* + Release* + InRelease
exclude: .~tmp~
This is an Ubuntu mirror - treat it kindly

receiving file list ... done
[...]
Number of files: 739,265 (reg: 673,071, dir: 53,745, link: 12,449)
Number of created files: 465 (reg: 465)
Number of deleted files: 1,115 (reg: 1,115)
Number of regular files transferred: 5,596
Total file size: 923,368,835,315 bytes
Total transferred file size: 3,478,372,685 bytes
Literal data: 923,505,065 bytes
Matched data: 2,554,867,620 bytes
File list size: 27,136,261
File list generation time: 2.652 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 9,220,280
Total bytes received: 956,829,786

sent 9,220,280 bytes  received 956,829,786 bytes  30,668,256.06 bytes/sec
total size is 923,368,835,315  speedup is 955.82
#################### SYNC STEP 2 OF 2 IS OK, WITH 3 ATTEMPT ####################
Sun, 18 Mar 2018 12:53:24 +0100