La Fibre
Datacenter et équipements réseaux => Datacenter => Datacenter => Discussion démarrée par: Anonyme le 12 janvier 2022 à 21:12:51
-
Nvidia propose un datacenter virtuel.
https://air.nvidia.com/SimulationsAll
"
Your Cumulus VXLAN EVPN Symmetric virtual data center is ready!
You can access it directly at https://air.nvidia.com/
Your virtual data center will automatically go to sleep on January 13, 2022 at 07:28 UTC but you can extend this using the "Add Time" button.
Your virtual data center will expire on January 15, 2022 at 19:27 UTC, but don't worry! You can always come back and request another one.
Please feel free to contact us at citc-support@nvidia.com if you have any questions or need assistance.
Thank you,
The NVIDIA Networking Team "
-
L'intéret est de comprendre l'architecture.
Des AS privés, un routage BGP interne sont utilisés pour le routage.
pine01 login: cumulus
Password:
Last login: Wed Aug 25 09:45:59 PDT 2021 from 192.168.200.1 on pts/0
Linux spine01 4.19.0-cl-1-amd64 #1 SMP Debian 4.19.176-1+cl4.4.0u1 (2021-06-25) x86_64
#########################################################
Successfully logged in to: spine01
#########################################################
cumulus@spine01:mgmt:~$ net show bgp
<ipv4/prefixlen> : An IPv4 address and prefix length
<ipv4> : an IPv4 address
<ipv6/prefixlen> : An IPv6 address and prefix length
<ipv6> : An IPv6 address
evpn : Ethernet VPN
ipv4 : Internet Protocol version 4
ipv6 : Internet Protocol version 6
json : Print output in json
l2vpn : Layer-2 VPN
large-community-info : add help text
neighbor : A BGP, OSPF, PIM, etc neighbor
nexthop : BGP nexthops
summary : Summary
update-groups : BGP update groups
vrf : Virtual routing and forwarding
<ENTER>
cumulus@spine01:mgmt:~$ net show bgp ne
neighbor : A BGP, OSPF, PIM, etc neighbor
nexthop : BGP nexthops
cumulus@spine01:mgmt:~$ net show bgp neighbor
BGP neighbor on swp1: fe80::4638:39ff:fe00:2, remote AS 65101, local AS 65100, external link
Hostname: leaf01
Member of peer-group underlay for session parameters
BGP version 4, remote router ID 10.10.10.1, local router ID 10.10.10.101
BGP state = Established, up for 06:26:23
Last read 00:00:01, Last write 00:00:02
Hold time is 9, keepalive interval is 3 seconds
Neighbor capabilities:
4 Byte AS: advertised and received
AddPath:
IPv4 Unicast: RX advertised IPv4 Unicast and received
L2VPN EVPN: RX advertised L2VPN EVPN and received
Extended nexthop: advertised and received
Address families by peer:
IPv4 Unicast
Route refresh: advertised and received(old & new)
Address Family IPv4 Unicast: advertised and received
Address Family L2VPN EVPN: advertised and received
Hostname Capability: advertised (name: spine01,domain name: n/a) received (name: leaf01,domain name: n/a)
Graceful Restart Capability: advertised and received
Remote Restart timer is 120 seconds
Address families by peer:
none
Graceful restart information:
End-of-RIB send: IPv4 Unicast, L2VPN EVPN
End-of-RIB received: IPv4 Unicast, L2VPN EVPN
Local GR Mode: Helper*
Remote GR Mode: Helper
R bit: False
Timers:
Configured Restart Time(sec): 120
Received Restart Time(sec): 120
IPv4 Unicast:
F bit: False
End-of-RIB sent: Yes
End-of-RIB sent after update: No
End-of-RIB received: Yes
Timers:
Configured Stale Path Time(sec): 360
Message statistics:
Inq depth is 0
Outq depth is 0
Sent Rcvd
Opens: 1 1
Notifications: 0 0
Updates: 270 250
Keepalives: 7727 7726
Route Refresh: 0 0
Capability: 0 0
Total: 7998 7977
Minimum time between advertisement runs is 0 seconds
For address family: IPv4 Unicast
underlay peer-group member
Update group 1, subgroup 1
Packet Queue length 0
Community attribute sent to this neighbor(all)
3 accepted prefixes
For address family: L2VPN EVPN
underlay peer-group member
Update group 2, subgroup 2
Packet Queue length 0
NEXT_HOP is propagated unchanged to this neighbor
Community attribute sent to this neighbor(all)
21 accepted prefixes
Connections established 1; dropped 0
Last reset 06:26:24, Waiting for peer OPEN
Local host: fe80::4638:39ff:fe00:1, Local port: 54564
Foreign host: fe80::4638:39ff:fe00:2, Foreign port: 179
Nexthop: 10.10.10.101
Nexthop global: fe80::4638:39ff:fe00:1
Nexthop local: fe80::4638:39ff:fe00:1
BGP connection: shared network
BGP Connect Retry Timer in Seconds: 10
Read thread: on Write thread: on FD used: 27
BGP neighbor on swp2: fe80::4638:39ff:fe00:a, remote AS 65102, local AS 65100, external link
Hostname: leaf02
Member of peer-group underlay for session parameters
BGP version 4, remote router ID 10.10.10.2, local router ID 10.10.10.101
BGP state = Established, up for 06:26:23
Last read 00:00:02, Last write 00:00:02
Hold time is 9, keepalive interval is 3 seconds
Neighbor capabilities:
4 Byte AS: advertised and received
AddPath:
IPv4 Unicast: RX advertised IPv4 Unicast and received
L2VPN EVPN: RX advertised L2VPN EVPN and received
Extended nexthop: advertised and received
Address families by peer:
IPv4 Unicast
Route refresh: advertised and received(old & new)
Address Family IPv4 Unicast: advertised and received
Address Family L2VPN EVPN: advertised and received
Hostname Capability: advertised (name: spine01,domain name: n/a) received (name: leaf02,domain name: n/a)
Graceful Restart Capability: advertised and received
Remote Restart timer is 120 seconds
Address families by peer:
none
Graceful restart information:
End-of-RIB send: IPv4 Unicast, L2VPN EVPN
End-of-RIB received: IPv4 Unicast, L2VPN EVPN
Local GR Mode: Helper*
Remote GR Mode: Helper
R bit: False
Timers:
Configured Restart Time(sec): 120
Received Restart Time(sec): 120
IPv4 Unicast:
F bit: False
End-of-RIB sent: Yes
End-of-RIB sent after update: No
End-of-RIB received: Yes
Timers:
Configured Stale Path Time(sec): 360
Message statistics:
Inq depth is 0
Outq depth is 0
Sent Rcvd
Opens: 1 1
Notifications: 0 0
Updates: 270 254
Keepalives: 7727 7726
Route Refresh: 0 0
Capability: 0 0
Total: 7998 7981
Minimum time between advertisement runs is 0 seconds
For address family: IPv4 Unicast
underlay peer-group member
Update group 1, subgroup 1
Packet Queue length 0
Community attribute sent to this neighbor(all)
3 accepted prefixes
For address family: L2VPN EVPN
underlay peer-group member
Update group 2, subgroup 2
Packet Queue length 0
NEXT_HOP is propagated unchanged to this neighbor
Community attribute sent to this neighbor(all)
21 accepted prefixes
Connections established 1; dropped 0
Last reset 06:26:24, Waiting for peer OPEN
Local host: fe80::4638:39ff:fe00:9, Local port: 51518
Foreign host: fe80::4638:39ff:fe00:a, Foreign port: 179
Nexthop: 10.10.10.101
Nexthop global: fe80::4638:39ff:fe00:9
Nexthop local: fe80::4638:39ff:fe00:9
BGP connection: shared network
BGP Connect Retry Timer in Seconds: 10
Read thread: on Write thread: on FD used: 24
BGP neighbor on swp3: fe80::4638:39ff:fe00:12, remote AS 65103, local AS 65100, external link
Hostname: leaf03
Member of peer-group underlay for session parameters
BGP version 4, remote router ID 10.10.10.3, local router ID 10.10.10.101
BGP state = Established, up for 06:26:23
Last read 00:00:01, Last write 00:00:02
Hold time is 9, keepalive interval is 3 seconds
Neighbor capabilities:
4 Byte AS: advertised and received
AddPath:
IPv4 Unicast: RX advertised IPv4 Unicast and received
L2VPN EVPN: RX advertised L2VPN EVPN and received
Extended nexthop: advertised and received
Address families by peer:
IPv4 Unicast
Route refresh: advertised and received(old & new)
Address Family IPv4 Unicast: advertised and received
Address Family L2VPN EVPN: advertised and received
Hostname Capability: advertised (name: spine01,domain name: n/a) received (name: leaf03,domain name: n/a)
Graceful Restart Capability: advertised and received
Remote Restart timer is 120 seconds
Address families by peer:
none
Graceful restart information:
End-of-RIB send: IPv4 Unicast, L2VPN EVPN
End-of-RIB received: IPv4 Unicast, L2VPN EVPN
Local GR Mode: Helper*
Remote GR Mode: Helper
R bit: False
Timers:
Configured Restart Time(sec): 120
Received Restart Time(sec): 120
IPv4 Unicast:
F bit: False
End-of-RIB sent: Yes
End-of-RIB sent after update: No
End-of-RIB received: Yes
Timers:
Configured Stale Path Time(sec): 360
Message statistics:
Inq depth is 0
Outq depth is 0
Sent Rcvd
Opens: 1 1
Notifications: 0 0
Updates: 270 247
Keepalives: 7727 7727
Route Refresh: 0 0
Capability: 0 0
Total: 7998 7975
Minimum time between advertisement runs is 0 seconds
For address family: IPv4 Unicast
underlay peer-group member
Update group 1, subgroup 1
Packet Queue length 0
Community attribute sent to this neighbor(all)
3 accepted prefixes
For address family: L2VPN EVPN
underlay peer-group member
Update group 2, subgroup 2
Packet Queue length 0
NEXT_HOP is propagated unchanged to this neighbor
Community attribute sent to this neighbor(all)
21 accepted prefixes
Connections established 1; dropped 0
Last reset 06:26:24, Waiting for peer OPEN
Local host: fe80::4638:39ff:fe00:11, Local port: 42258
Foreign host: fe80::4638:39ff:fe00:12, Foreign port: 179
Nexthop: 10.10.10.101
Nexthop global: fe80::4638:39ff:fe00:11
Nexthop local: fe80::4638:39ff:fe00:11
BGP connection: shared network
BGP Connect Retry Timer in Seconds: 10
Read thread: on Write thread: on FD used: 26
BGP neighbor on swp4: fe80::4638:39ff:fe00:1a, remote AS 65104, local AS 65100, external link
Hostname: leaf04
Member of peer-group underlay for session parameters
BGP version 4, remote router ID 10.10.10.4, local router ID 10.10.10.101
BGP state = Established, up for 06:26:23
Last read 00:00:01, Last write 00:00:02
Hold time is 9, keepalive interval is 3 seconds
Neighbor capabilities:
4 Byte AS: advertised and received
AddPath:
IPv4 Unicast: RX advertised IPv4 Unicast and received
L2VPN EVPN: RX advertised L2VPN EVPN and received
Extended nexthop: advertised and received
Address families by peer:
IPv4 Unicast
Route refresh: advertised and received(old & new)
Address Family IPv4 Unicast: advertised and received
Address Family L2VPN EVPN: advertised and received
Hostname Capability: advertised (name: spine01,domain name: n/a) received (name: leaf04,domain name: n/a)
Graceful Restart Capability: advertised and received
Remote Restart timer is 120 seconds
Address families by peer:
none
Graceful restart information:
End-of-RIB send: IPv4 Unicast, L2VPN EVPN
End-of-RIB received: IPv4 Unicast, L2VPN EVPN
Local GR Mode: Helper*
Remote GR Mode: Helper
R bit: False
Timers:
Configured Restart Time(sec): 120
Received Restart Time(sec): 120
IPv4 Unicast:
F bit: False
End-of-RIB sent: Yes
End-of-RIB sent after update: No
End-of-RIB received: Yes
Timers:
Configured Stale Path Time(sec): 360
Message statistics:
Inq depth is 0
Outq depth is 0
Sent Rcvd
Opens: 2 1
Notifications: 0 0
Updates: 270 236
Keepalives: 7727 7726
Route Refresh: 0 0
Capability: 0 0
Total: 7999 7963
Minimum time between advertisement runs is 0 seconds
For address family: IPv4 Unicast
underlay peer-group member
Update group 1, subgroup 1
Packet Queue length 0
Community attribute sent to this neighbor(all)
3 accepted prefixes
For address family: L2VPN EVPN
underlay peer-group member
Update group 2, subgroup 2
Packet Queue length 0
NEXT_HOP is propagated unchanged to this neighbor
Community attribute sent to this neighbor(all)
21 accepted prefixes
Connections established 1; dropped 0
Last reset 06:26:24, No AFI/SAFI activated for peer
Local host: fe80::4638:39ff:fe00:19, Local port: 179
Foreign host: fe80::4638:39ff:fe00:1a, Foreign port: 53020
Nexthop: 10.10.10.101
Nexthop global: fe80::4638:39ff:fe00:19
Nexthop local: fe80::4638:39ff:fe00:19
BGP connection: shared network
BGP Connect Retry Timer in Seconds: 10
Read thread: on Write thread: on FD used: 23
BGP neighbor on swp5: fe80::4638:39ff:fe00:22, remote AS 65163, local AS 65100, external link
Hostname: border01
Member of peer-group underlay for session parameters
BGP version 4, remote router ID 10.10.10.63, local router ID 10.10.10.101
BGP state = Established, up for 06:26:23
Last read 00:00:02, Last write 00:00:02
Hold time is 9, keepalive interval is 3 seconds
Neighbor capabilities:
4 Byte AS: advertised and received
AddPath:
IPv4 Unicast: RX advertised IPv4 Unicast and received
L2VPN EVPN: RX advertised L2VPN EVPN and received
Extended nexthop: advertised and received
Address families by peer:
IPv4 Unicast
Route refresh: advertised and received(old & new)
Address Family IPv4 Unicast: advertised and received
Address Family L2VPN EVPN: advertised and received
Hostname Capability: advertised (name: spine01,domain name: n/a) received (name: border01,domain name: n/a)
Graceful Restart Capability: advertised and received
Remote Restart timer is 120 seconds
Address families by peer:
none
Graceful restart information:
End-of-RIB send: IPv4 Unicast, L2VPN EVPN
End-of-RIB received: IPv4 Unicast, L2VPN EVPN
Local GR Mode: Helper*
Remote GR Mode: Helper
R bit: False
Timers:
Configured Restart Time(sec): 120
Received Restart Time(sec): 120
IPv4 Unicast:
F bit: False
End-of-RIB sent: Yes
End-of-RIB sent after update: No
End-of-RIB received: Yes
Timers:
Configured Stale Path Time(sec): 360
Message statistics:
Inq depth is 0
Outq depth is 0
Sent Rcvd
Opens: 1 1
Notifications: 0 0
Updates: 270 177
Keepalives: 7727 7727
Route Refresh: 0 0
Capability: 0 0
Total: 7998 7905
Minimum time between advertisement runs is 0 seconds
For address family: IPv4 Unicast
underlay peer-group member
Update group 1, subgroup 1
Packet Queue length 0
Community attribute sent to this neighbor(all)
3 accepted prefixes
For address family: L2VPN EVPN
underlay peer-group member
Update group 2, subgroup 2
Packet Queue length 0
NEXT_HOP is propagated unchanged to this neighbor
Community attribute sent to this neighbor(all)
3 accepted prefixes
Connections established 1; dropped 0
Last reset 06:26:24, Waiting for peer OPEN
Local host: fe80::4638:39ff:fe00:21, Local port: 33534
Foreign host: fe80::4638:39ff:fe00:22, Foreign port: 179
Nexthop: 10.10.10.101
Nexthop global: fe80::4638:39ff:fe00:21
Nexthop local: fe80::4638:39ff:fe00:21
BGP connection: shared network
BGP Connect Retry Timer in Seconds: 10
Read thread: on Write thread: on FD used: 28
BGP neighbor on swp6: fe80::4638:39ff:fe00:2a, remote AS 65164, local AS 65100, external link
Hostname: border02
Member of peer-group underlay for session parameters
BGP version 4, remote router ID 10.10.10.64, local router ID 10.10.10.101
BGP state = Established, up for 06:17:23
Last read 00:00:01, Last write 00:00:01
Hold time is 9, keepalive interval is 3 seconds
Neighbor capabilities:
4 Byte AS: advertised and received
AddPath:
IPv4 Unicast: RX advertised IPv4 Unicast and received
L2VPN EVPN: RX advertised L2VPN EVPN and received
Extended nexthop: advertised and received
Address families by peer:
IPv4 Unicast
Route refresh: advertised and received(old & new)
Address Family IPv4 Unicast: advertised and received
Address Family L2VPN EVPN: advertised and received
Hostname Capability: advertised (name: spine01,domain name: n/a) received (name: border02,domain name: n/a)
Graceful Restart Capability: advertised and received
Remote Restart timer is 120 seconds
Address families by peer:
none
Graceful restart information:
End-of-RIB send: IPv4 Unicast, L2VPN EVPN
End-of-RIB received: IPv4 Unicast, L2VPN EVPN
Local GR Mode: Helper*
Remote GR Mode: Helper
R bit: False
Timers:
Configured Restart Time(sec): 120
Received Restart Time(sec): 120
IPv4 Unicast:
F bit: False
End-of-RIB sent: Yes
End-of-RIB sent after update: Yes
End-of-RIB received: Yes
Timers:
Configured Stale Path Time(sec): 360
Message statistics:
Inq depth is 0
Outq depth is 0
Sent Rcvd
Opens: 2 2
Notifications: 2 0
Updates: 327 275
Keepalives: 7728 7725
Route Refresh: 0 0
Capability: 0 0
Total: 8059 8002
Minimum time between advertisement runs is 0 seconds
For address family: IPv4 Unicast
underlay peer-group member
Update group 1, subgroup 1
Packet Queue length 0
Community attribute sent to this neighbor(all)
3 accepted prefixes
For address family: L2VPN EVPN
underlay peer-group member
Update group 2, subgroup 2
Packet Queue length 0
NEXT_HOP is propagated unchanged to this neighbor
Community attribute sent to this neighbor(all)
3 accepted prefixes
Connections established 2; dropped 1
Last reset 06:17:24, Notification sent (Hold Timer Expired)
Local host: fe80::4638:39ff:fe00:29, Local port: 38956
Foreign host: fe80::4638:39ff:fe00:2a, Foreign port: 179
Nexthop: 10.10.10.101
Nexthop global: fe80::4638:39ff:fe00:29
Nexthop local: fe80::4638:39ff:fe00:29
BGP connection: shared network
BGP Connect Retry Timer in Seconds: 10
Read thread: on Write thread: on FD used: 25
cumulus@spine01:mgmt:~$ timed out waiting for input: auto-logout
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Welcome to
Login with: cumulus/CumulusLinux!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
spine01 login:
-
Le livre est librement téléchargeable sur le site de Nvidia, et aussi en pièce jointe.
Je recommande sa lecture.
https://resource.nvidia.com/en-us-bgp-datacenter
Le code de configuration sur github
https://github.com/oreillymedia/bgp_in_the_data_center
-
Connexion out-of-band
philippemarques@Philippes-MacBook-Pro ~ % ssh -p 12314 cumulus@worker01.air.nvidia.com
The authenticity of host '[worker01.air.nvidia.com]:12314 ([147.75.47.205]:12314)' can't be established.
ED25519 key fingerprint is SHA256:6FXartZ/3RcoO2Fx4z7OxeHzUk7HhX0F2HxIQe7Jlvc.
This host key is known by the following other names/addresses:
~/.ssh/known_hosts:32: [worker08.air.nvidia.com]:28366
~/.ssh/known_hosts:34: [worker08.air.nvidia.com]:22378
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '[worker01.air.nvidia.com]:12314' (ED25519) to the list of known hosts.
You are required to change your password immediately (root enforced)
Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-151-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Fri Jan 14 03:25:00 UTC 2022
System load: 0.0 Processes: 106
Usage of /: 29.2% of 9.29GB Users logged in: 0
Memory usage: 23% IP address for eth0: 169.254.0.2
Swap usage: 0% IP address for eth1: 192.168.200.1
* Super-optimized for small spaces - read how we shrank the memory
footprint of MicroK8s to make it the smallest full K8s around.
https://ubuntu.com/blog/microk8s-memory-optimisation
25 updates can be applied immediately.
16 of these updates are standard security updates.
To see these additional updates run: apt list --upgradable
Last login: Thu Sep 2 15:52:09 2021
WARNING: Your password has expired.
You must change your password now and login again!
Changing password for cumulus.
(current) UNIX password:
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
Connection to worker01.air.nvidia.com closed.
philippemarques@Philippes-MacBook-Pro ~ % ssh -p 12314 cumulus@worker01.air.nvidia.com
Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-151-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Fri Jan 14 03:25:38 UTC 2022
System load: 0.08 Processes: 108
Usage of /: 29.2% of 9.29GB Users logged in: 0
Memory usage: 23% IP address for eth0: 169.254.0.2
Swap usage: 0% IP address for eth1: 192.168.200.1
* Super-optimized for small spaces - read how we shrank the memory
footprint of MicroK8s to make it the smallest full K8s around.
https://ubuntu.com/blog/microk8s-memory-optimisation
25 updates can be applied immediately.
16 of these updates are standard security updates.
To see these additional updates run: apt list --upgradable
New release '20.04.3 LTS' available.
Run 'do-release-upgrade' to upgrade to it.
Last login: Fri Jan 14 03:25:00 2022 from fd01:1:1:22b4::1
cumulus@oob-mgmt-server:~$
-
Parce qu'il faut bien s'amuser un peu.
Ubuntu 18.04.5 LTS server01 ttyS0
server01 login: cumulus
Password:
Last login: Wed Aug 25 09:54:05 PDT 2021 from 192.168.200.1 on pts/0
Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-140-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Thu Jan 13 19:33:37 PST 2022
System load: 0.0 Processes: 88
Usage of /: 21.0% of 9.29GB Users logged in: 0
Memory usage: 38% IP address for eth0: 192.168.200.31
Swap usage: 0% IP address for uplink: 10.1.10.101
89 packages can be updated.
59 of these updates are security updates.
To see these additional updates run: apt list --upgradable
cumulus@server01:~$ sudo passwd root
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
cumulus@server01:~$ su root
Password:
root@server01:/home/cumulus# cd /etc/
root@server01:/etc# ls
NetworkManager hosts.allow polkit-1
X11 hosts.deny pollinate
acpi init.d popularity-contest.conf
adduser.conf initramfs-tools profile
alternatives inputrc profile.d
apm iproute2 protocols
apparmor iscsi python
apparmor.d issue python2.7
apport issue.net python3
apt kernel python3.6
at.deny kernel-img.conf rc0.d
bash.bashrc landscape rc1.d
bash_completion ld.so.cache rc2.d
bash_completion.d ld.so.conf rc3.d
bindresvport.blacklist ld.so.conf.d rc4.d
binfmt.d ldap rc5.d
byobu legal rc6.d
ca-certificates libaudit.conf rcS.d
ca-certificates.conf libpaper.d resolv.conf
ca-certificates.conf.dpkg-old lintianrc rmt
calendar lldpd.d rpc
cloud locale.alias rsyslog.conf
console-setup locale.gen rsyslog.d
cowpoke.conf localtime screenrc
cron.d logcheck securetty
cron.daily login.defs security
cron.hourly logrotate.conf selinux
cron.monthly logrotate.d sensors.d
cron.weekly lsb-release sensors3.conf
crontab ltrace.conf services
cryptsetup-initramfs lvm sgml
crypttab machine-id shadow
cumulus-air magic shadow-
dbus-1 magic.mime shells
debconf.conf mailcap skel
debian_version mailcap.order sos.conf
default manpath.config ssh
deluser.conf mdadm ssl
depmod.d mime.types subgid
devscripts.conf mke2fs.conf subgid-
dhcp modprobe.d subuid
dnsmasq.d modules subuid-
dnsmasq.d-available modules-load.d sudoers
dpkg mtab sudoers.d
dput.cf nanorc sysctl.conf
ec2_version netplan sysctl.d
emacs netq systemd
environment network terminfo
ethertypes networkd-dispatcher timezone
fonts networks tmpfiles.d
fstab newt ucf.conf
fuse.conf nsswitch.conf udev
gai.conf ntp.conf ufw
groff opt update-manager
group os-release update-motd.d
group- overlayroot.conf update-notifier
grub.d overlayroot.local.conf updatedb.conf
gshadow pam.conf vim
gshadow- pam.d vmware-tools
gss papersize vtrgb
hdparm.conf passwd wgetrc
host.conf passwd- xdg
hostname perl xml
hosts pm zsh_command_not_found
root@server01:/etc# cat shadow
daemon:*:18719:0:99999:7:::
bin:*:18719:0:99999:7:::
sys:*:18719:0:99999:7:::
sync:*:18719:0:99999:7:::
games:*:18719:0:99999:7:::
man:*:18719:0:99999:7:::
lp:*:18719:0:99999:7:::
mail:*:18719:0:99999:7:::
news:*:18719:0:99999:7:::
uucp:*:18719:0:99999:7:::
proxy:*:18719:0:99999:7:::
www-data:*:18719:0:99999:7:::
backup:*:18719:0:99999:7:::
list:*:18719:0:99999:7:::
irc:*:18719:0:99999:7:::
gnats:*:18719:0:99999:7:::
nobody:*:18719:0:99999:7:::
systemd-network:*:18719:0:99999:7:::
systemd-resolve:*:18719:0:99999:7:::
syslog:*:18719:0:99999:7:::
messagebus:*:18719:0:99999:7:::
_apt:*:18719:0:99999:7:::
lxd:*:18719:0:99999:7:::
uuidd:*:18719:0:99999:7:::
dnsmasq:*:18719:0:99999:7:::
landscape:*:18719:0:99999:7:::
sshd:*:18719:0:99999:7:::
pollinate:*:18719:0:99999:7:::
ubuntu:!:18724:0:99999:7:::
cumulus:$1$8WOBD1pA$uph5ks6BG29F12dt/ORmE0:18724:0:99999:7:::
_lldpd:!:18724:0:99999:7:::
ntp:*:18864:0:99999:7:::
root@server01:/etc#
-
Parce que le but du jeu, c'est quand même de jouer avec les VxLAN et EVPN
border02 login: cumulus
Password:
Last login: Wed Aug 25 09:52:53 PDT 2021 from 192.168.200.1 on pts/0
Linux border02 4.19.0-cl-1-amd64 #1 SMP Debian 4.19.176-1+cl4.4.0u1 (2021-06-25) x86_64
#########################################################
Successfully logged in to: border02
#########################################################
cumulus@border02:mgmt:~$ net show
bfd : Bidirectional forwarding detection
bgp : Border Gateway Protocol
bridge : a layer2 bridge
clag : Multi-Chassis Link Aggregation
commit : apply the commit buffer to the system
configuration : settings, configuration state, etc
counters : net show counters
debugs : Debugs
dhcp-snoop : DHCP snooping for IPv4
dhcp-snoop6 : DHCP snooping for IPv6
dot1x : Configure, Enable, Delete or Show IEEE 802.1X EAPOL
evpn : Ethernet VPN
hostname : local hostname
igmp : Internet Group Management Protocol
interface : An interface, such as swp1, swp2, etc.
ip : Internet Protocol version 4/6
ipv6 : Internet Protocol version 6
lldp : Link Layer Discovery Protocol
mpls : Multiprotocol Label Switching
mroute : Static unicast routes in MRIB for multicast RPF lookup
msdp : Multicast Source Discovery Protocol
neighbor : A BGP, OSPF, PIM, etc neighbor
ospf : Open Shortest Path First (OSPFv2)
ospf6 : Open Shortest Path First (OSPFv3)
package : A Cumulus Linux package name
pbr : Policy Based Routing
pim : Protocol Independent Multicast
port-mirror : port-mirror
port-security : Port security
ptp : Precision Time Protocol
roce : Enable RoCE on all interfaces, default mode is lossless
rollback : revert to a previous configuration state
route : EVPN route information
route-map : Route-map
snmp-server : Configure the SNMP server
system : System
time : Time
version : Version number
vrf : Virtual routing and forwarding
vrrp : Virtual Router Redundancy Protocol
cumulus@border02:mgmt:~$ net show evpn
access-vlan : access VLANs
arp-cache : ARP and ND cache
es : Ethernet Segment (Type-4) route
es-evi : ES per EVI
json : Print output in json
l2-nh : Layer 2 nexthops
mac : Media Access Control
next-hops : Next-hops
rmac : Router Mac
vni : VXLAN Network Identifier
<ENTER>
cumulus@border02:mgmt:~$ net show evpn vni
VNI Type VxLAN IF # MACs # ARPs # Remote VTEPs Tenant VRF
102 L2 vni102 2 2 0 BLUE
101 L2 vni101 2 1 0 RED
4001 L3 vniRED 2 4 n/a RED
4002 L3 vniBLUE 2 4 n/a BLUE
cumulus@border02:mgmt:~$
-
Ce qu'il se passe du côté du Firewall et de l'aide à la configuration des VTEP
oot@fw1:mgmt:~# vtep-ctl --help
vtep-ctl: VTEP configuration utility
usage: vtep-ctl [OPTIONS] COMMAND [ARG...]
VTEP commands:
show print overview of database contents
Manager commands:
get-manager print the managers
del-manager delete the managers
set-manager TARGET... set the list of managers to TARGET...
Physical Switch commands:
add-ps PS create a new physical switch named PS
del-ps PS delete PS and all of its ports
list-ps print the names of all the physical switches
ps-exists PS exit 2 if PS does not exist
Port commands:
list-ports PS print the names of all the ports on PS
add-port PS PORT add network device PORT to PS
del-port PS PORT delete PORT from PS
Logical Switch commands:
add-ls LS create a new logical switch named LS
del-ls LS delete LS and all of its ports
list-ls print the names of all the logical switches
ls-exists LS exit 2 if LS does not exist
bind-ls PS PORT VLAN LS bind LS to VLAN on PORT
unbind-ls PS PORT VLAN unbind logical switch on VLAN from PORT
list-bindings PS PORT list bindings for PORT on PS
set-replication-mode LS MODE set replication mode on LS
get-replication-mode LS get replication mode on LS
Logical Router commands:
add-lr LR create a new logical router named LR
del-lr LR delete LR
list-lr print the names of all the logical routers
lr-exists LR exit 2 if LR does not exist
MAC binding commands:
add-ucast-local LS MAC [ENCAP] IP add ucast local entry in LS
del-ucast-local LS MAC del ucast local entry from LS
add-mcast-local LS MAC [ENCAP] IP add mcast local entry in LS
del-mcast-local LS MAC [ENCAP] IP del mcast local entry from LS
clear-local-macs LS clear local mac entries
list-local-macs LS list local mac entries
add-ucast-remote LS MAC [ENCAP] IP add ucast remote entry in LS
del-ucast-remote LS MAC del ucast remote entry from LS
add-mcast-remote LS MAC [ENCAP] IP add mcast remote entry in LS
del-mcast-remote LS MAC [ENCAP] IP del mcast remote entry from LS
clear-remote-macs LS clear remote mac entries
list-remote-macs LS list remote mac entries
Database commands:
list TBL [REC] list RECord (or all records) in TBL
find TBL CONDITION... list records satisfying CONDITION in TBL
get TBL REC COL[:KEY] print values of COLumns in RECord in TBL
set TBL REC COL[:KEY]=VALUE set COLumn values in RECord in TBL
add TBL REC COL [KEY=]VALUE add (KEY=)VALUE to COLumn in RECord in TBL
remove TBL REC COL [KEY=]VALUE remove (KEY=)VALUE from COLumn
clear TBL REC COL clear values from COLumn in RECord in TBL
create TBL COL[:KEY]=VALUE create and initialize new record
destroy TBL REC delete RECord from TBL
wait-until TBL REC [COL[:KEY]=VALUE] wait until condition is true
Potentially unsafe database commands require --force option.
Options:
--db=DATABASE connect to DATABASE
(default: unix:/var/run/openvswitch/db.sock)
-t, --timeout=SECS wait at most SECS seconds
--dry-run do not commit changes to database
--oneline print exactly one line of output per command
Output formatting options:
-f, --format=FORMAT set output formatting to FORMAT
("table", "html", "csv", or "json")
-d, --data=FORMAT set table cell output formatting to
FORMAT ("string", "bare", or "json")
--no-headings omit table heading row
--pretty pretty-print JSON in output
--bare equivalent to "--format=list --data=bare --no-headings"
Logging options:
-vSPEC, --verbose=SPEC set logging levels
-v, --verbose set maximum verbosity level
--log-file[=FILE] enable logging to specified FILE
(default: /var/log/openvswitch/vtep-ctl.log)
--syslog-method=(libc|unix:file|udp:ip:port)
specify how to send messages to syslog daemon
--syslog-target=HOST:PORT also send syslog msgs to HOST:PORT via UDP
--no-syslog equivalent to --verbose=vtep_ctl:syslog:warn
Active database connection methods:
tcp:IP:PORT PORT at remote IP
ssl:IP:PORT SSL PORT at remote IP
unix:FILE Unix domain socket named FILE
Passive database connection methods:
ptcp:PORT[:IP] listen to TCP PORT on IP
pssl:PORT[:IP] listen for SSL on PORT on IP
punix:FILE listen on Unix domain socket FILE
PKI configuration (required to use SSL):
-p, --private-key=FILE file with private key
-c, --certificate=FILE file with certificate for private key
-C, --ca-cert=FILE file with peer CA certificate
SSL options:
--ssl-protocols=PROTOS list of SSL protocols to enable
--ssl-ciphers=CIPHERS list of SSL ciphers to enable
Other options:
-h, --help display this help message
-V, --version display version information
root@fw1:mgmt:~# vtysh
Hello, this is FRRouting (version 7.5+cl4.4.0u4).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
fw1# sh
sharp show
fw1# show r
route-map router-id running-config
route-map-unused rpki
fw1# show running-config
Building configuration...
Current configuration:
!
frr version 7.5+cl4.4.0u4
frr defaults datacenter
hostname fw1
log syslog informational
zebra nexthop proto only
service integrated-vtysh-config
!
ip route 10.1.10.0/24 10.1.101.1
ip route 10.1.20.0/24 10.1.101.1
ip route 10.1.30.0/24 10.1.102.1
!
vrf mgmt
ip route 0.0.0.0/0 192.168.200.1
exit-vrf
!
line vty
!
end
fw1# iptables -L
% Unknown command: iptables -L
fw1# exit
root@fw1:mgmt:~# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
DROP all -- 240.0.0.0/5 anywhere
DROP all -- 127.0.0.0/8 anywhere
DROP all -- base-address.mcast.net/4 anywhere
DROP all -- 255.255.255.255 anywhere
SETCLASS udp -- anywhere anywhere udp dpt:3785 SETCLASS class:7
POLICE udp -- anywhere anywhere udp dpt:3785 POLICE mode:pkt rate:2000 burst:2000
SETCLASS udp -- anywhere anywhere udp dpt:3784 SETCLASS class:7
POLICE udp -- anywhere anywhere udp dpt:3784 POLICE mode:pkt rate:2000 burst:2000
SETCLASS udp -- anywhere anywhere udp dpt:4784 SETCLASS class:7
POLICE udp -- anywhere anywhere udp dpt:4784 POLICE mode:pkt rate:2000 burst:2000
SETCLASS ospf -- anywhere anywhere SETCLASS class:7
POLICE ospf -- anywhere anywhere POLICE mode:pkt rate:2000 burst:2000
SETCLASS pim -- anywhere anywhere SETCLASS class:6
POLICE pim -- anywhere anywhere POLICE mode:pkt rate:2000 burst:2000
SETCLASS tcp -- anywhere anywhere tcp dpt:639 SETCLASS class:6
POLICE tcp -- anywhere anywhere tcp dpt:639 POLICE mode:pkt rate:2000 burst:2000
SETCLASS tcp -- anywhere anywhere tcp spt:639 SETCLASS class:6
POLICE tcp -- anywhere anywhere tcp spt:639 POLICE mode:pkt rate:2000 burst:2000
SETCLASS tcp -- anywhere anywhere tcp dpt:bgp SETCLASS class:7
POLICE tcp -- anywhere anywhere tcp dpt:bgp POLICE mode:pkt rate:2000 burst:2000
SETCLASS tcp -- anywhere anywhere tcp spt:bgp SETCLASS class:7
POLICE tcp -- anywhere anywhere tcp spt:bgp POLICE mode:pkt rate:2000 burst:2000
SETCLASS tcp -- anywhere anywhere tcp dpt:5342 SETCLASS class:7
POLICE tcp -- anywhere anywhere tcp dpt:5342 POLICE mode:pkt rate:2000 burst:2000
SETCLASS tcp -- anywhere anywhere tcp spt:5342 SETCLASS class:7
POLICE tcp -- anywhere anywhere tcp spt:5342 POLICE mode:pkt rate:2000 burst:2000
SETCLASS icmp -- anywhere anywhere SETCLASS class:2
POLICE icmp -- anywhere anywhere POLICE mode:pkt rate:100 burst:40
SETCLASS udp -- anywhere anywhere udp dpts:bootps:bootpc SETCLASS class:2
POLICE udp -- anywhere anywhere udp dpt:bootps POLICE mode:pkt rate:100 burst:100
POLICE udp -- anywhere anywhere udp dpt:bootpc POLICE mode:pkt rate:100 burst:100
SETCLASS tcp -- anywhere anywhere tcp dpts:67:68 SETCLASS class:2
POLICE tcp -- anywhere anywhere tcp dpt:67 POLICE mode:pkt rate:100 burst:100
POLICE tcp -- anywhere anywhere tcp dpt:68 POLICE mode:pkt rate:100 burst:100
SETCLASS igmp -- anywhere anywhere SETCLASS class:6
POLICE igmp -- anywhere anywhere POLICE mode:pkt rate:300 burst:100
SETCLASS vrrp -- anywhere anywhere SETCLASS class:7
POLICE vrrp -- anywhere anywhere POLICE mode:pkt rate:2000 burst:2000
POLICE all -- anywhere anywhere ADDRTYPE match dst-type LOCAL POLICE mode:pkt rate:1000 burst:1000 class:2
POLICE all -- anywhere anywhere ADDRTYPE match dst-type IPROUTER POLICE mode:pkt rate:400 burst:100 class:2
SETCLASS all -- anywhere anywhere SETCLASS class:0
Chain FORWARD (policy ACCEPT)
target prot opt source destination
DROP all -- 240.0.0.0/5 anywhere
DROP all -- 127.0.0.0/8 anywhere
DROP all -- base-address.mcast.net/4 anywhere
DROP all -- 255.255.255.255 anywhere
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
root@fw1:mgmt:~#
-
Ce qu'il se passe du côté des routeurs de bordure
order01 login: cumulus
Password:
Last login: Wed Aug 25 09:52:53 PDT 2021 from 192.168.200.1 on pts/0
Linux border01 4.19.0-cl-1-amd64 #1 SMP Debian 4.19.176-1+cl4.4.0u1 (2021-06-25) x86_64
#########################################################
Successfully logged in to: border01
#########################################################
cumulus@border01:mgmt:~$ vtysh
% Can't open configuration file /etc/frr/vtysh.conf due to 'Permission denied'.
Exiting: failed to connect to any daemons.
Hint: if this seems wrong, try running me as a privileged user!
cumulus@border01:mgmt:~$ sudo passwd root
New password:
Retype new password:
passwd: password updated successfully
cumulus@border01:mgmt:~$ su root
Password:
root@border01:mgmt:/home/cumulus# vtysh
Hello, this is FRRouting (version 7.5+cl4.4.0u4).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
border01# show running-config
Building configuration...
Current configuration:
!
frr version 7.5+cl4.4.0u4
frr defaults datacenter
hostname border01
log syslog informational
zebra nexthop proto only
ip msdp mesh-group rpmesh source 10.10.10.63
ip msdp mesh-group rpmesh member 10.10.10.64
ip pim rp 10.10.100.100 239.1.1.0/24
ip pim keep-alive-timer 3600
ip pim ecmp
service integrated-vtysh-config
!
vrf BLUE
ip route 10.1.10.0/24 10.1.102.4
ip route 10.1.20.0/24 10.1.102.4
vni 4002
exit-vrf
!
vrf RED
ip route 10.1.30.0/24 10.1.101.4
vni 4001
exit-vrf
!
vrf mgmt
ip route 0.0.0.0/0 192.168.200.1
exit-vrf
!
interface bond1
evpn mh es-df-pref 50000
evpn mh es-id 1
evpn mh es-sys-mac 44:38:39:be:ef:ff
!
interface swp51
evpn mh uplink
ip pim
!
interface swp52
evpn mh uplink
ip pim
!
interface swp53
evpn mh uplink
ip pim
!
interface swp54
evpn mh uplink
ip pim
!
interface lo
ip igmp
ip pim
ip pim use-source 10.10.10.63
!
router bgp 65163
bgp router-id 10.10.10.63
neighbor underlay peer-group
neighbor underlay remote-as external
neighbor swp51 interface peer-group underlay
neighbor swp52 interface peer-group underlay
neighbor swp53 interface peer-group underlay
neighbor swp54 interface peer-group underlay
!
address-family ipv4 unicast
redistribute connected
exit-address-family
!
address-family l2vpn evpn
neighbor underlay activate
advertise-all-vni
exit-address-family
!
router bgp 65163 vrf RED
bgp router-id 10.10.10.63
!
address-family ipv4 unicast
redistribute static
exit-address-family
!
address-family l2vpn evpn
advertise ipv4 unicast
exit-address-family
!
router bgp 65163 vrf BLUE
bgp router-id 10.10.10.63
!
address-family ipv4 unicast
redistribute static
exit-address-family
!
address-family l2vpn evpn
advertise ipv4 unicast
exit-address-family
!
line vty
!
end
border01#
-
Si j'ai bien compris, c'est une démo pour leur solution "NVIDIA CUMULUS Linux".
(Je ne savais même pas que NVIDIA faisait du software).
Je n'ai pas tout compris ce que c'était, ce "NVIDIA CUMULUS". Tu peux nous expliquer? Ca sert à quoi? C'est un "OS réseau", mais je ne sais pas ce que c'est un "OS réseau".
Leon.
-
C'est une sorte de GNS3 sur le cloud, une démo de cumulus Linux.
Manifestement c'est un axe de développement pour Nvidia, ayant racheté Mellanox ( et leur Hardware bien connu pour les devices réseaux).
Nvidia connu pour ses chips GPU , se lance dans les devices réseaux ,effectivement c'est un démonstration pour le produit Cumulus Linux.
Basé sur Debian, cela intègre des briques Ansible, FRR etc.
Mais c'est comme le montre l'image de l'architecture, une architecture Datacenter simulée, à base de Leaf, Spine, de VxLAN, EVPN, VTEP.
A ma connaissance, il n'y a pas beaucoup de produits gratuits permettant d'implémenter ce type de solutions sur le marché.
Tu peux aller chez Cisco, Juniper, Arista etc. tu y laissera un bras.
Tous les devices, peuvent être implémentés en Hardware, les configurations seront identiques.
Une réflexion que je me suis faite, et que l'approfondirais à l'occasion, c'est cette structure Leaf, Spine, en commutation et Routing.
Cela ne vous fait pas penser à une Matrice 2x2 du plan ?
Je m'explique, si on un prends un cristal adjacent, avec chaque atome dans une structure cristalline adjacente, et que l'on dit que chaque noeuds est une composante de ce cristal, il y a un moyen de construire une structure hautement scalable. Tous ces éléments n'ont pas besoin d'être physiquement géographiquement proches, mais cela permettrait de construire un maillage en plusieurs dimensions. Imaginez une structure en 3 dimensions, puis N dimensions. Si personne ne comprends ce que je raconte, mais la structure que j'évoque est hautement robuste, résiliante et scalable pour un "core Network".
-
C'est Wikipedia qui décrit le mieux Cumulus-Linux pour les ignares (dont je fais partie). Parce que les pubs de NVIDIA remplies de buzz-words, ça n'explique rien.
https://en.wikipedia.org/wiki/Cumulus_Networks#Cumulus_Linux
Cumulus Linux
Cumulus Linux was their open Linux based networking operating system for bare metal switches. It's been based on the Debian Linux distribution.[13]
In a 2017 Gartner report Cumulus Networks was highlighted as a pioneer of open source networking for developing an open source networking operating system in a market where hardware vendors usually delivered proprietary operating systems pre-installed. According to Gartner, Cumulus Networks had worked around the lack of vendor support for open source networking by deploying bare metal switches with the Cumulus Linux operating system in large corporate networks. 32 percent of the Fortune 50 companies used the Cumulus Linux operating system in their data centers in 2017.[14]
OK, donc c'est un OS open source pour switches et routeurs, créé par une société "Cumulus Networks" qui a été récemment rachetée par NVIDIA, et c'est complémentaire des produits Mellanox que NVIDIA a également acheté.
Leon.
-
Tu es moins ignare que tu veux bien le laisser croire.
Et au delà de tes considérations, c'est l'aspect architectural et structurel, l'aspect le plus intéressant, du moins c'est le propos de ce post.
[edit]
D'ailleurs je viens de remarquer que j'en suis à 633 posts, il m'en reste 33, et à 666 j'arrêterai. ;D
[/edit]