RancherOS Cluster: Unterschied zwischen den Versionen

Aus C3D2
Wechseln zu: Navigation, Suche
(SmartOS - KVM - RancherOS Installation)
 
(43 dazwischenliegende Versionen von 2 Benutzern werden nicht angezeigt)
Zeile 1: Zeile 1:
 +
{{Vorlage:anachronistisch}}
 +
 
== Ankündigung ==
 
== Ankündigung ==
  
Zeile 27: Zeile 29:
 
* 2 x Dell R510?
 
* 2 x Dell R510?
  
== Umsetzung: Sun May 19 ==
+
== Host ==
 +
 
 +
1HE Bladeserver: [[Supermicro 808-12#Board rechte Seite]] ([[Server/server7]])
 +
 
 +
=== Host Einstellungen BIOS ===
 +
 
 +
== Umsetzung ==
  
=== Wahl des Servers ===
+
; Datum: 2019-05-19
  
* '''Server7'''
+
== Betriebssytem ==
  
=== BIOS Einstellungen ===
+
=== Installation Betriebssystem ===
  
* '''Hyperthreading''' aus (weil broken by design)
+
[[SmartOS]]
* '''Aggressive Link Power Management''' aus
 
* '''NUMA''' an
 
* boot nur vom USB-Stick
 
  
=== SmartOS Installation ===
+
==== Installation Betriebssystem Vorbereitung ====
  
==== SmartOS Bootvorgang ====
+
Starten vom Massenspeicher USB-Stick
  
[[Datei:C3d2_smartos_2.jpg]]
+
Bootvorgang
 +
[[Datei:c3d2 smartos 2.jpg]]
  
==== SmartOS Installationsablauf ====
+
==== Installation Betriebssystem Durchführung ====
  
[[Datei:C3d2_smartos_install_1.jpg]]
+
[[Datei:c3d2 smartos install 1.jpg]]
  
[[Datei:C3d2_smartos_install_2.jpg]]
+
[[Datei:c3d2 smartos install 2.jpg]]
  
[[Datei:C3d2_smartos_install_3.jpg]]
+
[[Datei:c3d2 smartos install 3.jpg]]
  
[[Datei:C3d2_smartos_install_4.jpg]]
+
[[Datei:c3d2 smartos install 4.jpg]]
  
[[Datei:C3d2_smartos_install_5.jpg]]
+
[[Datei:c3d2 smartos install 5.jpg]]
  
[[Datei:C3d2_smartos_install_6.jpg]]
+
[[Datei:c3d2 smartos install 6.jpg]]
  
[[Datei:C3d2_smartos_install_7.jpg]]
+
[[Datei:c3d2 smartos install 7.jpg]]
  
==== SmartOS Nachoptimierungen ====
+
==== Installation Betriebssystem Nachbereitung ====
  
 +
: <code>zfs list</code>
 
<source>
 
<source>
[root@server7smarti1 ~]# zfs list
 
 
NAME                USED  AVAIL  REFER  MOUNTPOINT
 
NAME                USED  AVAIL  REFER  MOUNTPOINT
 
zones                137G  723G  784K  /zones
 
zones                137G  723G  784K  /zones
Zeile 77: Zeile 83:
 
zones/usbkey        120K  723G  120K  legacy
 
zones/usbkey        120K  723G  120K  legacy
 
zones/var          1,24M  723G  1,24M  legacy
 
zones/var          1,24M  723G  1,24M  legacy
[root@server7smarti1 ~]#
+
</source>
[root@server7smarti1 ~]# zfs create zones/c3d2.de
+
 
[root@server7smarti1 ~]# zfs create zones/c3d2.de/admin
+
: <code>zfs create zones/c3d2.de</code>
[root@server7smarti1 ~]# zfs create zones/c3d2.de/iso
+
: <code>zfs create zones/c3d2.de/admin</code>
[root@server7smarti1 ~]#
+
: <code>zfs create zones/c3d2.de/iso</code>
[root@server7smarti1 ~]# zfs list -o name | egrep -v "NAME" | xargs -L 1 -I % zfs set checksum=on %
+
 
[root@server7smarti1 ~]# zfs set checksum=noparity zones/dump
+
 
[root@server7smarti1 ~]# zfs list -o name | egrep -v "NAME" | xargs -L 1 -I % zfs set compression=lz4 %
+
: <code>zfs list -o name | egrep -v "NAME" | xargs -L 1 -I % zfs set checksum=on %</code>
[root@server7smarti1 ~]# zfs set compression=off zones/dump
+
: <code>zfs set checksum=noparity zones/dump</code>
[root@server7smarti1 ~]# zfs set compression=off zones/swap
+
: <code>zfs list -o name | egrep -v "NAME" | xargs -L 1 -I % zfs set compression=lz4 %</code>
[root@server7smarti1 ~]# zfs list -o name | egrep -v "NAME" | xargs -L 1 -I % zfs set dedup=off %
+
: <code>zfs set compression=off zones/dump</code>
[root@server7smarti1 ~]# zfs list -o name | egrep -v "NAME" | xargs -L 1 -I % zfs set atime=off %
+
: <code>zfs set compression=off zones/swap</code>
 +
: <code>zfs list -o name | egrep -v "NAME" | xargs -L 1 -I % zfs set dedup=off %</code>
 +
: <code>zfs list -o name | egrep -v "NAME" | xargs -L 1 -I % zfs set atime=off %</code>
 +
<source>
 
cannot set property for 'zones/dump': 'atime' does not apply to datasets of this type
 
cannot set property for 'zones/dump': 'atime' does not apply to datasets of this type
 
cannot set property for 'zones/swap': 'atime' does not apply to datasets of this type
 
cannot set property for 'zones/swap': 'atime' does not apply to datasets of this type
[root@server7smarti1 ~]# zfs list -o name | egrep -v "NAME" | xargs -L 1 -I % zfs set primarycache=all %
 
[root@server7smarti1 ~]# zfs set primarycache=metadata zones/swap
 
[root@server7smarti1 ~]#
 
[root@server7smarti1 ~]# cd /zones/c3d2.de/admin
 
 
</source>
 
</source>
 +
: <code>zfs list -o name | egrep -v "NAME" | xargs -L 1 -I % zfs set primarycache=all %</code>
 +
: <code>zfs set primarycache=metadata zones/swap</code>
 +
<!--
 +
: <code>cd /zones/c3d2.de/admin</code>
 +
!-->
  
 
===== Default System Config mit Germany Keymap =====
 
===== Default System Config mit Germany Keymap =====
  
 
<source>
 
<source>
╭─daniel at it-daniel in ~ using
+
: <code>ssh root@172.22.99.245</code>
╰─○ ssh root@172.22.99.245
+
<source>
 
Password:
 
Password:
 
- SmartOS (build: 20190510T131809Z)
 
- SmartOS (build: 20190510T131809Z)
[root@server7smarti1 ~]# cat /usbkey/config
+
</source>
 +
: <code>cat /usbkey/config</code>
 +
<source>
 
#
 
#
 
# This file was auto-generated and must be source-able by bash.
 
# This file was auto-generated and must be source-able by bash.
Zeile 130: Zeile 142:
 
### ### ### // C3D2 ### ### ###
 
### ### ### // C3D2 ### ### ###
 
# EOF
 
# EOF
[root@server7smarti1 ~]# sync; reboot
+
</source>
 +
 
 +
: <code>sync; reboot</code>
 +
<source>
 
Connection to 172.22.99.245 closed by remote host.
 
Connection to 172.22.99.245 closed by remote host.
 
Connection to 172.22.99.245 closed.
 
Connection to 172.22.99.245 closed.
╭─daniel at it-daniel in ~ using
 
╰─○
 
 
</source>
 
</source>
  
 
==== SmartOS Backup Script ====
 
==== SmartOS Backup Script ====
  
 +
: <code>cd /zones/c3d2.de/admin</code>
 +
: <code>cd /zones/c3d2.de/adminwget --no-check-certificate https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.conf</code>
 
<source>
 
<source>
[root@server7smarti1 /zones/c3d2.de/admin]# wget --no-check-certificate https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.conf
 
 
--2019-05-19 09:02:01--  https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.conf
 
--2019-05-19 09:02:01--  https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.conf
 
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.12.133
 
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.12.133
Zeile 153: Zeile 167:
  
 
2019-05-19 09:02:01 (8,35 MB/s) - ‘smartos-zone-backup.conf’ saved [160/160]
 
2019-05-19 09:02:01 (8,35 MB/s) - ‘smartos-zone-backup.conf’ saved [160/160]
 +
</source>
  
[root@server7smarti1 /zones/c3d2.de/admin]# wget --no-check-certificate https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.exclude
+
: <code>wget --no-check-certificate https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.exclude</code>
 +
<source>
 
--2019-05-19 09:02:08--  https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.exclude
 
--2019-05-19 09:02:08--  https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.exclude
 
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.12.133
 
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.12.133
Zeile 167: Zeile 183:
  
 
2019-05-19 09:02:08 (0,00 B/s) - ‘smartos-zone-backup.exclude’ saved [0/0]
 
2019-05-19 09:02:08 (0,00 B/s) - ‘smartos-zone-backup.exclude’ saved [0/0]
 +
</source>
  
[root@server7smarti1 /zones/c3d2.de/admin]# wget --no-check-certificate https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.include
+
: <code>wget --no-check-certificate https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.include</code>
 +
<source>
 
--2019-05-19 09:02:12--  https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.include
 
--2019-05-19 09:02:12--  https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.include
 
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.12.133
 
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.12.133
Zeile 181: Zeile 199:
  
 
2019-05-19 09:02:13 (0,00 B/s) - ‘smartos-zone-backup.include’ saved [0/0]
 
2019-05-19 09:02:13 (0,00 B/s) - ‘smartos-zone-backup.include’ saved [0/0]
 +
</source>
  
[root@server7smarti1 /zones/c3d2.de/admin]# wget --no-check-certificate https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.log
+
: <code>wget --no-check-certificate https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.log</code>
 +
<source>
 
--2019-05-19 09:02:20--  https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.log
 
--2019-05-19 09:02:20--  https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.log
 
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.112.133
 
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.112.133
Zeile 195: Zeile 215:
  
 
2019-05-19 09:02:20 (0,00 B/s) - ‘smartos-zone-backup.log’ saved [0/0]
 
2019-05-19 09:02:20 (0,00 B/s) - ‘smartos-zone-backup.log’ saved [0/0]
 +
</source>
  
[root@server7smarti1 /zones/c3d2.de/admin]# wget --no-check-certificate https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.sh
+
: <code>wget --no-check-certificate https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.sh</code>
 +
<source>
 
--2019-05-19 09:02:22--  https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.sh
 
--2019-05-19 09:02:22--  https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.sh
 
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.112.133
 
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.112.133
Zeile 209: Zeile 231:
  
 
2019-05-19 09:02:24 (1,15 MB/s) - ‘smartos-zone-backup.sh’ saved [11392/11392]
 
2019-05-19 09:02:24 (1,15 MB/s) - ‘smartos-zone-backup.sh’ saved [11392/11392]
 +
</source>
  
[root@server7smarti1 /zones/c3d2.de/admin]#
+
: <code>chmod 0755 smartos-zone-backup.sh</code>
[root@server7smarti1 /zones/c3d2.de/admin]# chmod 0755 smartos-zone-backup.sh
+
: <code>ls -all</code>
[root@server7smarti1 /zones/c3d2.de/admin]# ls -all
+
<source>
 
total 47
 
total 47
 
drwxr-xr-x  2 root    root          7 Mai 19 09:02 .
 
drwxr-xr-x  2 root    root          7 Mai 19 09:02 .
Zeile 221: Zeile 244:
 
-rw-r--r--  1 root    root          0 Mai 19 09:02 smartos-zone-backup.log
 
-rw-r--r--  1 root    root          0 Mai 19 09:02 smartos-zone-backup.log
 
-rwxr-xr-x  1 root    root      11392 Mai 19 09:02 smartos-zone-backup.sh
 
-rwxr-xr-x  1 root    root      11392 Mai 19 09:02 smartos-zone-backup.sh
[root@server7smarti1 /zones/c3d2.de/admin]#
 
 
</source>
 
</source>
  
Zeile 228: Zeile 250:
 
==== alte Konfiguration ====
 
==== alte Konfiguration ====
  
 +
: <code>ipmitool lan print</code>
 
<source>
 
<source>
[root@smarti ~]# ipmitool lan print
 
 
Set in Progress        : Set Complete
 
Set in Progress        : Set Complete
 
Auth Type Support      : NONE MD2 MD5 PASSWORD
 
Auth Type Support      : NONE MD2 MD5 PASSWORD
Zeile 258: Zeile 280:
 
                         :    a=ADMIN
 
                         :    a=ADMIN
 
                         :    O=OEM
 
                         :    O=OEM
[root@smarti ~]#
 
 
</source>
 
</source>
  
 
==== neu konfigurieren ====
 
==== neu konfigurieren ====
  
 +
: <code>ipmitool lan set 1 ipsrc static</code>
 +
: <code>ipmitool lan set 1 ipaddr 172.22.99.244</code>
 
<source>
 
<source>
[root@smarti ~]#
 
[root@smarti ~]# ipmitool lan set 1 ipsrc static
 
[root@smarti ~]# ipmitool lan set 1 ipaddr 172.22.99.244
 
 
Setting LAN IP Address to 172.22.99.244
 
Setting LAN IP Address to 172.22.99.244
[root@smarti ~]# ipmitool lan set 1 netmask 255.255.255.0
+
</source>
 +
: <code>ipmitool lan set 1 netmask 255.255.255.0</code>
 +
<source>
 
Setting LAN Subnet Mask to 255.255.255.0
 
Setting LAN Subnet Mask to 255.255.255.0
[root@smarti ~]# ipmitool lan set 1 defgw ipaddr 172.22.99.1
+
</source>
 +
: <code>ipmitool lan set 1 defgw ipaddr 172.22.99.1</code>
 +
<source>
 
Setting LAN Default Gateway IP to 172.22.99.1
 
Setting LAN Default Gateway IP to 172.22.99.1
[root@smarti ~]# ipmitool lan set 1 defgw macaddr 0a:14:48:01:07:00
+
</source>
 +
: <code>ipmitool lan set 1 defgw macaddr 0a:14:48:01:07:00</code>
 +
<source>
 
Setting LAN Default Gateway MAC to 0a:14:48:01:07:00
 
Setting LAN Default Gateway MAC to 0a:14:48:01:07:00
[root@smarti ~]# ipmitool lan set 1 arp respond on
+
</source>
 +
: <code>ipmitool lan set 1 arp respond on</code>
 +
<source>
 
Enabling BMC-generated ARP responses
 
Enabling BMC-generated ARP responses
[root@smarti ~]# ipmitool lan set 1 auth ADMIN MD5
 
[root@smarti ~]# ipmitool lan set 1 access on
 
[root@smarti ~]#
 
 
</source>
 
</source>
 +
: <code>ipmitool lan set 1 auth ADMIN MD5</code>
 +
: <code>ipmitool lan set 1 access on</code>
  
 
==== neue Konfiguration ====
 
==== neue Konfiguration ====
  
 +
: <code>ipmitool lan print</code>
 
<source>
 
<source>
[root@smarti ~]# ipmitool lan print
 
 
Set in Progress        : Set Complete
 
Set in Progress        : Set Complete
 
Auth Type Support      : NONE MD2 MD5 PASSWORD
 
Auth Type Support      : NONE MD2 MD5 PASSWORD
Zeile 313: Zeile 340:
 
                         :    a=ADMIN
 
                         :    a=ADMIN
 
                         :    O=OEM
 
                         :    O=OEM
[root@smarti ~]#
 
 
</source>
 
</source>
  
Zeile 393: Zeile 419:
 
* Upgrade unbedingt per Firefox (nicht Google-Chrome / Chromium) durchführen
 
* Upgrade unbedingt per Firefox (nicht Google-Chrome / Chromium) durchführen
  
<source>
+
<pre>
 
   Module Name        Existing Version        New Version   
 
   Module Name        Existing Version        New Version   
 
   IPMI_FW            02.16                  03.36
 
   IPMI_FW            02.16                  03.36
</source>
+
</pre>
  
 
* bei Problemen mit der '''Java Security''' lassen sich alte IPMI Versionen auch lokal aus den hohen Sicherheitseinstellungen excluden, mittels:
 
* bei Problemen mit der '''Java Security''' lassen sich alte IPMI Versionen auch lokal aus den hohen Sicherheitseinstellungen excluden, mittels:
  
<source>
+
: <code>javaws -viewer</code>
javaws -viewer
+
 
</source>
+
=== SmartOS - LX-Zone (Debian) erstellen ===
  
 
=== SmartOS - KVM erstellen ===
 
=== SmartOS - KVM erstellen ===
Zeile 410: Zeile 436:
 
* Template ZFS Dataset erstellen
 
* Template ZFS Dataset erstellen
  
<source>
+
: <code>zfs create zones/c3d2.de/templates</code>
[root@server7smarti1 ~]#
 
[root@server7smarti1 ~]# zfs create zones/c3d2.de/templates
 
[root@server7smarti1 ~]#
 
</source>
 
  
 
* ISO downloaden
 
* ISO downloaden
  
 +
: <code>cd /zones/c3d2.de/iso/</code>
 +
: <code>wget --no-check-certificate https://releases.rancher.com/os/v1.5.1/rancheros.iso</code>
 
<source>
 
<source>
[root@server7smarti1 ~]#
 
[root@server7smarti1 ~]# cd /zones/c3d2.de/iso/
 
[root@server7smarti1 /zones/c3d2.de/iso]#
 
[root@server7smarti1 /zones/c3d2.de/iso]# wget --no-check-certificate https://releases.rancher.com/os/v1.5.1/rancheros.iso
 
 
--2019-05-19 09:28:08--  https://releases.rancher.com/os/v1.5.1/rancheros.iso
 
--2019-05-19 09:28:08--  https://releases.rancher.com/os/v1.5.1/rancheros.iso
 
Resolving releases.rancher.com (releases.rancher.com)... 104.24.16.51, 104.24.17.51, 2606:4700:20::6818:1033, ...
 
Resolving releases.rancher.com (releases.rancher.com)... 104.24.16.51, 104.24.17.51, 2606:4700:20::6818:1033, ...
Zeile 435: Zeile 455:
  
 
2019-05-19 09:28:23 (9,60 MB/s) - ‘rancheros.iso’ saved [135266304/135266304]
 
2019-05-19 09:28:23 (9,60 MB/s) - ‘rancheros.iso’ saved [135266304/135266304]
 
+
</source>
[root@server7smarti1 /zones/c3d2.de/iso]# ls -al
+
: <code>ls -al</code>
 +
<source>
 
total 261987
 
total 261987
 
drwxr-xr-x  2 root    root          3 Mai 19 09:28 .
 
drwxr-xr-x  2 root    root          3 Mai 19 09:28 .
 
drwxr-xr-x  5 root    root          5 Mai 19 09:27 ..
 
drwxr-xr-x  5 root    root          5 Mai 19 09:27 ..
 
-rw-r--r--  1 root    root    135266304 Feb. 11 17:14 rancheros.iso
 
-rw-r--r--  1 root    root    135266304 Feb. 11 17:14 rancheros.iso
[root@server7smarti1 /zones/c3d2.de/iso]# mv rancheros.iso rancheros151.iso
+
</source>
[root@server7smarti1 /zones/c3d2.de/iso]#
+
: <code>mv rancheros.iso rancheros151.iso</code>
[root@server7smarti1 /zones/c3d2.de/iso]# ls -al
+
: <code>ls -al</code>
 +
<source>
 
total 261987
 
total 261987
 
drwxr-xr-x  2 root    root          3 Mai 19 09:29 .
 
drwxr-xr-x  2 root    root          3 Mai 19 09:29 .
 
drwxr-xr-x  5 root    root          5 Mai 19 09:27 ..
 
drwxr-xr-x  5 root    root          5 Mai 19 09:27 ..
 
-rw-r--r--  1 root    root    135266304 Feb. 11 17:14 rancheros151.iso
 
-rw-r--r--  1 root    root    135266304 Feb. 11 17:14 rancheros151.iso
[root@server7smarti1 /zones/c3d2.de/iso]#
 
 
</source>
 
</source>
  
 
==== RancherOS KVM erstellen ====
 
==== RancherOS KVM erstellen ====
  
 +
: <code>cd /zones/c3d2.de/templates/</code>
 +
: <code>cat 1.server7smarti1-admin-kvm-ranchercluster1.json</code>
 
<source>
 
<source>
[root@server7smarti1 ~]#
 
[root@server7smarti1 ~]# cd /zones/c3d2.de/templates/
 
[root@server7smarti1 /zones/c3d2.de/templates]#
 
[root@server7smarti1 /zones/c3d2.de/templates]# cat 1.server7smarti1-admin-kvm-ranchercluster1.json
 
 
{
 
{
 
   "brand": "kvm",
 
   "brand": "kvm",
Zeile 496: Zeile 515:
 
   "qemu_extra_opts": "-k de"
 
   "qemu_extra_opts": "-k de"
 
}
 
}
 
+
</source>
[root@server7smarti1 /zones/c3d2.de/templates]# vmadm create -f 1.server7smarti1-admin-kvm-ranchercluster1.json
+
: <code>vmadm create -f 1.server7smarti1-admin-kvm-ranchercluster1.json</code>
 +
<source>
 
Successfully created VM 3516ab22-69b0-e327-95ec-f9be8852ee44
 
Successfully created VM 3516ab22-69b0-e327-95ec-f9be8852ee44
[root@server7smarti1 /zones/c3d2.de/templates]#
+
</source>
[root@server7smarti1 /zones/c3d2.de/templates]# sleep 30
+
: <code>sleep 30</code>
[root@server7smarti1 /zones/c3d2.de/templates]#
+
: <code>vmadm kill 3516ab22-69b0-e327-95ec-f9be8852ee44</code>
[root@server7smarti1 /zones/c3d2.de/templates]# vmadm kill 3516ab22-69b0-e327-95ec-f9be8852ee44
+
<source>
 
Sent signal "SIGTERM" to init process for VM 3516ab22-69b0-e327-95ec-f9be8852ee44
 
Sent signal "SIGTERM" to init process for VM 3516ab22-69b0-e327-95ec-f9be8852ee44
[root@server7smarti1 /zones/c3d2.de/templates]#
+
</source>
[root@server7smarti1 /zones/c3d2.de/templates]# sleep 10
+
: <code>sleep 10</code>
[root@server7smarti1 /zones/c3d2.de/templates]#
+
: <code>vmadm list</code>
[root@server7smarti1 /zones/c3d2.de/templates]# vmadm list
+
<source>
 
UUID                                  TYPE  RAM      STATE            ALIAS
 
UUID                                  TYPE  RAM      STATE            ALIAS
 
3516ab22-69b0-e327-95ec-f9be8852ee44  KVM  16384    stopped          server7smarti1-admin-kvm-ranchercluster1
 
3516ab22-69b0-e327-95ec-f9be8852ee44  KVM  16384    stopped          server7smarti1-admin-kvm-ranchercluster1
[root@server7smarti1 /zones/c3d2.de/templates]#
 
 
</source>
 
</source>
  
 
==== RancherOS ISO in die Non-Global Zone kopieren ====
 
==== RancherOS ISO in die Non-Global Zone kopieren ====
  
<source>
+
 
[root@server7smarti1 /zones/c3d2.de/templates]# cp /zones/c3d2.de/iso/rancheros151.iso /zones/3516ab22-69b0-e327-95ec-f9be8852ee44/root
+
: <code>cp /zones/c3d2.de/iso/rancheros151.iso /zones/3516ab22-69b0-e327-95ec-f9be8852ee44/root</code>
[root@server7smarti1 /zones/c3d2.de/templates]#
 
</source>
 
  
 
==== KVM Installation mit der ISO beginnen ====
 
==== KVM Installation mit der ISO beginnen ====
  
 +
: <code>vmadm list</code>
 
<source>
 
<source>
[root@server7smarti1 /zones/c3d2.de/templates]#
 
[root@server7smarti1 /zones/c3d2.de/templates]# vmadm list
 
 
UUID                                  TYPE  RAM      STATE            ALIAS
 
UUID                                  TYPE  RAM      STATE            ALIAS
 
3516ab22-69b0-e327-95ec-f9be8852ee44  KVM  16384    stopped          server7smarti1-admin-kvm-ranchercluster1
 
3516ab22-69b0-e327-95ec-f9be8852ee44  KVM  16384    stopped          server7smarti1-admin-kvm-ranchercluster1
[root@server7smarti1 /zones/c3d2.de/templates]#
+
</source>
[root@server7smarti1 /zones/c3d2.de/templates]# vmadm start 3516ab22-69b0-e327-95ec-f9be8852ee44 order=cd,once=d cdrom=rancheros151.iso,ide
+
: <code>vmadm start 3516ab22-69b0-e327-95ec-f9be8852ee44 order=cd,once=d cdrom=rancheros151.iso,ide</code>
 +
<source>
 
Successfully started VM 3516ab22-69b0-e327-95ec-f9be8852ee44
 
Successfully started VM 3516ab22-69b0-e327-95ec-f9be8852ee44
[root@server7smarti1 /zones/c3d2.de/templates]#
+
</source>
[root@server7smarti1 /zones/c3d2.de/templates]# vmadm info 3516ab22-69b0-e327-95ec-f9be8852ee44 vnc
+
: <code>vmadm info 3516ab22-69b0-e327-95ec-f9be8852ee44 vnc</code>
 +
<source>
 
{
 
{
 
   "vnc": {
 
   "vnc": {
Zeile 540: Zeile 558:
 
   }
 
   }
 
}
 
}
[root@server7smarti1 /zones/c3d2.de/templates]#
+
</source>
[root@server7smarti1 /zones/c3d2.de/templates]# vmadm list
+
: <code>vmadm list</code>
 +
<source>
 
UUID                                  TYPE  RAM      STATE            ALIAS
 
UUID                                  TYPE  RAM      STATE            ALIAS
 
3516ab22-69b0-e327-95ec-f9be8852ee44  KVM  16384    running          server7smarti1-admin-kvm-ranchercluster1
 
3516ab22-69b0-e327-95ec-f9be8852ee44  KVM  16384    running          server7smarti1-admin-kvm-ranchercluster1
[root@server7smarti1 /zones/c3d2.de/templates]#
 
 
</source>
 
</source>
  
Zeile 551: Zeile 569:
 
==== RancherOS Live System Environment ====
 
==== RancherOS Live System Environment ====
  
[[Datei:C3d2_smartos_kvm_rancheros_install_1.jpg]]
+
[[Datei:c3d2 smartos kvm rancheros install 1.jpg]]
  
 
==== Passwort vom User: rancher umbenennen und Remote SSH Login durchführen ====
 
==== Passwort vom User: rancher umbenennen und Remote SSH Login durchführen ====
  
<source>
+
: <code>sudo su</code>
[rancher@server7smarti1-admin-kvm-ranchercluster1 ~]$ sudo su
+
: <code>cd</code>
[root@server7smarti1-admin-kvm-ranchercluster1 rancher]# cd
+
: <code>passwd rancher</code>
[root@server7smarti1-admin-kvm-ranchercluster1 ~]#
+
: <code>exit</code>
[root@server7smarti1-admin-kvm-ranchercluster1 ~]# passwd rancher
 
[root@server7smarti1-admin-kvm-ranchercluster1 ~]#
 
[root@server7smarti1-admin-kvm-ranchercluster1 ~]# exit
 
</source>
 
  
[[Datei:C3d2_smartos_kvm_rancheros_install_2.jpg]]
+
[[Datei:c3d2 smartos kvm rancheros install 2.jpg]]
  
 
==== RancherOS auf die Festplatte installieren (Provisionierung) ====
 
==== RancherOS auf die Festplatte installieren (Provisionierung) ====
Zeile 570: Zeile 584:
 
* https://rancher.com/docs/os/v1.2/en/running-rancheros/server/install-to-disk/
 
* https://rancher.com/docs/os/v1.2/en/running-rancheros/server/install-to-disk/
  
 +
: <code>ssh rancher@172.22.99.246</code>
 
<source>
 
<source>
╭─daniel at it-daniel in ~ using
 
╰─○ ssh rancher@172.22.99.246
 
 
The authenticity of host '172.22.99.246 (172.22.99.246)' can't be established.
 
The authenticity of host '172.22.99.246 (172.22.99.246)' can't be established.
 
ECDSA key fingerprint is SHA256:Rfhqajk+ZOvEnXJRbD2gaoorArJotQOyIKCV0APk3gs.
 
ECDSA key fingerprint is SHA256:Rfhqajk+ZOvEnXJRbD2gaoorArJotQOyIKCV0APk3gs.
Zeile 578: Zeile 591:
 
Warning: Permanently added '172.22.99.246' (ECDSA) to the list of known hosts.
 
Warning: Permanently added '172.22.99.246' (ECDSA) to the list of known hosts.
 
rancher@172.22.99.246's password:
 
rancher@172.22.99.246's password:
[rancher@server7smarti1-admin-kvm-ranchercluster1 ~]$
+
</source>
[rancher@server7smarti1-admin-kvm-ranchercluster1 ~]$ sudo su
+
: <code>sudo su</code>
[root@server7smarti1-admin-kvm-ranchercluster1 rancher]# cd
+
: <code>cd</code>
[root@server7smarti1-admin-kvm-ranchercluster1 ~]#
+
: <code>ls -al</code>
[root@server7smarti1-admin-kvm-ranchercluster1 ~]# ls -al
+
<source>
 
total 4
 
total 4
 
drwx------    1 root    root            80 May 19 09:52 .
 
drwx------    1 root    root            80 May 19 09:52 .
Zeile 588: Zeile 601:
 
-rw-------    1 root    root            20 May 19 09:52 .bash_history
 
-rw-------    1 root    root            20 May 19 09:52 .bash_history
 
drwxr-xr-x    2 root    root            40 May 19 09:44 .ssh
 
drwxr-xr-x    2 root    root            40 May 19 09:44 .ssh
[root@server7smarti1-admin-kvm-ranchercluster1 ~]#
+
</source>
[root@server7smarti1-admin-kvm-ranchercluster1 ~]# vi cloud-config.yml
+
: <code>vi cloud-config.yml</code>
[root@server7smarti1-admin-kvm-ranchercluster1 ~]# cat cloud-config.yml
+
: <code>cat cloud-config.yml</code>
 +
<source>
 
#cloud-config
 
#cloud-config
  
Zeile 599: Zeile 613:
 
     autoformat:
 
     autoformat:
 
       - /dev/vda
 
       - /dev/vda
 
+
</source>
[root@server7smarti1-admin-kvm-ranchercluster1 ~]# sudo ros config validate -i cloud-config.yml
+
: <code>sudo ros config validate -i cloud-config.yml</code>
[root@server7smarti1-admin-kvm-ranchercluster1 ~]#
+
: <code>sudo ros install -c cloud-config.yml -d /dev/vda</code>
[root@server7smarti1-admin-kvm-ranchercluster1 ~]# sudo ros install -c cloud-config.yml -d /dev/vda
+
<source>
 
INFO[0000] No install type specified...defaulting to generic
 
INFO[0000] No install type specified...defaulting to generic
 
Installing from rancher/os:v1.5.1
 
Installing from rancher/os:v1.5.1
Zeile 625: Zeile 639:
 
Writing superblocks and filesystem accounting information: done
 
Writing superblocks and filesystem accounting information: done
  
 +
</source>
 +
<source>
 
Continue with reboot [y/N]: y
 
Continue with reboot [y/N]: y
 +
</source>
 +
<source>
 
INFO[0029] Rebooting
 
INFO[0029] Rebooting
 
INFO[0029] Setting reboot timeout to 60 (rancher.shutdown_timeout set to 60)
 
INFO[0029] Setting reboot timeout to 60 (rancher.shutdown_timeout set to 60)
Zeile 639: Zeile 657:
 
.Connection to 172.22.99.246 closed by remote host.
 
.Connection to 172.22.99.246 closed by remote host.
 
Connection to 172.22.99.246 closed.
 
Connection to 172.22.99.246 closed.
╭─daniel at it-daniel in ~ using
 
╰─○
 
 
</source>
 
</source>
  
 
==== erster Login mittels Public Keys ====
 
==== erster Login mittels Public Keys ====
  
 +
: <code>ssh -i /home/daniel/.ssh/id_plitc_ed25519 rancher@172.22.99.246</code>
 
<source>
 
<source>
╭─daniel at it-daniel in ~ using
 
╰─○ ssh -i /home/daniel/.ssh/id_plitc_ed25519 rancher@172.22.99.246
 
 
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
 
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
 
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!    @
 
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!    @
Zeile 663: Zeile 678:
 
ECDSA host key for 172.22.99.246 has changed and you have requested strict checking.
 
ECDSA host key for 172.22.99.246 has changed and you have requested strict checking.
 
Host key verification failed.
 
Host key verification failed.
 
+
</source>
╭─daniel at it-daniel in ~ using
+
: <code>ssh-keygen -f "/home/daniel/.ssh/known_hosts" -R "172.22.99.246"</code>
╰─○ ssh-keygen -f "/home/daniel/.ssh/known_hosts" -R "172.22.99.246"
+
<source>
 
# Host 172.22.99.246 found: line 278
 
# Host 172.22.99.246 found: line 278
 
/home/daniel/.ssh/known_hosts updated.
 
/home/daniel/.ssh/known_hosts updated.
 
Original contents retained as /home/daniel/.ssh/known_hosts.old
 
Original contents retained as /home/daniel/.ssh/known_hosts.old
 +
</source>
 +
: <code>ssh-keygen -f "/home/daniel/.ssh/known_hosts" -R "172.22.99.246"</code>
 +
<source>
 +
Host 172.22.99.246 not found in /home/daniel/.ssh/known_hosts
 +
</source>
 +
: <code>ssh -i /home/daniel/.ssh/id_plitc_ed25519 rancher@172.22.99.246</code>
 +
<source>
 +
Enter passphrase for key '/home/daniel/.ssh/id_plitc_ed25519':
 +
</source>
 +
: <code>sudo su</code>
 +
: <code>cd</code>
 +
 +
==== RancherOS - Nachoptimierungen ====
 +
 +
===== statische Netzwerk Konfiguration =====
 +
 +
* (ganz wichtig!)
 +
 +
: <code>ifconfig eth0</code>
 +
<source>
 +
eth0      Link encap:Ethernet  HWaddr 52:E5:76:CB:F1:9C
 +
          inet addr:172.22.99.246  Bcast:172.22.99.255  Mask:255.255.255.0
 +
          inet6 addr: fe80::50e5:76ff:fecb:f19c/64 Scope:Link
 +
          inet6 addr: 2a02:8106:208:5201:50e5:76ff:fecb:f19c/64 Scope:Global
 +
          inet6 addr: fd23:42:c3d2:523:50e5:76ff:fecb:f19c/64 Scope:Global
 +
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
 +
          RX packets:196667 errors:0 dropped:1 overruns:0 frame:0
 +
          TX packets:9800 errors:0 dropped:0 overruns:0 carrier:0
 +
          collisions:0 txqueuelen:1000
 +
          RX bytes:146184992 (139.4 MiB)  TX bytes:613199 (598.8 KiB)
 +
</source>
 +
: <code>ros config set rancher.network.interfaces.eth0.address 172.22.99.246/24</code>
 +
: <code>ros config set rancher.network.interfaces.eth0.gateway 172.22.99.1</code>
 +
: <code>ros config set rancher.network.interfaces.eth0.mtu 1500</code>
 +
: <code>ros config set rancher.network.interfaces.eth0.dhcp false</code>
 +
: <code>ros config set rancher.network.dns.nameservers "['9.9.9.9','149.112.112.112']"</code>
 +
 +
: <code>ros config set hostname server7smarti1-admin-kvm-ranchercluster1</code>
 +
 +
: <code>sync</code>
 +
 +
: <code>cat /var/lib/rancher/conf/cloud-config.yml | head -n 15</code>
 +
<source>
 +
hostname: server7smarti1-admin-kvm-ranchercluster1
 +
rancher:
 +
  network:
 +
    dns:
 +
      nameservers:
 +
      - 9.9.9.9
 +
      - 149.112.112.112
 +
    interfaces:
 +
      eth0:
 +
        address: 172.22.99.246/24
 +
        dhcp: false
 +
        gateway: 172.22.99.1
 +
        mtu: 1500
 +
  ssh:
 +
    keys:
 +
</source>
 +
: <code>reboot</code>
 +
 +
===== Kernelparameter =====
 +
 +
* https://rancher.com/docs/os/v1.1/en/configuration/adding-kernel-parameters/
 +
** tsc=reliable
 +
*** (bringt ca 5-10 % mehr Performance!")
 +
 +
: <code>ros config syslinux</code>
 +
 +
<source>
 +
tsc=reliable
 +
</source>
 +
 +
: <code>sync; reboot</code>
 +
 +
* nach Reboot
 +
 +
: <code>cat /proc/cmdline</code>
 +
<source>
 +
BOOT_IMAGE=../vmlinuz-4.14.85-rancher printk.devkmsg=on rancher.state.dev=LABEL=RANCHER_STATE rancher.state.wait panic=10 console=tty0 tsc=reliable  initrd=../initrd-v1.5.1
 +
</source>
 +
 +
=== Rancher UI (Headnode) ===
 +
 +
* Service Address: http://172.22.99.246:8080
 +
** User: root
 +
** PW: (wie gehabt)
 +
 +
* Headnode einrichten
 +
 +
: <code>sudo docker run -d --restart=always -p 8080:8080 rancher/server</code>
 +
<source>
 +
Unable to find image 'rancher/server:latest' locally
 +
latest: Pulling from rancher/server
 +
bae382666908: Pull complete
 +
29ede3c02ff2: Pull complete
 +
da4e69f33106: Pull complete
 +
8d43e5f5d27f: Pull complete
 +
b0de1abb17d6: Pull complete
 +
422f47db4517: Pull complete
 +
79d37de643ce: Pull complete
 +
69d13e08a4fe: Pull complete
 +
2ddfd3c6a2b7: Pull complete
 +
bc433fed3823: Pull complete
 +
b82e188df556: Pull complete
 +
dae2802428a4: Pull complete
 +
effdbd93afcb: Pull complete
 +
a4fcc35085ad: Pull complete
 +
e8234323b6c4: Pull complete
 +
d3f751a5d9cc: Pull complete
 +
d4b24e84b43b: Pull complete
 +
da9d7264902d: Pull complete
 +
df2b31306256: Pull complete
 +
c2238fcf71c2: Pull complete
 +
10c7c4a52421: Pull complete
 +
Digest: sha256:290e94536b32665d0ff537c2b947804faeed2768cd8652f0088a0d7e1acced75
 +
Status: Downloaded newer image for rancher/server:latest
 +
482bd209a572ab19700ef3a83bf0338bd1582a0ffdc07861dbe5da89a2ad0ed7
 +
</source>
 +
:<code>docker ps</code>
 +
<source>
 +
CONTAINER ID        IMAGE              COMMAND                  CREATED            STATUS              PORTS                              NAMES
 +
482bd209a572        rancher/server      "/usr/bin/entry /usr…"  16 seconds ago      Up 4 seconds        3306/tcp, 0.0.0.0:8080->8080/tcp  compassionate_elbakyan
 +
</source>
 +
 +
* lokale Authentifizierung eingestellt
 +
 +
[[Datei:c3d2 smartos kvm rancheros ui 1.jpg]]
 +
 +
==== Host Registration URL einrichten ====
 +
 +
[[Datei:c3d2 smartos kvm rancheros ui 2.jpg]]
  
╭─daniel at it-daniel in ~ using
+
==== neues Cattle Environment Template erstellen ====
╰─○ ssh-keygen -f "/home/daniel/.ssh/known_hosts" -R "172.22.99.246"
+
 
Host 172.22.99.246 not found in /home/daniel/.ssh/known_hosts
+
* '''mit VXLAN statt IPsec als Overlay Network Cross-Host Communication'''
 +
 
 +
[[Datei:c3d2 smartos kvm rancheros ui 3.jpg]]
 +
 
 +
==== neues Cattle Environment erstellen ====
 +
 
 +
[[Datei:c3d2 smartos kvm rancheros ui 4.jpg]]
 +
 
 +
* '''Default''' Environment entfernen
 +
 
 +
[[Datei:c3d2 smartos kvm rancheros ui 5.jpg]]
 +
 
 +
[[Datei:c3d2 smartos kvm rancheros ui 6.jpg]]
 +
 
 +
==== Agent auf dem Headnode installieren ====
 +
 
 +
; Wichtig: Es muss immer die -e CATTLE_AGENT_IP mit expliziter IP-Adresse angegeben werden, sonst bricht nach einer Weile die Verbindung zwischen den Compute-Nodes im Cluster!
 +
 
 +
: <code>ssh -p 2222 -i /home/daniel/.ssh/id_c3d2_ed25519 rancher@localhost</code>
 +
<source>
 +
Enter passphrase for key '/home/daniel/.ssh/id_c3d2_ed25519':
 +
</source>
 +
: <code>sudo su</code>
 +
: <code>cd</code>
 +
 
 +
: <code>docker ps</code>
 +
<source>
 +
CONTAINER ID        IMAGE              COMMAND                  CREATED            STATUS              PORTS                              NAMES
 +
482bd209a572        rancher/server      "/usr/bin/entry /usr…"   2 hours ago        Up 2 hours          3306/tcp, 0.0.0.0:8080->8080/tcp  compassionate_elbakyan
 +
</source>
 +
: <code>sudo docker run --rm --privileged -e CATTLE_AGENT_IP=172.22.99.246 -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.2.11 http://172.22.99.246:8080/v1/scripts/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX</code>
 +
<source>
 +
Unable to find image 'rancher/agent:v1.2.11' locally
 +
v1.2.11: Pulling from rancher/agent
 +
b3e1c725a85f: Pull complete
 +
6a710864a9fc: Pull complete
 +
d0ac3b234321: Pull complete
 +
87f567b5cf58: Pull complete
 +
063e24b217c4: Pull complete
 +
d0a3f58caef0: Pull complete
 +
16914729cfd3: Pull complete
 +
bbad862633b9: Pull complete
 +
3cf9849d7f3c: Pull complete
 +
Digest: sha256:0fba3fb10108f7821596dc5ad4bfa30e93426d034cd3471f6ccd3afb5f87a963
 +
Status: Downloaded newer image for rancher/agent:v1.2.11
 +
 
 +
INFO: Running Agent Registration Process, CATTLE_URL=http://172.22.99.246:8080/v1
 +
INFO: Attempting to connect to: http://172.22.99.246:8080/v1
 +
INFO: http://172.22.99.246:8080/v1 is accessible
 +
INFO: Configured Host Registration URL info: CATTLE_URL=http://172.22.99.246:8080/v1 ENV_URL=http://172.22.99.246:8080/v1
 +
INFO: Inspecting host capabilities
 +
INFO: Boot2Docker: false
 +
INFO: Host writable: true
 +
INFO: Token: xxxxxxxx
 +
INFO: Running registration
 +
INFO: Printing Environment
 +
INFO: ENV: CATTLE_ACCESS_KEY=XXXXXXXXXXXXXXXXXX
 +
INFO: ENV: CATTLE_AGENT_IP=172.22.99.246
 +
INFO: ENV: CATTLE_HOME=/var/lib/cattle
 +
INFO: ENV: CATTLE_REGISTRATION_ACCESS_KEY=registrationToken
 +
INFO: ENV: CATTLE_REGISTRATION_SECRET_KEY=xxxxxxx
 +
INFO: ENV: CATTLE_SECRET_KEY=xxxxxxx
 +
INFO: ENV: CATTLE_URL=http://172.22.99.246:8080/v1
 +
INFO: ENV: DETECTED_CATTLE_AGENT_IP=172.17.0.1
 +
INFO: ENV: RANCHER_AGENT_IMAGE=rancher/agent:v1.2.11
 +
INFO: Launched Rancher Agent: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx
 +
</source>
 +
 
 +
: <code>docker ps</code>
 +
<source>
 +
CONTAINER ID        IMAGE                            COMMAND                  CREATED              STATUS              PORTS                              NAMES
 +
9e96b29adb80        rancher/net:v0.11.9              "/rancher-entrypoint…"  29 seconds ago      Up 28 seconds                                          r-vxlan-vxlan-router-1-48b90d24
 +
58f7c1f85962        rancher/dns:v0.17.4              "/rancher-entrypoint…"  38 seconds ago      Up 37 seconds                                          r-network-services-metadata-dns-1-5975f2cb
 +
f5518d71078d        rancher/healthcheck:v0.3.8        "/.r/r /rancher-entr…"  41 seconds ago      Up 40 seconds                                          r-healthcheck-healthcheck-1-a5ec9f27
 +
209afda61fa3        rancher/metadata:v0.10.4          "/rancher-entrypoint…"  46 seconds ago      Up 45 seconds                                          r-network-services-metadata-1-d49e63b7
 +
49d91d5d2785        rancher/scheduler:v0.8.6          "/.r/r /rancher-entr…"  47 seconds ago      Up 46 seconds                                          r-scheduler-scheduler-1-3933ae84
 +
b062bbf2beba        rancher/network-manager:v0.7.22  "/rancher-entrypoint…"  53 seconds ago      Up 52 seconds                                          r-network-services-network-manager-1-d78bd33c
 +
c75e1c0d6c74        rancher/net:holder                "/.r/r /rancher-entr…"  55 seconds ago      Up 54 seconds                                          r-vxlan-vxlan-1-15cf7e5d
 +
3304d69c3be2        rancher/net:v0.13.1              "/rancher-entrypoint…"  56 seconds ago      Up 56 seconds                                          r-vxlan-cni-driver-1-a1e2d7e3
 +
596e621e7b45        rancher/agent:v1.2.11            "/run.sh run"            About a minute ago  Up About a minute                                      rancher-agent
 +
482bd209a572        rancher/server                    "/usr/bin/entry /usr…"  2 hours ago          Up 2 hours          3306/tcp, 0.0.0.0:8080->8080/tcp  compassionate_elbakyan
 +
</source>
 +
 
 +
[[Datei:c3d2 smartos kvm rancheros ui 7.jpg]]
 +
 
 +
==== Nachkorrekturen ====
 +
 
 +
; Wichtig: den Nodes immer eine eindeutige '''Scheduler IP''' zuweisen!
 +
 
 +
[[Datei:c3d2 smartos kvm rancheros ui 8.jpg]]
 +
 
 +
==== Statusanzeige ====
 +
 
 +
[[Datei:c3d2 smartos kvm rancheros ui 9.jpg]]
 +
 
 +
=== Rancher (Compute) Nodes hinzufügen ===
 +
 
 +
: <code>sudo docker run --rm --privileged -e CATTLE_AGENT_IP=172.22.99.XXX -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.2.11 http://172.22.99.246:8080/v1/scripts/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX</code>
 +
 
 +
=== Rancher - zusätzliche SSH Keys hinzufügen ===
 +
 
 +
: <code>cat new_ssh_keys.yml</code>
 +
<source>
 +
ssh_authorized_keys:
 +
- ssh-ed25519 XXXXXXXXX XXX
 +
- ssh-rsa XXXXXXXXX XXX
 +
</source>
 +
: <code>ros config merge -i new_ssh_keys.yml</code>
 +
: <code>vi /var/lib/rancher/conf/cloud-config.yml</code>
 +
: <code>sync</code>
 +
: <code>reboot</code>
 +
<source>
 +
[            ] reboot:info: Setting reboot timeout to 60 (rancher.shutdown_timeout set to 60)
 +
.........^[            ] reboot:info: Setting reboot timeout to 60 (rancher.shutdown_timeout set to 60)
 +
.=.[            ] reboot:info: Stopping /docker : 7e7d0702c70d
 +
.....................N.....................[            ] reboot:info: Stopping /ntp : f3a62bbc731c
 +
..?..[            ] reboot:info: Stopping /network : afb8d57014ff
 +
..<..[            ] reboot:info: Stopping /udev : de4ef7e5ac94
 +
..C..[            ] reboot:info: Stopping /system-cron : 59cc92c9b25b
 +
...?...[            ] reboot:info: Stopping /syslog : facbd277afe9
 +
..=..[            ] reboot:info: Stopping /acpid : fae77120e4dd
 +
...J...[            ] reboot:info: Console Stopping [/console] : 3ec527245aaf
 +
Connection to localhost closed by remote host.
 +
Connection to localhost closed.
 +
</source>
 +
 
 +
=== Rancher CLI (command-line interface) ===
 +
 
 +
==== Rancher API Key Zugriff ====
 +
 
 +
* das passende '''Rancher CLI''' Package (Binary) lässt sich (rechts unten) über den Rancher UI Link downloaden
 +
 
 +
* anschließend generiert man sich einen API Key und konfiguriert den Rancher CLI
 +
 
 +
[[Datei:c3d2 smartos kvm rancheros cli 1.jpg]]
 +
 
 +
==== lokales Volume erstellen ====
 +
 
 +
Es funktioniert genau wie angenommen:
 +
 
 +
* Über den Rancher CLI erstellt man ein lokales Volume
 +
* dieses ist standardmäßig auf inactive gesetzt und wird auch nicht per Rancher UI angezeigt / aufgelistet, da hier nur Volumes unter dem Typ Rancher-NFS aufgelistet werden
 +
(sofern dieses Plugin im Environment Template, beim initialen Cluster Bootstrap, eingebunden wurde)
 +
* die Verwaltung mittels Rancher CLI ist recht komfortabel
 +
 
 +
'''Rancher CLI'''
 +
 
 +
alle Volumes auflisten
 +
: <code>./rancher volume -a</code>
 +
local Volume erstellen
 +
: <code>./rancher volume create --driver local c3d2-data</code>
 +
Volume löschen
 +
: <code>./rancher volume rm 1v389</code>
 +
 
 +
'''per Rancher UI'''
  
╭─daniel at it-daniel in ~ using
+
docker-compose file mit volume mount
╰─○ ssh -i /home/daniel/.ssh/id_plitc_ed25519 rancher@172.22.99.246
+
: <code>cd /gitlab/docker-compose/productive/local/alpine-linux</code>
Enter passphrase for key '/home/daniel/.ssh/id_plitc_ed25519':
+
: <code>cat docker-compose.yaml</code>
 +
<source>
 +
version: '2'
  
[rancher@server7smarti1-admin-kvm-ranchercluster1 ~]$
+
services:
[rancher@server7smarti1-admin-kvm-ranchercluster1 ~]$ sudo su
+
  alpine:
[root@server7smarti1-admin-kvm-ranchercluster1 rancher]#
+
    image: alpine
[root@server7smarti1-admin-kvm-ranchercluster1 rancher]# cd
+
    stdin_open: true
[root@server7smarti1-admin-kvm-ranchercluster1 ~]#
+
    volumes:
 +
    - c3d2-data:/c3d2-data
 +
    volume_driver: local
 
</source>
 
</source>

Aktuelle Version vom 16. Januar 2020, 10:39 Uhr



Vorlage:anachronistisch
Achtung!
Anachronistisch!
Dieser Artikel enthält keine relevanten Informationen zu den aktuellen Verhältnissen. Ferner handelt es sich um einen archivierten Artikel.


Inhaltsverzeichnis

Ankündigung

Hallo,

das WE komme ich in den Club und hatte folgendes vor:

Eine minimalistische Ramdisk bauen, welche Netzwerkunterstützung und das Tool DD beinhaltet. Lauffähig soll es auf i386 Hardware mit max. 16 MB RAM! werden.

Freitag: (heute spät am Abend)
1. SmartOS Server als Virtualisierungsumgebung aufsetzen
2. RancherOS Cluster aufsetzen
3. Draw.IO aufsetzen, für Datenfluss Dokumentation

Samstag:
1. grundlegendes Verständnis erarbeiten was alles für den  Bau eines Images benötigt wird

Sonntag: (nice to have)
1. Templates für Docker Images erstellen und im Rancher Cluster testen

Freue mich auf eventuelle Mitstreiter 

Aufbau

  • 2 x FSC RX300 S6?
  • 2 x Dell R510?

Host

1HE Bladeserver: Supermicro 808-12#Board rechte Seite (Server/server7)

Host Einstellungen BIOS

Umsetzung

Datum
2019-05-19

Betriebssytem

Installation Betriebssystem

SmartOS

Installation Betriebssystem Vorbereitung

Starten vom Massenspeicher USB-Stick

Bootvorgang C3d2 smartos 2.jpg

Installation Betriebssystem Durchführung

C3d2 smartos install 1.jpg

C3d2 smartos install 2.jpg

C3d2 smartos install 3.jpg

C3d2 smartos install 4.jpg

C3d2 smartos install 5.jpg

C3d2 smartos install 6.jpg

C3d2 smartos install 7.jpg

Installation Betriebssystem Nachbereitung

zfs list
NAME                 USED  AVAIL  REFER  MOUNTPOINT
zones                137G   723G   784K  /zones
zones/archive         96K   723G    96K  /zones/archive
zones/config         124K   723G   124K  legacy
zones/cores          192K   723G    96K  none
zones/cores/global    96K  10,0G    96K  /zones/global/cores
zones/dump          4,92G   723G  4,92G  -
zones/opt             96K   723G    96K  legacy
zones/swap           132G   855G    56K  -
zones/usbkey         120K   723G   120K  legacy
zones/var           1,24M   723G  1,24M  legacy
zfs create zones/c3d2.de
zfs create zones/c3d2.de/admin
zfs create zones/c3d2.de/iso


zfs list -o name | egrep -v "NAME" | xargs -L 1 -I % zfs set checksum=on %
zfs set checksum=noparity zones/dump
zfs list -o name | egrep -v "NAME" | xargs -L 1 -I % zfs set compression=lz4 %
zfs set compression=off zones/dump
zfs set compression=off zones/swap
zfs list -o name | egrep -v "NAME" | xargs -L 1 -I % zfs set dedup=off %
zfs list -o name | egrep -v "NAME" | xargs -L 1 -I % zfs set atime=off %
cannot set property for 'zones/dump': 'atime' does not apply to datasets of this type
cannot set property for 'zones/swap': 'atime' does not apply to datasets of this type
zfs list -o name | egrep -v "NAME" | xargs -L 1 -I % zfs set primarycache=all %
zfs set primarycache=metadata zones/swap
Default System Config mit Germany Keymap
: <code>ssh root@172.22.99.245</code>
<source>
Password:
- SmartOS (build: 20190510T131809Z)
cat /usbkey/config
#
# This file was auto-generated and must be source-able by bash.
#
### ### ### C3D2 // ### ### ###

admin_nic=00:25:90:4f:1c:3c
admin_ip=172.22.99.245
admin_netmask=255.255.255.0
admin_network=
admin_gateway=172.22.99.245

headnode_default_gateway=172.22.99.1

dns_resolvers=9.9.9.9,149.112.112.112
dns_domain=c3d2.local

ntp_hosts=0.smartos.pool.ntp.org
compute_node_ntp_hosts=172.22.99.245

hostname=server7smarti1

default_keymap=germany

### ### ### // C3D2 ### ### ###
# EOF
sync; reboot
Connection to 172.22.99.245 closed by remote host.
Connection to 172.22.99.245 closed.

SmartOS Backup Script

cd /zones/c3d2.de/admin
cd /zones/c3d2.de/adminwget --no-check-certificate https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.conf
--2019-05-19 09:02:01--  https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.conf
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.12.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.12.133|:443... connected.
WARNING: cannot verify raw.githubusercontent.com's certificate, issued by ‘CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US’:
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 160 [text/plain]
Saving to: ‘smartos-zone-backup.conf’

smartos-zone-backup.conf                            100%[================================================================================================================>]     160  --.-KB/s    in 0s

2019-05-19 09:02:01 (8,35 MB/s) - ‘smartos-zone-backup.conf’ saved [160/160]
wget --no-check-certificate https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.exclude
--2019-05-19 09:02:08--  https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.exclude
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.12.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.12.133|:443... connected.
WARNING: cannot verify raw.githubusercontent.com's certificate, issued by ‘CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US’:
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 0 [text/plain]
Saving to: ‘smartos-zone-backup.exclude’

smartos-zone-backup.exclude                             [ <=>                                                                                                             ]       0  --.-KB/s    in 0s

2019-05-19 09:02:08 (0,00 B/s) - ‘smartos-zone-backup.exclude’ saved [0/0]
wget --no-check-certificate https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.include
--2019-05-19 09:02:12--  https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.include
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.12.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.12.133|:443... connected.
WARNING: cannot verify raw.githubusercontent.com's certificate, issued by ‘CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US’:
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 0 [text/plain]
Saving to: ‘smartos-zone-backup.include’

smartos-zone-backup.include                             [ <=>                                                                                                             ]       0  --.-KB/s    in 0s

2019-05-19 09:02:13 (0,00 B/s) - ‘smartos-zone-backup.include’ saved [0/0]
wget --no-check-certificate https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.log
--2019-05-19 09:02:20--  https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.log
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.112.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.112.133|:443... connected.
WARNING: cannot verify raw.githubusercontent.com's certificate, issued by ‘CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US’:
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 0 [text/plain]
Saving to: ‘smartos-zone-backup.log’

smartos-zone-backup.log                                 [ <=>                                                                                                             ]       0  --.-KB/s    in 0s

2019-05-19 09:02:20 (0,00 B/s) - ‘smartos-zone-backup.log’ saved [0/0]
wget --no-check-certificate https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.sh
--2019-05-19 09:02:22--  https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.sh
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.112.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.112.133|:443... connected.
WARNING: cannot verify raw.githubusercontent.com's certificate, issued by ‘CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US’:
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 11392 (11K) [text/plain]
Saving to: ‘smartos-zone-backup.sh’

smartos-zone-backup.sh                              100%[================================================================================================================>]  11,12K  --.-KB/s    in 0,009s

2019-05-19 09:02:24 (1,15 MB/s) - ‘smartos-zone-backup.sh’ saved [11392/11392]
chmod 0755 smartos-zone-backup.sh
ls -all
total 47
drwxr-xr-x   2 root     root           7 Mai 19 09:02 .
drwxr-xr-x   4 root     root           4 Mai 19 08:52 ..
-rw-r--r--   1 root     root         160 Mai 19 09:02 smartos-zone-backup.conf
-rw-r--r--   1 root     root           0 Mai 19 09:02 smartos-zone-backup.exclude
-rw-r--r--   1 root     root           0 Mai 19 09:02 smartos-zone-backup.include
-rw-r--r--   1 root     root           0 Mai 19 09:02 smartos-zone-backup.log
-rwxr-xr-x   1 root     root       11392 Mai 19 09:02 smartos-zone-backup.sh

IPMI per SmartOS (mittels ipmitool) konfigurieren

alte Konfiguration

ipmitool lan print
Set in Progress         : Set Complete
Auth Type Support       : NONE MD2 MD5 PASSWORD
Auth Type Enable        : Callback : MD2 MD5 PASSWORD
                        : User     : MD2 MD5 PASSWORD
                        : Operator : MD2 MD5 PASSWORD
                        : Admin    : MD2 MD5 PASSWORD
                        : OEM      : MD2 MD5 PASSWORD
IP Address Source       : Static Address
IP Address              : 10.0.3.23
Subnet Mask             : 255.255.255.0
MAC Address             : 00:25:90:2f:3d:fa
SNMP Community String   : public
IP Header               : TTL=0x00 Flags=0x00 Precedence=0x00 TOS=0x00
BMC ARP Control         : ARP Responses Enabled, Gratuitous ARP Disabled
Default Gateway IP      : 10.0.3.254
Default Gateway MAC     : 00:00:00:00:00:00
Backup Gateway IP       : 0.0.0.0
Backup Gateway MAC      : 00:00:00:00:00:00
802.1q VLAN ID          : Disabled
802.1q VLAN Priority    : 0
RMCP+ Cipher Suites     : 1,2,3,6,7,8,11,12
Cipher Suite Priv Max   : aaaaXXaaaXXaaXX
                        :     X=Cipher Suite Unused
                        :     c=CALLBACK
                        :     u=USER
                        :     o=OPERATOR
                        :     a=ADMIN
                        :     O=OEM

neu konfigurieren

ipmitool lan set 1 ipsrc static
ipmitool lan set 1 ipaddr 172.22.99.244
Setting LAN IP Address to 172.22.99.244
ipmitool lan set 1 netmask 255.255.255.0
Setting LAN Subnet Mask to 255.255.255.0
ipmitool lan set 1 defgw ipaddr 172.22.99.1
Setting LAN Default Gateway IP to 172.22.99.1
ipmitool lan set 1 defgw macaddr 0a:14:48:01:07:00
Setting LAN Default Gateway MAC to 0a:14:48:01:07:00
ipmitool lan set 1 arp respond on
Enabling BMC-generated ARP responses
ipmitool lan set 1 auth ADMIN MD5
ipmitool lan set 1 access on

neue Konfiguration

ipmitool lan print
Set in Progress         : Set Complete
Auth Type Support       : NONE MD2 MD5 PASSWORD
Auth Type Enable        : Callback : MD2 MD5 PASSWORD
                        : User     : MD2 MD5 PASSWORD
                        : Operator : MD2 MD5 PASSWORD
                        : Admin    : MD5
                        : OEM      : MD2 MD5 PASSWORD
IP Address Source       : Static Address
IP Address              : 172.22.99.244
Subnet Mask             : 255.255.255.0
MAC Address             : 00:25:90:2f:3d:fa
SNMP Community String   : public
IP Header               : TTL=0x00 Flags=0x00 Precedence=0x00 TOS=0x00
BMC ARP Control         : ARP Responses Enabled, Gratuitous ARP Disabled
Default Gateway IP      : 172.22.99.1
Default Gateway MAC     : 0a:14:48:01:07:00
Backup Gateway IP       : 0.0.0.0
Backup Gateway MAC      : 00:00:00:00:00:00
802.1q VLAN ID          : Disabled
802.1q VLAN Priority    : 0
RMCP+ Cipher Suites     : 1,2,3,6,7,8,11,12
Cipher Suite Priv Max   : aaaaXXaaaXXaaXX
                        :     X=Cipher Suite Unused
                        :     c=CALLBACK
                        :     u=USER
                        :     o=OPERATOR
                        :     a=ADMIN
                        :     O=OEM
  • neues ADMIN Passwort per IPMI Web UI gesetzt, siehe: http://172.22.99.244
    • PW Hinweis: (Denk an unsere Erde)

IPMI Firmware Update

  • Mainboard Version herausfinden
[root@smarti ~]# sysinfo
{
  "Live Image": "20190510T131809Z",
  "System Type": "SunOS",
  "Boot Time": "1558250640",
  "SDC Version": "7.0",
  "Manufacturer": "Supermicro",
  "Product": "X9DRT-F/IBQF/IBFF",
  "Serial Number": "0123456789",
  "SKU Number": "To be filled by O.E.M.",
  "HW Version": "0123456789",
  "HW Family": "To be filled by O.E.M.",
  "Setup": "false",
  "VM Capable": true,
  "Bhyve Capable": true,
  "Bhyve Max Vcpus": 32,
  "HVM API": true,
  "CPU Type": "Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz",
  "CPU Virtualization": "vmx",
  "CPU Physical Cores": 2,
  "Admin NIC Tag": "admin",
  "Admin IP": "dhcp",
  "UUID": "00000000-0000-0000-0000-0025904f1c3c",
  "Hostname": "smarti",
  "CPU Total Cores": 20,
  "MiB of Memory": "131038",
  "Zpool": "zones",
  "Zpool Disks": "c1t6479A71D12653333d0,c3t1d0",
  "Zpool Profile": "striped",
  "Zpool Creation": 1558250466,
  "Zpool Size in GiB": 860,
  "Disks": {
    "c1t6479A71D12653333d0": {"Size in GB": 960},
    "c3t1d0": {"Size in GB": 250}
  },
  "Boot Parameters": {
    "module_name_0": "environment",
    "console": "text",
    "boot_args": "",
    "bootargs": "",
    "bootfile": "unix",
    "os_console": "text",
    "root_shadow": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXxx",
    "screen_#cols": "158",
    "screen_#rows": "63",
    "screen_font": "8x16",
    "screen_height": "1024",
    "screen_width": "1280",
    "smartos": "true"
  },
  "Network Interfaces": {
    "igb0": {"MAC Address": "00:25:90:4f:1c:3c", "ip4addr": "172.22.99.186", "Link Status": "up", "NIC Names": ["admin"]},
    "igb1": {"MAC Address": "00:25:90:4f:1c:3d", "ip4addr": "", "Link Status": "down", "NIC Names": []}
  },
  "Virtual Network Interfaces": {
  },
  "Link Aggregations": {
  }
}
[root@smarti ~]#
  • Upgrade per Web UI
  • Upgrade unbedingt per Firefox (nicht Google-Chrome / Chromium) durchführen
  Module Name         Existing Version        New Version  
  IPMI_FW             02.16                   03.36
  • bei Problemen mit der Java Security lassen sich alte IPMI Versionen auch lokal aus den hohen Sicherheitseinstellungen excluden, mittels:
javaws -viewer

SmartOS - LX-Zone (Debian) erstellen

SmartOS - KVM erstellen

Vorbereitung

  • Template ZFS Dataset erstellen
zfs create zones/c3d2.de/templates
  • ISO downloaden
cd /zones/c3d2.de/iso/
wget --no-check-certificate https://releases.rancher.com/os/v1.5.1/rancheros.iso
--2019-05-19 09:28:08--  https://releases.rancher.com/os/v1.5.1/rancheros.iso
Resolving releases.rancher.com (releases.rancher.com)... 104.24.16.51, 104.24.17.51, 2606:4700:20::6818:1033, ...
Connecting to releases.rancher.com (releases.rancher.com)|104.24.16.51|:443... connected.
WARNING: cannot verify releases.rancher.com's certificate, issued by ‘CN=COMODO ECC Domain Validation Secure Server CA 2,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB’:
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 135266304 (129M) [application/x-iso9660-image]
Saving to: ‘rancheros.iso’

rancheros.iso                                       100%[================================================================================================================>] 129,00M  8,81MB/s    in 13s

2019-05-19 09:28:23 (9,60 MB/s) - ‘rancheros.iso’ saved [135266304/135266304]
ls -al
total 261987
drwxr-xr-x   2 root     root           3 Mai 19 09:28 .
drwxr-xr-x   5 root     root           5 Mai 19 09:27 ..
-rw-r--r--   1 root     root     135266304 Feb. 11 17:14 rancheros.iso
mv rancheros.iso rancheros151.iso
ls -al
total 261987
drwxr-xr-x   2 root     root           3 Mai 19 09:29 .
drwxr-xr-x   5 root     root           5 Mai 19 09:27 ..
-rw-r--r--   1 root     root     135266304 Feb. 11 17:14 rancheros151.iso

RancherOS KVM erstellen

cd /zones/c3d2.de/templates/
cat 1.server7smarti1-admin-kvm-ranchercluster1.json
{
  "brand": "kvm",
  "autoboot": true,
  "alias": "server7smarti1-admin-kvm-ranchercluster1",
  "hostname": "server7smarti1-admin-kvm-ranchercluster1",
  "resolvers": [
    "9.9.9.9",
    "149.112.112.112"
  ],
  "nics": [
    {
      "nic_tag": "admin",
      "ip": "172.22.99.246",
      "ips": ["172.22.99.246/24", "addrconf"],
      "netmask": "255.255.255.0",
      "gateway": "172.22.99.1",
      "model": "virtio",
      "allow_restricted_traffic": true,
      "primary": true
    }
  ],
  "vcpus": "6",
  "ram": "16384",
  "disks": [
    {
      "boot": true,
      "model": "virtio",
      "compression": "lz4",
      "size": 131072,
      "block_size": 8192
    }
  ],
  "internal_metadata": {},
  "vnc_port": "10001",
  "vnc_password": "c3d2",
  "cpu_type": "qemu64",
  "qemu_extra_opts": "-k de"
}
vmadm create -f 1.server7smarti1-admin-kvm-ranchercluster1.json
Successfully created VM 3516ab22-69b0-e327-95ec-f9be8852ee44
sleep 30
vmadm kill 3516ab22-69b0-e327-95ec-f9be8852ee44
Sent signal "SIGTERM" to init process for VM 3516ab22-69b0-e327-95ec-f9be8852ee44
sleep 10
vmadm list
UUID                                  TYPE  RAM      STATE             ALIAS
3516ab22-69b0-e327-95ec-f9be8852ee44  KVM   16384    stopped           server7smarti1-admin-kvm-ranchercluster1

RancherOS ISO in die Non-Global Zone kopieren

cp /zones/c3d2.de/iso/rancheros151.iso /zones/3516ab22-69b0-e327-95ec-f9be8852ee44/root

KVM Installation mit der ISO beginnen

vmadm list
UUID                                  TYPE  RAM      STATE             ALIAS
3516ab22-69b0-e327-95ec-f9be8852ee44  KVM   16384    stopped           server7smarti1-admin-kvm-ranchercluster1
vmadm start 3516ab22-69b0-e327-95ec-f9be8852ee44 order=cd,once=d cdrom=rancheros151.iso,ide
Successfully started VM 3516ab22-69b0-e327-95ec-f9be8852ee44
vmadm info 3516ab22-69b0-e327-95ec-f9be8852ee44 vnc
{
  "vnc": {
    "host": "172.22.99.245",
    "port": 10001,
    "display": 4101,
    "password": "c3d2"
  }
}
vmadm list
UUID                                  TYPE  RAM      STATE             ALIAS
3516ab22-69b0-e327-95ec-f9be8852ee44  KVM   16384    running           server7smarti1-admin-kvm-ranchercluster1

SmartOS - KVM - RancherOS Installation

RancherOS Live System Environment

C3d2 smartos kvm rancheros install 1.jpg

Passwort vom User: rancher umbenennen und Remote SSH Login durchführen

sudo su
cd
passwd rancher
exit

C3d2 smartos kvm rancheros install 2.jpg

RancherOS auf die Festplatte installieren (Provisionierung)

ssh rancher@172.22.99.246
The authenticity of host '172.22.99.246 (172.22.99.246)' can't be established.
ECDSA key fingerprint is SHA256:Rfhqajk+ZOvEnXJRbD2gaoorArJotQOyIKCV0APk3gs.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.22.99.246' (ECDSA) to the list of known hosts.
rancher@172.22.99.246's password:
sudo su
cd
ls -al
total 4
drwx------    1 root     root            80 May 19 09:52 .
drwxr-xr-x    1 root     root           140 May 19 09:44 ..
-rw-------    1 root     root            20 May 19 09:52 .bash_history
drwxr-xr-x    2 root     root            40 May 19 09:44 .ssh
vi cloud-config.yml
cat cloud-config.yml
#cloud-config

ssh_authorized_keys:
- ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAy9jaGaFOwpVr8eqUUqbs4YMOqzIpPVu5kyjZ9i3ZwC daniel@notebook1-plitc
rancher:
  state:
    autoformat:
      - /dev/vda
sudo ros config validate -i cloud-config.yml
sudo ros install -c cloud-config.yml -d /dev/vda
INFO[0000] No install type specified...defaulting to generic
Installing from rancher/os:v1.5.1
Continue [y/N]: y
INFO[0003] start !isoinstallerloaded
INFO[0004] trying to load /bootiso/rancheros/installer.tar.gz
Loaded image: rancher/os-installer:latest
INFO[0005] Loaded images from /bootiso/rancheros/installer.tar.gz
INFO[0005] starting installer container for rancher/os-installer:latest (new)
Installing from rancher/os-installer:latest
mke2fs 1.44.5 (15-Dec-2018)
64-bit filesystem support is not enabled.  The larger fields afforded by this feature enable full-strength checksumming.  Pass -O 64bit to rectify.
Creating filesystem with 33553920 4k blocks and 33554432 inodes
Filesystem UUID: 0f5e881b-ed38-41a7-b4f5-27543f46bd0d
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424, 20480000, 23887872

Allocating group tables: done
Writing inode tables: done
Creating journal (131072 blocks): done
Writing superblocks and filesystem accounting information: done
Continue with reboot [y/N]: y
INFO[0029] Rebooting
INFO[0029] Setting reboot timeout to 60 (rancher.shutdown_timeout set to 60)
.......^[            ] reboot:info: Setting reboot timeout to 60 (rancher.shutdown_timeout set to 60)
.=.[            ] reboot:info: Stopping /docker : b392faabaa0d
...........D...........[            ] reboot:info: Stopping /ntp : 53844dffa8a3
..?..[            ] reboot:info: Stopping /network : 9110824e3b85
...=...[            ] reboot:info: Stopping /udev : 5eb5aac54f4e
..C..[            ] reboot:info: Stopping /system-cron : 9ca03f465020
..=..[            ] reboot:info: Stopping /acpid : 4c0102079e9f
...>..[            ] reboot:info: Stopping /syslog : 6e8c71c91a67
.I..[            ] reboot:info: Console Stopping [/console] : 8575065938f2
.Connection to 172.22.99.246 closed by remote host.
Connection to 172.22.99.246 closed.

erster Login mittels Public Keys

ssh -i /home/daniel/.ssh/id_plitc_ed25519 rancher@172.22.99.246
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
SHA256:+TZdWdRG/CUdH3iJ1pNoPp303EhO+6M7qthxdn/AltI.
Please contact your system administrator.
Add correct host key in /home/daniel/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /home/daniel/.ssh/known_hosts:278
  remove with:
  ssh-keygen -f "/home/daniel/.ssh/known_hosts" -R "172.22.99.246"
ECDSA host key for 172.22.99.246 has changed and you have requested strict checking.
Host key verification failed.
ssh-keygen -f "/home/daniel/.ssh/known_hosts" -R "172.22.99.246"
# Host 172.22.99.246 found: line 278
/home/daniel/.ssh/known_hosts updated.
Original contents retained as /home/daniel/.ssh/known_hosts.old
ssh-keygen -f "/home/daniel/.ssh/known_hosts" -R "172.22.99.246"
Host 172.22.99.246 not found in /home/daniel/.ssh/known_hosts
ssh -i /home/daniel/.ssh/id_plitc_ed25519 rancher@172.22.99.246
Enter passphrase for key '/home/daniel/.ssh/id_plitc_ed25519':
sudo su
cd

RancherOS - Nachoptimierungen

statische Netzwerk Konfiguration
  • (ganz wichtig!)
ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 52:E5:76:CB:F1:9C
          inet addr:172.22.99.246  Bcast:172.22.99.255  Mask:255.255.255.0
          inet6 addr: fe80::50e5:76ff:fecb:f19c/64 Scope:Link
          inet6 addr: 2a02:8106:208:5201:50e5:76ff:fecb:f19c/64 Scope:Global
          inet6 addr: fd23:42:c3d2:523:50e5:76ff:fecb:f19c/64 Scope:Global
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:196667 errors:0 dropped:1 overruns:0 frame:0
          TX packets:9800 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:146184992 (139.4 MiB)  TX bytes:613199 (598.8 KiB)
ros config set rancher.network.interfaces.eth0.address 172.22.99.246/24
ros config set rancher.network.interfaces.eth0.gateway 172.22.99.1
ros config set rancher.network.interfaces.eth0.mtu 1500
ros config set rancher.network.interfaces.eth0.dhcp false
ros config set rancher.network.dns.nameservers "['9.9.9.9','149.112.112.112']"
ros config set hostname server7smarti1-admin-kvm-ranchercluster1
sync
cat /var/lib/rancher/conf/cloud-config.yml | head -n 15
hostname: server7smarti1-admin-kvm-ranchercluster1
rancher:
  network:
    dns:
      nameservers:
      - 9.9.9.9
      - 149.112.112.112
    interfaces:
      eth0:
        address: 172.22.99.246/24
        dhcp: false
        gateway: 172.22.99.1
        mtu: 1500
  ssh:
    keys:
reboot
Kernelparameter
ros config syslinux
tsc=reliable
sync; reboot
  • nach Reboot
cat /proc/cmdline
BOOT_IMAGE=../vmlinuz-4.14.85-rancher printk.devkmsg=on rancher.state.dev=LABEL=RANCHER_STATE rancher.state.wait panic=10 console=tty0 tsc=reliable  initrd=../initrd-v1.5.1

Rancher UI (Headnode)

  • Headnode einrichten
sudo docker run -d --restart=always -p 8080:8080 rancher/server
Unable to find image 'rancher/server:latest' locally
latest: Pulling from rancher/server
bae382666908: Pull complete
29ede3c02ff2: Pull complete
da4e69f33106: Pull complete
8d43e5f5d27f: Pull complete
b0de1abb17d6: Pull complete
422f47db4517: Pull complete
79d37de643ce: Pull complete
69d13e08a4fe: Pull complete
2ddfd3c6a2b7: Pull complete
bc433fed3823: Pull complete
b82e188df556: Pull complete
dae2802428a4: Pull complete
effdbd93afcb: Pull complete
a4fcc35085ad: Pull complete
e8234323b6c4: Pull complete
d3f751a5d9cc: Pull complete
d4b24e84b43b: Pull complete
da9d7264902d: Pull complete
df2b31306256: Pull complete
c2238fcf71c2: Pull complete
10c7c4a52421: Pull complete
Digest: sha256:290e94536b32665d0ff537c2b947804faeed2768cd8652f0088a0d7e1acced75
Status: Downloaded newer image for rancher/server:latest
482bd209a572ab19700ef3a83bf0338bd1582a0ffdc07861dbe5da89a2ad0ed7
docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                              NAMES
482bd209a572        rancher/server      "/usr/bin/entry /usr…"   16 seconds ago      Up 4 seconds        3306/tcp, 0.0.0.0:8080->8080/tcp   compassionate_elbakyan
  • lokale Authentifizierung eingestellt

C3d2 smartos kvm rancheros ui 1.jpg

Host Registration URL einrichten

C3d2 smartos kvm rancheros ui 2.jpg

neues Cattle Environment Template erstellen

  • mit VXLAN statt IPsec als Overlay Network Cross-Host Communication

C3d2 smartos kvm rancheros ui 3.jpg

neues Cattle Environment erstellen

C3d2 smartos kvm rancheros ui 4.jpg

  • Default Environment entfernen

C3d2 smartos kvm rancheros ui 5.jpg

C3d2 smartos kvm rancheros ui 6.jpg

Agent auf dem Headnode installieren

Wichtig
Es muss immer die -e CATTLE_AGENT_IP mit expliziter IP-Adresse angegeben werden, sonst bricht nach einer Weile die Verbindung zwischen den Compute-Nodes im Cluster!
ssh -p 2222 -i /home/daniel/.ssh/id_c3d2_ed25519 rancher@localhost
Enter passphrase for key '/home/daniel/.ssh/id_c3d2_ed25519':
sudo su
cd
docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                              NAMES
482bd209a572        rancher/server      "/usr/bin/entry /usr…"   2 hours ago         Up 2 hours          3306/tcp, 0.0.0.0:8080->8080/tcp   compassionate_elbakyan
sudo docker run --rm --privileged -e CATTLE_AGENT_IP=172.22.99.246 -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.2.11 http://172.22.99.246:8080/v1/scripts/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Unable to find image 'rancher/agent:v1.2.11' locally
v1.2.11: Pulling from rancher/agent
b3e1c725a85f: Pull complete
6a710864a9fc: Pull complete
d0ac3b234321: Pull complete
87f567b5cf58: Pull complete
063e24b217c4: Pull complete
d0a3f58caef0: Pull complete
16914729cfd3: Pull complete
bbad862633b9: Pull complete
3cf9849d7f3c: Pull complete
Digest: sha256:0fba3fb10108f7821596dc5ad4bfa30e93426d034cd3471f6ccd3afb5f87a963
Status: Downloaded newer image for rancher/agent:v1.2.11

INFO: Running Agent Registration Process, CATTLE_URL=http://172.22.99.246:8080/v1
INFO: Attempting to connect to: http://172.22.99.246:8080/v1
INFO: http://172.22.99.246:8080/v1 is accessible
INFO: Configured Host Registration URL info: CATTLE_URL=http://172.22.99.246:8080/v1 ENV_URL=http://172.22.99.246:8080/v1
INFO: Inspecting host capabilities
INFO: Boot2Docker: false
INFO: Host writable: true
INFO: Token: xxxxxxxx
INFO: Running registration
INFO: Printing Environment
INFO: ENV: CATTLE_ACCESS_KEY=XXXXXXXXXXXXXXXXXX
INFO: ENV: CATTLE_AGENT_IP=172.22.99.246
INFO: ENV: CATTLE_HOME=/var/lib/cattle
INFO: ENV: CATTLE_REGISTRATION_ACCESS_KEY=registrationToken
INFO: ENV: CATTLE_REGISTRATION_SECRET_KEY=xxxxxxx
INFO: ENV: CATTLE_SECRET_KEY=xxxxxxx
INFO: ENV: CATTLE_URL=http://172.22.99.246:8080/v1
INFO: ENV: DETECTED_CATTLE_AGENT_IP=172.17.0.1
INFO: ENV: RANCHER_AGENT_IMAGE=rancher/agent:v1.2.11
INFO: Launched Rancher Agent: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx
docker ps
CONTAINER ID        IMAGE                             COMMAND                  CREATED              STATUS              PORTS                              NAMES
9e96b29adb80        rancher/net:v0.11.9               "/rancher-entrypoint…"   29 seconds ago       Up 28 seconds                                          r-vxlan-vxlan-router-1-48b90d24
58f7c1f85962        rancher/dns:v0.17.4               "/rancher-entrypoint…"   38 seconds ago       Up 37 seconds                                          r-network-services-metadata-dns-1-5975f2cb
f5518d71078d        rancher/healthcheck:v0.3.8        "/.r/r /rancher-entr…"   41 seconds ago       Up 40 seconds                                          r-healthcheck-healthcheck-1-a5ec9f27
209afda61fa3        rancher/metadata:v0.10.4          "/rancher-entrypoint…"   46 seconds ago       Up 45 seconds                                          r-network-services-metadata-1-d49e63b7
49d91d5d2785        rancher/scheduler:v0.8.6          "/.r/r /rancher-entr…"   47 seconds ago       Up 46 seconds                                          r-scheduler-scheduler-1-3933ae84
b062bbf2beba        rancher/network-manager:v0.7.22   "/rancher-entrypoint…"   53 seconds ago       Up 52 seconds                                          r-network-services-network-manager-1-d78bd33c
c75e1c0d6c74        rancher/net:holder                "/.r/r /rancher-entr…"   55 seconds ago       Up 54 seconds                                          r-vxlan-vxlan-1-15cf7e5d
3304d69c3be2        rancher/net:v0.13.1               "/rancher-entrypoint…"   56 seconds ago       Up 56 seconds                                          r-vxlan-cni-driver-1-a1e2d7e3
596e621e7b45        rancher/agent:v1.2.11             "/run.sh run"            About a minute ago   Up About a minute                                      rancher-agent
482bd209a572        rancher/server                    "/usr/bin/entry /usr…"   2 hours ago          Up 2 hours          3306/tcp, 0.0.0.0:8080->8080/tcp   compassionate_elbakyan

C3d2 smartos kvm rancheros ui 7.jpg

Nachkorrekturen

Wichtig
den Nodes immer eine eindeutige Scheduler IP zuweisen!

C3d2 smartos kvm rancheros ui 8.jpg

Statusanzeige

C3d2 smartos kvm rancheros ui 9.jpg

Rancher (Compute) Nodes hinzufügen

sudo docker run --rm --privileged -e CATTLE_AGENT_IP=172.22.99.XXX -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.2.11 http://172.22.99.246:8080/v1/scripts/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Rancher - zusätzliche SSH Keys hinzufügen

cat new_ssh_keys.yml
ssh_authorized_keys:
- ssh-ed25519 XXXXXXXXX XXX
- ssh-rsa XXXXXXXXX XXX
ros config merge -i new_ssh_keys.yml
vi /var/lib/rancher/conf/cloud-config.yml
sync
reboot
[            ] reboot:info: Setting reboot timeout to 60 (rancher.shutdown_timeout set to 60)
.........^[            ] reboot:info: Setting reboot timeout to 60 (rancher.shutdown_timeout set to 60)
.=.[            ] reboot:info: Stopping /docker : 7e7d0702c70d
.....................N.....................[            ] reboot:info: Stopping /ntp : f3a62bbc731c
..?..[            ] reboot:info: Stopping /network : afb8d57014ff
..<..[            ] reboot:info: Stopping /udev : de4ef7e5ac94
..C..[            ] reboot:info: Stopping /system-cron : 59cc92c9b25b
...?...[            ] reboot:info: Stopping /syslog : facbd277afe9
..=..[            ] reboot:info: Stopping /acpid : fae77120e4dd
...J...[            ] reboot:info: Console Stopping [/console] : 3ec527245aaf
Connection to localhost closed by remote host.
Connection to localhost closed.

Rancher CLI (command-line interface)

Rancher API Key Zugriff

  • das passende Rancher CLI Package (Binary) lässt sich (rechts unten) über den Rancher UI Link downloaden
  • anschließend generiert man sich einen API Key und konfiguriert den Rancher CLI

C3d2 smartos kvm rancheros cli 1.jpg

lokales Volume erstellen

Es funktioniert genau wie angenommen:

  • Über den Rancher CLI erstellt man ein lokales Volume
  • dieses ist standardmäßig auf inactive gesetzt und wird auch nicht per Rancher UI angezeigt / aufgelistet, da hier nur Volumes unter dem Typ Rancher-NFS aufgelistet werden

(sofern dieses Plugin im Environment Template, beim initialen Cluster Bootstrap, eingebunden wurde)

  • die Verwaltung mittels Rancher CLI ist recht komfortabel

Rancher CLI

alle Volumes auflisten

./rancher volume -a

local Volume erstellen

./rancher volume create --driver local c3d2-data

Volume löschen

./rancher volume rm 1v389

per Rancher UI

docker-compose file mit volume mount

cd /gitlab/docker-compose/productive/local/alpine-linux
cat docker-compose.yaml
version: '2'

services:
  alpine:
    image: alpine
    stdin_open: true
    volumes:
    - c3d2-data:/c3d2-data
    volume_driver: local