RancherOS Cluster: Unterschied zwischen den Versionen

Aus C3D2
Wechseln zu: Navigation, Suche
K (Rancher API Key Zugriff)
(lokales Volume erstellen)
 
Zeile 964: Zeile 964:
 
'''Rancher CLI'''
 
'''Rancher CLI'''
  
<source>
+
alle Volumes auflisten
#// alle Volumes auflisten
+
: <code>./rancher volume -a</code>
./rancher volume -a
+
local Volume erstellen
 
+
: <code>./rancher volume create --driver local c3d2-data</code>
#// local Volume erstellen
+
Volume löschen
./rancher volume create --driver local c3d2-data
+
: <code>./rancher volume rm 1v389</code>
 
 
#// Volume löschen
 
./rancher volume rm 1v389
 
</source>
 
  
 
'''per Rancher UI'''
 
'''per Rancher UI'''
  
 +
docker-compose file mit volume mount
 +
: <code>cd /gitlab/docker-compose/productive/local/alpine-linux</code>
 +
: <code>cat docker-compose.yaml</code>
 
<source>
 
<source>
#// docker-compose file mit volume mount
 
 
╭─daniel at it-daniel in /gitlab/docker-compose/productive/local/alpine-linux on master✘✘✘ using
 
╰─± cat docker-compose.yaml
 
 
version: '2'
 
version: '2'
  
Zeile 991: Zeile 986:
 
     - c3d2-data:/c3d2-data
 
     - c3d2-data:/c3d2-data
 
     volume_driver: local
 
     volume_driver: local
 
╭─daniel at it-daniel in /gitlab/docker-compose/productive/local/alpine-linux on master✘✘✘ using
 
╰─±
 
 
</source>
 
</source>

Aktuelle Version vom 22. Mai 2019, 17:54 Uhr

Inhaltsverzeichnis

Ankündigung

Hallo,

das WE komme ich in den Club und hatte folgendes vor:

Eine minimalistische Ramdisk bauen, welche Netzwerkunterstützung und das Tool DD beinhaltet. Lauffähig soll es auf i386 Hardware mit max. 16 MB RAM! werden.

Freitag: (heute spät am Abend)
1. SmartOS Server als Virtualisierungsumgebung aufsetzen
2. RancherOS Cluster aufsetzen
3. Draw.IO aufsetzen, für Datenfluss Dokumentation

Samstag:
1. grundlegendes Verständnis erarbeiten was alles für den  Bau eines Images benötigt wird

Sonntag: (nice to have)
1. Templates für Docker Images erstellen und im Rancher Cluster testen

Freue mich auf eventuelle Mitstreiter 

Aufbau

  • 2 x FSC RX300 S6?
  • 2 x Dell R510?

Host

1HE Bladeserver: Supermicro 808-12#Board rechte Seite (Server/server7)

Host Einstellungen BIOS

Umsetzung

Datum
2019-05-19

Betriebssytem

Installation Betriebssystem

SmartOS

Installation Betriebssystem Vorbereitung

Starten vom Massenspeicher USB-Stick

Bootvorgang C3d2 smartos 2.jpg

Installation Betriebssystem Durchführung

C3d2 smartos install 1.jpg

C3d2 smartos install 2.jpg

C3d2 smartos install 3.jpg

C3d2 smartos install 4.jpg

C3d2 smartos install 5.jpg

C3d2 smartos install 6.jpg

C3d2 smartos install 7.jpg

Installation Betriebssystem Nachbereitung

zfs list
NAME                 USED  AVAIL  REFER  MOUNTPOINT
zones                137G   723G   784K  /zones
zones/archive         96K   723G    96K  /zones/archive
zones/config         124K   723G   124K  legacy
zones/cores          192K   723G    96K  none
zones/cores/global    96K  10,0G    96K  /zones/global/cores
zones/dump          4,92G   723G  4,92G  -
zones/opt             96K   723G    96K  legacy
zones/swap           132G   855G    56K  -
zones/usbkey         120K   723G   120K  legacy
zones/var           1,24M   723G  1,24M  legacy
zfs create zones/c3d2.de
zfs create zones/c3d2.de/admin
zfs create zones/c3d2.de/iso


zfs list -o name | egrep -v "NAME" | xargs -L 1 -I % zfs set checksum=on %
zfs set checksum=noparity zones/dump
zfs list -o name | egrep -v "NAME" | xargs -L 1 -I % zfs set compression=lz4 %
zfs set compression=off zones/dump
zfs set compression=off zones/swap
zfs list -o name | egrep -v "NAME" | xargs -L 1 -I % zfs set dedup=off %
zfs list -o name | egrep -v "NAME" | xargs -L 1 -I % zfs set atime=off %
cannot set property for 'zones/dump': 'atime' does not apply to datasets of this type
cannot set property for 'zones/swap': 'atime' does not apply to datasets of this type
zfs list -o name | egrep -v "NAME" | xargs -L 1 -I % zfs set primarycache=all %
zfs set primarycache=metadata zones/swap
Default System Config mit Germany Keymap
: <code>ssh root@172.22.99.245</code>
<source>
Password:
- SmartOS (build: 20190510T131809Z)
cat /usbkey/config
#
# This file was auto-generated and must be source-able by bash.
#
### ### ### C3D2 // ### ### ###

admin_nic=00:25:90:4f:1c:3c
admin_ip=172.22.99.245
admin_netmask=255.255.255.0
admin_network=
admin_gateway=172.22.99.245

headnode_default_gateway=172.22.99.1

dns_resolvers=9.9.9.9,149.112.112.112
dns_domain=c3d2.local

ntp_hosts=0.smartos.pool.ntp.org
compute_node_ntp_hosts=172.22.99.245

hostname=server7smarti1

default_keymap=germany

### ### ### // C3D2 ### ### ###
# EOF
sync; reboot
Connection to 172.22.99.245 closed by remote host.
Connection to 172.22.99.245 closed.

SmartOS Backup Script

cd /zones/c3d2.de/admin
cd /zones/c3d2.de/adminwget --no-check-certificate https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.conf
--2019-05-19 09:02:01--  https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.conf
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.12.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.12.133|:443... connected.
WARNING: cannot verify raw.githubusercontent.com's certificate, issued by ‘CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US’:
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 160 [text/plain]
Saving to: ‘smartos-zone-backup.conf’

smartos-zone-backup.conf                            100%[================================================================================================================>]     160  --.-KB/s    in 0s

2019-05-19 09:02:01 (8,35 MB/s) - ‘smartos-zone-backup.conf’ saved [160/160]
wget --no-check-certificate https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.exclude
--2019-05-19 09:02:08--  https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.exclude
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.12.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.12.133|:443... connected.
WARNING: cannot verify raw.githubusercontent.com's certificate, issued by ‘CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US’:
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 0 [text/plain]
Saving to: ‘smartos-zone-backup.exclude’

smartos-zone-backup.exclude                             [ <=>                                                                                                             ]       0  --.-KB/s    in 0s

2019-05-19 09:02:08 (0,00 B/s) - ‘smartos-zone-backup.exclude’ saved [0/0]
wget --no-check-certificate https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.include
--2019-05-19 09:02:12--  https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.include
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.12.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.12.133|:443... connected.
WARNING: cannot verify raw.githubusercontent.com's certificate, issued by ‘CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US’:
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 0 [text/plain]
Saving to: ‘smartos-zone-backup.include’

smartos-zone-backup.include                             [ <=>                                                                                                             ]       0  --.-KB/s    in 0s

2019-05-19 09:02:13 (0,00 B/s) - ‘smartos-zone-backup.include’ saved [0/0]
wget --no-check-certificate https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.log
--2019-05-19 09:02:20--  https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.log
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.112.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.112.133|:443... connected.
WARNING: cannot verify raw.githubusercontent.com's certificate, issued by ‘CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US’:
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 0 [text/plain]
Saving to: ‘smartos-zone-backup.log’

smartos-zone-backup.log                                 [ <=>                                                                                                             ]       0  --.-KB/s    in 0s

2019-05-19 09:02:20 (0,00 B/s) - ‘smartos-zone-backup.log’ saved [0/0]
wget --no-check-certificate https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.sh
--2019-05-19 09:02:22--  https://raw.githubusercontent.com/ass-a2s/smartos-zone-backup/master/smartos-zone-backup.sh
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.112.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.112.133|:443... connected.
WARNING: cannot verify raw.githubusercontent.com's certificate, issued by ‘CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US’:
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 11392 (11K) [text/plain]
Saving to: ‘smartos-zone-backup.sh’

smartos-zone-backup.sh                              100%[================================================================================================================>]  11,12K  --.-KB/s    in 0,009s

2019-05-19 09:02:24 (1,15 MB/s) - ‘smartos-zone-backup.sh’ saved [11392/11392]
chmod 0755 smartos-zone-backup.sh
ls -all
total 47
drwxr-xr-x   2 root     root           7 Mai 19 09:02 .
drwxr-xr-x   4 root     root           4 Mai 19 08:52 ..
-rw-r--r--   1 root     root         160 Mai 19 09:02 smartos-zone-backup.conf
-rw-r--r--   1 root     root           0 Mai 19 09:02 smartos-zone-backup.exclude
-rw-r--r--   1 root     root           0 Mai 19 09:02 smartos-zone-backup.include
-rw-r--r--   1 root     root           0 Mai 19 09:02 smartos-zone-backup.log
-rwxr-xr-x   1 root     root       11392 Mai 19 09:02 smartos-zone-backup.sh

IPMI per SmartOS (mittels ipmitool) konfigurieren

alte Konfiguration

ipmitool lan print
Set in Progress         : Set Complete
Auth Type Support       : NONE MD2 MD5 PASSWORD
Auth Type Enable        : Callback : MD2 MD5 PASSWORD
                        : User     : MD2 MD5 PASSWORD
                        : Operator : MD2 MD5 PASSWORD
                        : Admin    : MD2 MD5 PASSWORD
                        : OEM      : MD2 MD5 PASSWORD
IP Address Source       : Static Address
IP Address              : 10.0.3.23
Subnet Mask             : 255.255.255.0
MAC Address             : 00:25:90:2f:3d:fa
SNMP Community String   : public
IP Header               : TTL=0x00 Flags=0x00 Precedence=0x00 TOS=0x00
BMC ARP Control         : ARP Responses Enabled, Gratuitous ARP Disabled
Default Gateway IP      : 10.0.3.254
Default Gateway MAC     : 00:00:00:00:00:00
Backup Gateway IP       : 0.0.0.0
Backup Gateway MAC      : 00:00:00:00:00:00
802.1q VLAN ID          : Disabled
802.1q VLAN Priority    : 0
RMCP+ Cipher Suites     : 1,2,3,6,7,8,11,12
Cipher Suite Priv Max   : aaaaXXaaaXXaaXX
                        :     X=Cipher Suite Unused
                        :     c=CALLBACK
                        :     u=USER
                        :     o=OPERATOR
                        :     a=ADMIN
                        :     O=OEM

neu konfigurieren

ipmitool lan set 1 ipsrc static
ipmitool lan set 1 ipaddr 172.22.99.244
Setting LAN IP Address to 172.22.99.244
ipmitool lan set 1 netmask 255.255.255.0
Setting LAN Subnet Mask to 255.255.255.0
ipmitool lan set 1 defgw ipaddr 172.22.99.1
Setting LAN Default Gateway IP to 172.22.99.1
ipmitool lan set 1 defgw macaddr 0a:14:48:01:07:00
Setting LAN Default Gateway MAC to 0a:14:48:01:07:00
ipmitool lan set 1 arp respond on
Enabling BMC-generated ARP responses
ipmitool lan set 1 auth ADMIN MD5
ipmitool lan set 1 access on

neue Konfiguration

ipmitool lan print
Set in Progress         : Set Complete
Auth Type Support       : NONE MD2 MD5 PASSWORD
Auth Type Enable        : Callback : MD2 MD5 PASSWORD
                        : User     : MD2 MD5 PASSWORD
                        : Operator : MD2 MD5 PASSWORD
                        : Admin    : MD5
                        : OEM      : MD2 MD5 PASSWORD
IP Address Source       : Static Address
IP Address              : 172.22.99.244
Subnet Mask             : 255.255.255.0
MAC Address             : 00:25:90:2f:3d:fa
SNMP Community String   : public
IP Header               : TTL=0x00 Flags=0x00 Precedence=0x00 TOS=0x00
BMC ARP Control         : ARP Responses Enabled, Gratuitous ARP Disabled
Default Gateway IP      : 172.22.99.1
Default Gateway MAC     : 0a:14:48:01:07:00
Backup Gateway IP       : 0.0.0.0
Backup Gateway MAC      : 00:00:00:00:00:00
802.1q VLAN ID          : Disabled
802.1q VLAN Priority    : 0
RMCP+ Cipher Suites     : 1,2,3,6,7,8,11,12
Cipher Suite Priv Max   : aaaaXXaaaXXaaXX
                        :     X=Cipher Suite Unused
                        :     c=CALLBACK
                        :     u=USER
                        :     o=OPERATOR
                        :     a=ADMIN
                        :     O=OEM
  • neues ADMIN Passwort per IPMI Web UI gesetzt, siehe: http://172.22.99.244
    • PW Hinweis: (Denk an unsere Erde)

IPMI Firmware Update

  • Mainboard Version herausfinden
[root@smarti ~]# sysinfo
{
  "Live Image": "20190510T131809Z",
  "System Type": "SunOS",
  "Boot Time": "1558250640",
  "SDC Version": "7.0",
  "Manufacturer": "Supermicro",
  "Product": "X9DRT-F/IBQF/IBFF",
  "Serial Number": "0123456789",
  "SKU Number": "To be filled by O.E.M.",
  "HW Version": "0123456789",
  "HW Family": "To be filled by O.E.M.",
  "Setup": "false",
  "VM Capable": true,
  "Bhyve Capable": true,
  "Bhyve Max Vcpus": 32,
  "HVM API": true,
  "CPU Type": "Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz",
  "CPU Virtualization": "vmx",
  "CPU Physical Cores": 2,
  "Admin NIC Tag": "admin",
  "Admin IP": "dhcp",
  "UUID": "00000000-0000-0000-0000-0025904f1c3c",
  "Hostname": "smarti",
  "CPU Total Cores": 20,
  "MiB of Memory": "131038",
  "Zpool": "zones",
  "Zpool Disks": "c1t6479A71D12653333d0,c3t1d0",
  "Zpool Profile": "striped",
  "Zpool Creation": 1558250466,
  "Zpool Size in GiB": 860,
  "Disks": {
    "c1t6479A71D12653333d0": {"Size in GB": 960},
    "c3t1d0": {"Size in GB": 250}
  },
  "Boot Parameters": {
    "module_name_0": "environment",
    "console": "text",
    "boot_args": "",
    "bootargs": "",
    "bootfile": "unix",
    "os_console": "text",
    "root_shadow": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXxx",
    "screen_#cols": "158",
    "screen_#rows": "63",
    "screen_font": "8x16",
    "screen_height": "1024",
    "screen_width": "1280",
    "smartos": "true"
  },
  "Network Interfaces": {
    "igb0": {"MAC Address": "00:25:90:4f:1c:3c", "ip4addr": "172.22.99.186", "Link Status": "up", "NIC Names": ["admin"]},
    "igb1": {"MAC Address": "00:25:90:4f:1c:3d", "ip4addr": "", "Link Status": "down", "NIC Names": []}
  },
  "Virtual Network Interfaces": {
  },
  "Link Aggregations": {
  }
}
[root@smarti ~]#
  • Upgrade per Web UI
  • Upgrade unbedingt per Firefox (nicht Google-Chrome / Chromium) durchführen
  Module Name         Existing Version        New Version  
  IPMI_FW             02.16                   03.36
  • bei Problemen mit der Java Security lassen sich alte IPMI Versionen auch lokal aus den hohen Sicherheitseinstellungen excluden, mittels:
javaws -viewer

SmartOS - LX-Zone (Debian) erstellen

SmartOS - KVM erstellen

Vorbereitung

  • Template ZFS Dataset erstellen
zfs create zones/c3d2.de/templates
  • ISO downloaden
cd /zones/c3d2.de/iso/
wget --no-check-certificate https://releases.rancher.com/os/v1.5.1/rancheros.iso
--2019-05-19 09:28:08--  https://releases.rancher.com/os/v1.5.1/rancheros.iso
Resolving releases.rancher.com (releases.rancher.com)... 104.24.16.51, 104.24.17.51, 2606:4700:20::6818:1033, ...
Connecting to releases.rancher.com (releases.rancher.com)|104.24.16.51|:443... connected.
WARNING: cannot verify releases.rancher.com's certificate, issued by ‘CN=COMODO ECC Domain Validation Secure Server CA 2,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB’:
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 135266304 (129M) [application/x-iso9660-image]
Saving to: ‘rancheros.iso’

rancheros.iso                                       100%[================================================================================================================>] 129,00M  8,81MB/s    in 13s

2019-05-19 09:28:23 (9,60 MB/s) - ‘rancheros.iso’ saved [135266304/135266304]
ls -al
total 261987
drwxr-xr-x   2 root     root           3 Mai 19 09:28 .
drwxr-xr-x   5 root     root           5 Mai 19 09:27 ..
-rw-r--r--   1 root     root     135266304 Feb. 11 17:14 rancheros.iso
mv rancheros.iso rancheros151.iso
ls -al
total 261987
drwxr-xr-x   2 root     root           3 Mai 19 09:29 .
drwxr-xr-x   5 root     root           5 Mai 19 09:27 ..
-rw-r--r--   1 root     root     135266304 Feb. 11 17:14 rancheros151.iso

RancherOS KVM erstellen

cd /zones/c3d2.de/templates/
cat 1.server7smarti1-admin-kvm-ranchercluster1.json
{
  "brand": "kvm",
  "autoboot": true,
  "alias": "server7smarti1-admin-kvm-ranchercluster1",
  "hostname": "server7smarti1-admin-kvm-ranchercluster1",
  "resolvers": [
    "9.9.9.9",
    "149.112.112.112"
  ],
  "nics": [
    {
      "nic_tag": "admin",
      "ip": "172.22.99.246",
      "ips": ["172.22.99.246/24", "addrconf"],
      "netmask": "255.255.255.0",
      "gateway": "172.22.99.1",
      "model": "virtio",
      "allow_restricted_traffic": true,
      "primary": true
    }
  ],
  "vcpus": "6",
  "ram": "16384",
  "disks": [
    {
      "boot": true,
      "model": "virtio",
      "compression": "lz4",
      "size": 131072,
      "block_size": 8192
    }
  ],
  "internal_metadata": {},
  "vnc_port": "10001",
  "vnc_password": "c3d2",
  "cpu_type": "qemu64",
  "qemu_extra_opts": "-k de"
}
vmadm create -f 1.server7smarti1-admin-kvm-ranchercluster1.json
Successfully created VM 3516ab22-69b0-e327-95ec-f9be8852ee44
sleep 30
vmadm kill 3516ab22-69b0-e327-95ec-f9be8852ee44
Sent signal "SIGTERM" to init process for VM 3516ab22-69b0-e327-95ec-f9be8852ee44
sleep 10
vmadm list
UUID                                  TYPE  RAM      STATE             ALIAS
3516ab22-69b0-e327-95ec-f9be8852ee44  KVM   16384    stopped           server7smarti1-admin-kvm-ranchercluster1

RancherOS ISO in die Non-Global Zone kopieren

cp /zones/c3d2.de/iso/rancheros151.iso /zones/3516ab22-69b0-e327-95ec-f9be8852ee44/root

KVM Installation mit der ISO beginnen

vmadm list
UUID                                  TYPE  RAM      STATE             ALIAS
3516ab22-69b0-e327-95ec-f9be8852ee44  KVM   16384    stopped           server7smarti1-admin-kvm-ranchercluster1
vmadm start 3516ab22-69b0-e327-95ec-f9be8852ee44 order=cd,once=d cdrom=rancheros151.iso,ide
Successfully started VM 3516ab22-69b0-e327-95ec-f9be8852ee44
vmadm info 3516ab22-69b0-e327-95ec-f9be8852ee44 vnc
{
  "vnc": {
    "host": "172.22.99.245",
    "port": 10001,
    "display": 4101,
    "password": "c3d2"
  }
}
vmadm list
UUID                                  TYPE  RAM      STATE             ALIAS
3516ab22-69b0-e327-95ec-f9be8852ee44  KVM   16384    running           server7smarti1-admin-kvm-ranchercluster1

SmartOS - KVM - RancherOS Installation

RancherOS Live System Environment

C3d2 smartos kvm rancheros install 1.jpg

Passwort vom User: rancher umbenennen und Remote SSH Login durchführen

sudo su
cd
passwd rancher
exit

C3d2 smartos kvm rancheros install 2.jpg

RancherOS auf die Festplatte installieren (Provisionierung)

ssh rancher@172.22.99.246
The authenticity of host '172.22.99.246 (172.22.99.246)' can't be established.
ECDSA key fingerprint is SHA256:Rfhqajk+ZOvEnXJRbD2gaoorArJotQOyIKCV0APk3gs.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.22.99.246' (ECDSA) to the list of known hosts.
rancher@172.22.99.246's password:
sudo su
cd
ls -al
total 4
drwx------    1 root     root            80 May 19 09:52 .
drwxr-xr-x    1 root     root           140 May 19 09:44 ..
-rw-------    1 root     root            20 May 19 09:52 .bash_history
drwxr-xr-x    2 root     root            40 May 19 09:44 .ssh
vi cloud-config.yml
cat cloud-config.yml
#cloud-config

ssh_authorized_keys:
- ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAy9jaGaFOwpVr8eqUUqbs4YMOqzIpPVu5kyjZ9i3ZwC daniel@notebook1-plitc
rancher:
  state:
    autoformat:
      - /dev/vda
sudo ros config validate -i cloud-config.yml
sudo ros install -c cloud-config.yml -d /dev/vda
INFO[0000] No install type specified...defaulting to generic
Installing from rancher/os:v1.5.1
Continue [y/N]: y
INFO[0003] start !isoinstallerloaded
INFO[0004] trying to load /bootiso/rancheros/installer.tar.gz
Loaded image: rancher/os-installer:latest
INFO[0005] Loaded images from /bootiso/rancheros/installer.tar.gz
INFO[0005] starting installer container for rancher/os-installer:latest (new)
Installing from rancher/os-installer:latest
mke2fs 1.44.5 (15-Dec-2018)
64-bit filesystem support is not enabled.  The larger fields afforded by this feature enable full-strength checksumming.  Pass -O 64bit to rectify.
Creating filesystem with 33553920 4k blocks and 33554432 inodes
Filesystem UUID: 0f5e881b-ed38-41a7-b4f5-27543f46bd0d
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424, 20480000, 23887872

Allocating group tables: done
Writing inode tables: done
Creating journal (131072 blocks): done
Writing superblocks and filesystem accounting information: done
Continue with reboot [y/N]: y
INFO[0029] Rebooting
INFO[0029] Setting reboot timeout to 60 (rancher.shutdown_timeout set to 60)
.......^[            ] reboot:info: Setting reboot timeout to 60 (rancher.shutdown_timeout set to 60)
.=.[            ] reboot:info: Stopping /docker : b392faabaa0d
...........D...........[            ] reboot:info: Stopping /ntp : 53844dffa8a3
..?..[            ] reboot:info: Stopping /network : 9110824e3b85
...=...[            ] reboot:info: Stopping /udev : 5eb5aac54f4e
..C..[            ] reboot:info: Stopping /system-cron : 9ca03f465020
..=..[            ] reboot:info: Stopping /acpid : 4c0102079e9f
...>..[            ] reboot:info: Stopping /syslog : 6e8c71c91a67
.I..[            ] reboot:info: Console Stopping [/console] : 8575065938f2
.Connection to 172.22.99.246 closed by remote host.
Connection to 172.22.99.246 closed.

erster Login mittels Public Keys

ssh -i /home/daniel/.ssh/id_plitc_ed25519 rancher@172.22.99.246
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
SHA256:+TZdWdRG/CUdH3iJ1pNoPp303EhO+6M7qthxdn/AltI.
Please contact your system administrator.
Add correct host key in /home/daniel/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /home/daniel/.ssh/known_hosts:278
  remove with:
  ssh-keygen -f "/home/daniel/.ssh/known_hosts" -R "172.22.99.246"
ECDSA host key for 172.22.99.246 has changed and you have requested strict checking.
Host key verification failed.
ssh-keygen -f "/home/daniel/.ssh/known_hosts" -R "172.22.99.246"
# Host 172.22.99.246 found: line 278
/home/daniel/.ssh/known_hosts updated.
Original contents retained as /home/daniel/.ssh/known_hosts.old
ssh-keygen -f "/home/daniel/.ssh/known_hosts" -R "172.22.99.246"
Host 172.22.99.246 not found in /home/daniel/.ssh/known_hosts
ssh -i /home/daniel/.ssh/id_plitc_ed25519 rancher@172.22.99.246
Enter passphrase for key '/home/daniel/.ssh/id_plitc_ed25519':
sudo su
cd

RancherOS - Nachoptimierungen

statische Netzwerk Konfiguration
  • (ganz wichtig!)
ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 52:E5:76:CB:F1:9C
          inet addr:172.22.99.246  Bcast:172.22.99.255  Mask:255.255.255.0
          inet6 addr: fe80::50e5:76ff:fecb:f19c/64 Scope:Link
          inet6 addr: 2a02:8106:208:5201:50e5:76ff:fecb:f19c/64 Scope:Global
          inet6 addr: fd23:42:c3d2:523:50e5:76ff:fecb:f19c/64 Scope:Global
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:196667 errors:0 dropped:1 overruns:0 frame:0
          TX packets:9800 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:146184992 (139.4 MiB)  TX bytes:613199 (598.8 KiB)
ros config set rancher.network.interfaces.eth0.address 172.22.99.246/24
ros config set rancher.network.interfaces.eth0.gateway 172.22.99.1
ros config set rancher.network.interfaces.eth0.mtu 1500
ros config set rancher.network.interfaces.eth0.dhcp false
ros config set rancher.network.dns.nameservers "['9.9.9.9','149.112.112.112']"
ros config set hostname server7smarti1-admin-kvm-ranchercluster1
sync
cat /var/lib/rancher/conf/cloud-config.yml | head -n 15
hostname: server7smarti1-admin-kvm-ranchercluster1
rancher:
  network:
    dns:
      nameservers:
      - 9.9.9.9
      - 149.112.112.112
    interfaces:
      eth0:
        address: 172.22.99.246/24
        dhcp: false
        gateway: 172.22.99.1
        mtu: 1500
  ssh:
    keys:
reboot
Kernelparameter
ros config syslinux
tsc=reliable
sync; reboot
  • nach Reboot
cat /proc/cmdline
BOOT_IMAGE=../vmlinuz-4.14.85-rancher printk.devkmsg=on rancher.state.dev=LABEL=RANCHER_STATE rancher.state.wait panic=10 console=tty0 tsc=reliable  initrd=../initrd-v1.5.1

Rancher UI (Headnode)

  • Headnode einrichten
sudo docker run -d --restart=always -p 8080:8080 rancher/server
Unable to find image 'rancher/server:latest' locally
latest: Pulling from rancher/server
bae382666908: Pull complete
29ede3c02ff2: Pull complete
da4e69f33106: Pull complete
8d43e5f5d27f: Pull complete
b0de1abb17d6: Pull complete
422f47db4517: Pull complete
79d37de643ce: Pull complete
69d13e08a4fe: Pull complete
2ddfd3c6a2b7: Pull complete
bc433fed3823: Pull complete
b82e188df556: Pull complete
dae2802428a4: Pull complete
effdbd93afcb: Pull complete
a4fcc35085ad: Pull complete
e8234323b6c4: Pull complete
d3f751a5d9cc: Pull complete
d4b24e84b43b: Pull complete
da9d7264902d: Pull complete
df2b31306256: Pull complete
c2238fcf71c2: Pull complete
10c7c4a52421: Pull complete
Digest: sha256:290e94536b32665d0ff537c2b947804faeed2768cd8652f0088a0d7e1acced75
Status: Downloaded newer image for rancher/server:latest
482bd209a572ab19700ef3a83bf0338bd1582a0ffdc07861dbe5da89a2ad0ed7
docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                              NAMES
482bd209a572        rancher/server      "/usr/bin/entry /usr…"   16 seconds ago      Up 4 seconds        3306/tcp, 0.0.0.0:8080->8080/tcp   compassionate_elbakyan
  • lokale Authentifizierung eingestellt

C3d2 smartos kvm rancheros ui 1.jpg

Host Registration URL einrichten

C3d2 smartos kvm rancheros ui 2.jpg

neues Cattle Environment Template erstellen

  • mit VXLAN statt IPsec als Overlay Network Cross-Host Communication

C3d2 smartos kvm rancheros ui 3.jpg

neues Cattle Environment erstellen

C3d2 smartos kvm rancheros ui 4.jpg

  • Default Environment entfernen

C3d2 smartos kvm rancheros ui 5.jpg

C3d2 smartos kvm rancheros ui 6.jpg

Agent auf dem Headnode installieren

Wichtig
Es muss immer die -e CATTLE_AGENT_IP mit expliziter IP-Adresse angegeben werden, sonst bricht nach einer Weile die Verbindung zwischen den Compute-Nodes im Cluster!
ssh -p 2222 -i /home/daniel/.ssh/id_c3d2_ed25519 rancher@localhost
Enter passphrase for key '/home/daniel/.ssh/id_c3d2_ed25519':
sudo su
cd
docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                              NAMES
482bd209a572        rancher/server      "/usr/bin/entry /usr…"   2 hours ago         Up 2 hours          3306/tcp, 0.0.0.0:8080->8080/tcp   compassionate_elbakyan
sudo docker run --rm --privileged -e CATTLE_AGENT_IP=172.22.99.246 -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.2.11 http://172.22.99.246:8080/v1/scripts/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Unable to find image 'rancher/agent:v1.2.11' locally
v1.2.11: Pulling from rancher/agent
b3e1c725a85f: Pull complete
6a710864a9fc: Pull complete
d0ac3b234321: Pull complete
87f567b5cf58: Pull complete
063e24b217c4: Pull complete
d0a3f58caef0: Pull complete
16914729cfd3: Pull complete
bbad862633b9: Pull complete
3cf9849d7f3c: Pull complete
Digest: sha256:0fba3fb10108f7821596dc5ad4bfa30e93426d034cd3471f6ccd3afb5f87a963
Status: Downloaded newer image for rancher/agent:v1.2.11

INFO: Running Agent Registration Process, CATTLE_URL=http://172.22.99.246:8080/v1
INFO: Attempting to connect to: http://172.22.99.246:8080/v1
INFO: http://172.22.99.246:8080/v1 is accessible
INFO: Configured Host Registration URL info: CATTLE_URL=http://172.22.99.246:8080/v1 ENV_URL=http://172.22.99.246:8080/v1
INFO: Inspecting host capabilities
INFO: Boot2Docker: false
INFO: Host writable: true
INFO: Token: xxxxxxxx
INFO: Running registration
INFO: Printing Environment
INFO: ENV: CATTLE_ACCESS_KEY=XXXXXXXXXXXXXXXXXX
INFO: ENV: CATTLE_AGENT_IP=172.22.99.246
INFO: ENV: CATTLE_HOME=/var/lib/cattle
INFO: ENV: CATTLE_REGISTRATION_ACCESS_KEY=registrationToken
INFO: ENV: CATTLE_REGISTRATION_SECRET_KEY=xxxxxxx
INFO: ENV: CATTLE_SECRET_KEY=xxxxxxx
INFO: ENV: CATTLE_URL=http://172.22.99.246:8080/v1
INFO: ENV: DETECTED_CATTLE_AGENT_IP=172.17.0.1
INFO: ENV: RANCHER_AGENT_IMAGE=rancher/agent:v1.2.11
INFO: Launched Rancher Agent: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx
docker ps
CONTAINER ID        IMAGE                             COMMAND                  CREATED              STATUS              PORTS                              NAMES
9e96b29adb80        rancher/net:v0.11.9               "/rancher-entrypoint…"   29 seconds ago       Up 28 seconds                                          r-vxlan-vxlan-router-1-48b90d24
58f7c1f85962        rancher/dns:v0.17.4               "/rancher-entrypoint…"   38 seconds ago       Up 37 seconds                                          r-network-services-metadata-dns-1-5975f2cb
f5518d71078d        rancher/healthcheck:v0.3.8        "/.r/r /rancher-entr…"   41 seconds ago       Up 40 seconds                                          r-healthcheck-healthcheck-1-a5ec9f27
209afda61fa3        rancher/metadata:v0.10.4          "/rancher-entrypoint…"   46 seconds ago       Up 45 seconds                                          r-network-services-metadata-1-d49e63b7
49d91d5d2785        rancher/scheduler:v0.8.6          "/.r/r /rancher-entr…"   47 seconds ago       Up 46 seconds                                          r-scheduler-scheduler-1-3933ae84
b062bbf2beba        rancher/network-manager:v0.7.22   "/rancher-entrypoint…"   53 seconds ago       Up 52 seconds                                          r-network-services-network-manager-1-d78bd33c
c75e1c0d6c74        rancher/net:holder                "/.r/r /rancher-entr…"   55 seconds ago       Up 54 seconds                                          r-vxlan-vxlan-1-15cf7e5d
3304d69c3be2        rancher/net:v0.13.1               "/rancher-entrypoint…"   56 seconds ago       Up 56 seconds                                          r-vxlan-cni-driver-1-a1e2d7e3
596e621e7b45        rancher/agent:v1.2.11             "/run.sh run"            About a minute ago   Up About a minute                                      rancher-agent
482bd209a572        rancher/server                    "/usr/bin/entry /usr…"   2 hours ago          Up 2 hours          3306/tcp, 0.0.0.0:8080->8080/tcp   compassionate_elbakyan

C3d2 smartos kvm rancheros ui 7.jpg

Nachkorrekturen

Wichtig
den Nodes immer eine eindeutige Scheduler IP zuweisen!

C3d2 smartos kvm rancheros ui 8.jpg

Statusanzeige

C3d2 smartos kvm rancheros ui 9.jpg

Rancher (Compute) Nodes hinzufügen

sudo docker run --rm --privileged -e CATTLE_AGENT_IP=172.22.99.XXX -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.2.11 http://172.22.99.246:8080/v1/scripts/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Rancher - zusätzliche SSH Keys hinzufügen

cat new_ssh_keys.yml
ssh_authorized_keys:
- ssh-ed25519 XXXXXXXXX XXX
- ssh-rsa XXXXXXXXX XXX
ros config merge -i new_ssh_keys.yml
vi /var/lib/rancher/conf/cloud-config.yml
sync
reboot
[            ] reboot:info: Setting reboot timeout to 60 (rancher.shutdown_timeout set to 60)
.........^[            ] reboot:info: Setting reboot timeout to 60 (rancher.shutdown_timeout set to 60)
.=.[            ] reboot:info: Stopping /docker : 7e7d0702c70d
.....................N.....................[            ] reboot:info: Stopping /ntp : f3a62bbc731c
..?..[            ] reboot:info: Stopping /network : afb8d57014ff
..<..[            ] reboot:info: Stopping /udev : de4ef7e5ac94
..C..[            ] reboot:info: Stopping /system-cron : 59cc92c9b25b
...?...[            ] reboot:info: Stopping /syslog : facbd277afe9
..=..[            ] reboot:info: Stopping /acpid : fae77120e4dd
...J...[            ] reboot:info: Console Stopping [/console] : 3ec527245aaf
Connection to localhost closed by remote host.
Connection to localhost closed.

Rancher CLI (command-line interface)

Rancher API Key Zugriff

  • das passende Rancher CLI Package (Binary) lässt sich (rechts unten) über den Rancher UI Link downloaden
  • anschließend generiert man sich einen API Key und konfiguriert den Rancher CLI

C3d2 smartos kvm rancheros cli 1.jpg

lokales Volume erstellen

Es funktioniert genau wie angenommen:

  • Über den Rancher CLI erstellt man ein lokales Volume
  • dieses ist standardmäßig auf inactive gesetzt und wird auch nicht per Rancher UI angezeigt / aufgelistet, da hier nur Volumes unter dem Typ Rancher-NFS aufgelistet werden

(sofern dieses Plugin im Environment Template, beim initialen Cluster Bootstrap, eingebunden wurde)

  • die Verwaltung mittels Rancher CLI ist recht komfortabel

Rancher CLI

alle Volumes auflisten

./rancher volume -a

local Volume erstellen

./rancher volume create --driver local c3d2-data

Volume löschen

./rancher volume rm 1v389

per Rancher UI

docker-compose file mit volume mount

cd /gitlab/docker-compose/productive/local/alpine-linux
cat docker-compose.yaml
version: '2'

services:
  alpine:
    image: alpine
    stdin_open: true
    volumes:
    - c3d2-data:/c3d2-data
    volume_driver: local