PROXMOX on diskless nodes with PXE boot and iSCSI target.

   

     The project task is to demonstrate the software installation process of PROXMOX. Working nodes do not use the hard drive and loaded from the boot server. The main task is to install Debian to boot from iSCSI target. It is not basic functionality, because Debian distributive is not support new installation to iSCSI target by default. You may recompile a kernel to insert special module with support iSCSI or use HELPER PC to install system files from bootstrapt. In this example we prepare configuration for 10 nodes with Proxmox v4 based on Debian jessie. In my lab i use very old PC, it  is only for example.

     For prepearing we need (from fig.):

Boot serv - CentOS, DHCP server, iSCSI target (tgtd), tftp server, may be NTP server.

=========== On server =============================
Create empty image files with size 10G (h1.dat)

dd if=/dev/zero of=/Data/h1.dat bs=$(( 1024 * 1024 )) count=10000

Create target and lun for each image file.

tgtadm --lld iscsi --op new --mode target --tid 1 -T iqn.h1:cluster
tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 1 -b /Data/h1.dat
tgtadm --lld iscsi --op bind --mode target --tid 1 -I ALL

Show access rights, params of target and luns.

tgtadm --lld iscsi --mode target --op show

For PXE boot generate pxe files.

====EmbeddedScript====
============http://rom-o-matic.net/gpxe/gpxe-1.0.1/contrib/rom-o-matic/build.php=======
dhcp net0
set keep-san 1
sanboot iscsi:10.230.0.1:::1:iqn.h1:cluster

Save this file as g1.pxe and put it to tftp root directory

Configure DHCP server to provide params to boot with .pxe file.

[root@boot dhcp]# cat /etc/dhcp/dhcpd.conf
ddns-update-style none;
authoritative;
log-facility local7;
subnet 10.230.0.0 netmask 255.255.0.0 {
  range 10.230.0.100 10.230.0.200;
  option domain-name-servers 8.8.8.8;
  option domain-name "game.org";
  option routers 10.230.0.1;
  default-lease-time 600;
  max-lease-time 7200;
  next-server 10.230.0.1;
}
host g1
    {
    hardware ethernet 00:17:31:4F:EE:EE;
    fixed-address 10.230.0.11;
    filename "g1.pxe";
    }
host g2
    {
    hardware ethernet 00:17:31:4F:EE:E6;
    fixed-address 10.230.0.12;
    filename "g2.pxe";
    }

From HELPER PC (with preinstalled Debian jessie and bootstrap) mount iSCSI target from Boot serv.

apt-get install debootstrap
============ On helper =============================
iscsiadm -m discovery -t sendtargets -p 10.230.0.1:3260
iscsiadm --mode node --targetname iqn.h1:cluster -p 10.230.0.1 --login

Check with dmesg or in /dev for new drive attached. In example we are looking for /dev/sdb
Then create a partition and format it.

fdisk /dev/sdb
---------------------------------- create 
---------------------------------- g   create a new empty GPT partition table
---------------------------------- n   add a new partition from 2048 to +9G
---------------------------------- create swap
---------------------------------- t   change a partition type #2 to type=14 (swap)
---------------------------------- w save
Device        Start      End  Sectors  Size Type
/dev/sdb1      2048 18876415 18874368    9G Linux filesystem
/dev/sdb2  18876416 20479966  1603551  783M Linux swap

mkfs.ext4 /dev/sdb1
mkswap /dev/sdb2


Mount this new iSCSI drive to /mnt/chroot and copy structure of system from bootstrap.

mkdir /mnt/chroot
mount /dev/sdb1 /mnt/chroot
debootstrap jessie /mnt/chroot
mount -t proc none /mnt/chroot/proc
mount -t sysfs none /mnt/chroot/sys
mount --bind /dev /mnt/chroot/dev

Change a root directory to premounted drive.

chroot /mnt/chroot /bin/bash
cp /proc/mounts /etc/mtab
sed -i '\|^/dev/sdb1|,$!d' /etc/mtab
blkid /dev/sdb1 /dev/sdb2
--------/dev/sdb1: UUID="bb90b8e8-e4cb-4d7f-9449-6c0c90c2bab7" TYPE="ext4" PARTUUID="98475002-3132-41b3-a6f4-698734d8a348"
--------/dev/sdb2: UUID="a1f53c9c-ba2a-4e86-99a0-ab33829cca84" TYPE="swap" PARTUUID="b7a4a696-52e4-4467-8661-db9bbf7e5bd8"

Insert new UUID to /etc/fstab

echo 'UUID=bb90b8e8-e4cb-4d7f-9449-6c0c90c2bab7 / ext4 errors=remount-ro 0 1' >> /etc/fstab
echo 'UUID=a1f53c9c-ba2a-4e86-99a0-ab33829cca84 none swap sw 0 0' >> /etc/fstab

apt-get install openssh-server locales
apt-get install linux-image-amd64 grub2 initramfs-tools
nano /etc/ssh/sshd_config

----change -----
FROM:PermitRootLogin without-password
TO:     PermitRootLogin yes

Configure grub to boot from iSCSI

================iSCSI boot configuration=========================
nano /etc/default/grub
---add ---- 
GRUB_CMDLINE_LINUX="ISCSI_INITIATOR=iqn.h1:client ISCSI_TARGET_NAME=iqn.h1:cluster ISCSI_TARGET_IP=10.230.0.1 root=UUID=e0da755a-e541-416b-b757-6f1d9e8fb075"
mkdir /etc/iscsi
touch /etc/iscsi/iscsi.initramfs
echo "InitiatorName=iqn.h1:client" > /etc/iscsi/initiatorname.iscsi
update-grub
update-initramfs -u
passwd

Configure network

nano /etc/network/interfaces
-- add------ 
auto lo
iface lo inet loopback
auto eth0 
allow-hotplug eth0 
iface eth0 inet dhcp
--------------------------------------------
nano /etc/hostname
---------add-----------
h1
-----------------------

nano /etc/hosts
------------add------------
127.0.0.1       localhost.localdomain localhost
10.230.0.12     h1.cluster h1 pvelocalhost
::1             localhost ip6-localhost ip6-loopback
ff02::1         ip6-allnodes
ff02::2         ip6-allrouters
-----------------------------------------------------------
exit
umount /mnt/chroot/{dev,proc,sys,}
iscsiadm -m node -T 'iqn.h1:cluster' -p 10.230.0.1 -u

Then reboot a system. It must be boot from iSCSI target and contain only Debian. Install a Proxmox from repository.

==============Install Proxmox VE===================
echo "deb http://download.proxmox.com/debian jessie pve-no-subscription" > /etc/apt/sources.list.d/pve-install-repo.list
wget -O- "http://download.proxmox.com/debian/key.asc" | apt-key add -
apt-get update && apt-get dist-upgrade
reboot
apt-get install proxmox-ve ntp ssh postfix ksm-control-daemon open-iscsi

reboot

And try to connect to Proxmox node.

---------------- connect to https://IP_ADDR:8006--------------------------