These are slapped-together notes, nothing more. If time and ROI allow, they will be cleaned up.
One power input is to regular 230VAC / 50Hz house current, and this goes to four chasis-mounted power sockets each controlled by a separate DPDT-relay. A second power input is +5 VDC to supply current to the relays. The data input connects to the Net5501 via JP5 and control the transistors.
The transistors control the relays. The relays control the house current going to the installation machines.
JP5 controls the transistors at +3.3 VDC, which in turn control the relays. The relays are powered from JP10 or JP15 at +5 VDC. The relay box is connected directly to the house current so the relays switch on and off 230 VAC. Each relay operates separately (See JP10 table below) and has a fuse rated at 2A.
In the picture below, the connection to the controller is a serial connection via USB to Serial adapter from an Asus EEE 701 running OpenBSD 4.4 and a customized FVWM.
Two Net 5501s
each with 2 x USB and 2 x Serial. One with 4 x 10/100 Mb/s
Ethernet, the other with 6 x 10/100 Mb/s Ethernet.
|
|
||||||||||||||||||||||
On the Net4801, JP10 is used only for powering the relays at +5 VDC, so we are not interested in the other pins.
Same for JP15 on the Net5501. JP15 is used there because JP10 is used to feed power to the motherboard. The regular power feed is blocked by the rack-mount casing.
The pins listed in Table 2 must be configured before use. The configuration can only occur while the system is running in securelevel 0. For this task, aside from the grounds, only pins 3, 5, 7 and 9 are used.
| JP5, User I/O | ||
|---|---|---|
| Pin | Device | Function |
| 01 | +3.3V | |
| 03 | gpio1 | 16 |
| 05 | gpio1 | 18 |
| 07 | gpio1 | 20 |
| 09 | gpio1 | 22 |
| 11 | GND | |
| 13 | ? | |
| 15 | gpio1 | 11 |
| 17 | GND | |
| 19 | ? | |
The intent is to use a pair of these switches with the dual net5501 setup below. Each board will act as a network switch, using PF, so that one ethernet port connects to the laboratory LAN and all the others connect to individual installation machines.
An alternate method would be to use a board with only two ethernet ports and then have a cheap hub on the back end. The drawback there, is that the individual installation machines are not isolated from eachother.
The user interface to the machines being installed is accessed via serial connections. These are provided by multi-head USB-to-serial adapters. These come in 2-, 4- and 8-port varieties. We have been using the 4-port, since with the Net4801's, it gives one Ethernet port, one serial port and one relay per installation machine, with one ethernet port left over for a connection to the network and as access point for the students.
dual port USB to serial adapter.
uhub1 at uhub0 port 1 "Genesys Logic USB2.0 Hub" rev 2.00/7.02 addr 2 uplcom0 at uhub1 port 2 "Prolific Technology Inc. USB-Serial Controller" rev 1.10/3.00 addr 3 ucom0 at uplcom0 uplcom1 at uhub1 port 3 "Prolific Technology Inc. USB-Serial Controller" rev 1.10/3.00 addr 4
quad port USB to serial adapter.
uplcom0 at uhub1 port 1 "Prolific Technology Inc. USB-Serial Controller" rev 1.10/3.00 addr 3 ucom0 at uplcom0 uplcom1 at uhub1 port 2 "Prolific Technology Inc. USB-Serial Controller" rev 1.10/3.00 addr 4 ucom1 at uplcom1 uplcom2 at uhub1 port 3 "Prolific Technology Inc. USB-Serial Controller" rev 1.10/3.00 addr 5 ucom2 at uplcom2 uplcom3 at uhub1 port 4 "Prolific Technology Inc. USB-Serial Controller" rev 1.10/3.00 addr 6 ucom3 at uplcom3
The choice of the USB-to-Serial adapters was almost random were. These were selected because they are the first ones I happened to get hold of. These use the Prolific chipset, but FTDI might also work and appears to be more readily available in the four-port model.
I stuck with the Prolific chipset for the second order, where the four-port adapters were purchased because payment and receiving at the polytechnic is such a collosal nightmare that I would lose probably 24 m-hr if returns were needed.
For the pins to be available during regular system operation, they must first be configured before the system leaves secure level 0. That can be done in the shell script /etc/rc.secure.
That is done to minimize the risk of accidental (or intentional) damage from a pin changing modes between input direction, output direction, bi-directional, open-drain output, push-pull output, tri-state (output disabled), internal pull-up enabled, internal pull-down enabled, invert input, or invert output. Not all hardware will tolerate all modes, so it is necessary to confine the possiblity during operation to acceptable modes.
Note:
gpioctlchanges significantly between OpenBSD 4.4 and
4.5.
/usr/sbin/gpioctl gpio1 16 set out pp; # JP5 pin 03 /usr/sbin/gpioctl gpio1 18 set out pp; # JP5 pin 05 /usr/sbin/gpioctl gpio1 20 set out pp; # JP5 pin 07 /usr/sbin/gpioctl gpio1 22 set out pp; # JP5 pin 09 /usr/sbin/gpioctl gpio1 11 set out pp; # JP5 pin 15
In this case, the four pins in use will use output direction, meaning that when they are turned on they will be at +3.3 VDC.
So, to toggle pin 9 on and off, gpioctl would take the following options:
gpioctl gpio1 21 2;
BIOS are a largely antiquated firmware application used for, among other things, booting x86 systems. The legacy has arrived to us from the old days. It needs to be mentioned here since it is involved in booting on x86 hardware. For such legacy devices, the BIOS need to be set to allow PXE booting. Some other setting related to power control may also be relevant, depending on the model of hardware.
The following three settings were needed in for the old Dell Optiplex 270s used for this test. The two key BIOS settings are that the machine automatically turns itself on after power is restored and that PXE Boot is the first method tried on startup.
Open Firmware is, in contrast, an IEEE standard and used on non-x86 hardware.
All of the above is used to turn on and off computers connected via serial and ethernet to the controller. Students log into the controller and can then power-cycle their machine to begin installation via PXE. DHCPd is configured to allow PXE-booting of system installers. The system installers are configured to use serial consoles for interaction with the student. USB-to-serial adapters are used to allow one controller to connect to many installation machines. Ethernet is used for the network installations.
In preparation for network installation, at least three services must be prepared: dhcpd, inetd, and PF. The actual files served and the specific configurations are dependent on which operating system is to be installed.
Once the ability to do a network install is in place, the next steps are to connect to the controller and use it to work with the other machines. A key here is that it can all take place remotely, with a distance of a few centimeters or a few thousand kilometers between the student and their server.
pfctl userland control of the packet filter (PF) device cu call up another system and act as a dial-in terminal pftop curses-based real time state and rule display for pf dhcpd The Internet Software Consortium DHCP Server inetd allow one daemon to invoke others, reducing system load tcpdump print the headers of packets on a network interface dmassage dmesg parser, tune the kernel nmap scan ports and fingerprint stack of network hosts p0f passive OS fingerprinting tool rsync mirroring/synchronization over low bandwidth links scanssh SSH remote version scanner screen multi-screen window manager snort highly flexible sniffer/NIDS squid proxy caching to minimize net traffic wol Wake On Lan client
There are a number of server systems available. The ones with checkmarks are the ones we have used successfully with this set up. The others might work, but haven't been tried yet. The prerequisites are that the system 1) allow network installation, 2) allow serial console interface, 3) be FOSS.
The boot option
methodmust be appended to the kernel in
isolinux.cfg. The boot options are listed in
http://docs.fedoraproject.org/install-guide/f10/en_US/ap-admin-options.html.
e.g.
append method=
Note that other hardware could be used, these are just the ones that happened to be used here, this time.
Relevant hardware manuals:
Relevant manual pages:
Note: apologies in advance for still using IPv4 for this.
Students connect via OpenSSHto a controlling gateway and from there carry out their installations. OpenSSH is a widely used, perhaps the most widely used, method of secure interactive connections. Guidance and supervision can be carried out through tunnelling krfbor other remote viewer from their desktop. krfbis available on many systems and on the systems where it is unavailable, it is interoperable with any other application supporting the Remote Frame Buffer protocol.
Working with server operating systems is the natural choice here since they do not use graphical components or require even a graphics card. That allows work to be done over a serial connection from the gateway, which, when combined with the ability to turn the machine on and off, gives as much access as would be available anyway even sitting physically beside the machine.
For reasons of convenience, simplicity, cleaness of implementation, plus other factors, OpenBSD is used as the platform for the jukebox. A larger runner-up would be Debian.
There are four components.
OpenBSD's implementation of the ISC's DHCPd is used to point to the modified second stage bootloader used to complete the netboot. Installation of OpenBSD uses pxebootand the Linuxes use pxelinux.0.
subnet 10.2.200.0 netmask 255.255.255.128 {
option routers 10.2.200.1;
filename "pxeboot";
range 10.2.200.32 10.2.200.63;
}
Linux is almost the same, only the second stage bootloader has a different name.
subnet 10.2.200.0 netmask 255.255.255.128 {
option routers 10.2.200.1;
filename "pxelinux.0";
range 10.2.200.32 10.2.200.63;
}
tftp, a rudimentery file transfer protocol, is used to serve the installation media. inetdis used to provide on-demand tftp service rather than leaving the tftp server running all the time. The full path to the directory containing the installation media must be given, this is obviously different for each system.
tftp dgram udp wait root /usr/libexec/tftpd
tftpd -l -s /var/pxebooters/obsd-4.4-i386/netboot
tftp dgram udp6 wait root /usr/libexec/tftpd
tftpd -l -s /var/pxebooters/obsd-4.4-i386/netboot
It appears simplest to me to just reset the home directory for the user ftp than to try to move files around.
We are still using that sad and embarassing hack, NAT, and that outdated networking protocol IPv4.
pass in on $ext_if proto tcp to ($ext_if) port ftp
pass in on $ext_if proto udp to ($ext_if) port { bootps, bootpc,
tftp }
The installation menu must be told which console settings to use. For some versions and distros this is all handled inside pxelinux.cfg/default. In others, defaultis used to set the installer menu, then one of the .cfg file in boot-screens must have the console settings appended to the kernel parameters. grep for "append" among files ending in .cfg
SERIAL 0 19200 0
The installation itself must continue on the console, so the console settings must be appended to the kernel boot parameters.
append . . . console=ttyS0,19200n8
set image /bsd.rd set tty com0 stty com0 19200
Partition a second CF unit for FFS and install OpenBSD, using
onlythe set
bsd.rdso that there is something to boot from if
needed. Then
dump-
restoredata from one chip to the other.
mount /dev/sd1a /mnt time dump -f - / | (cd /mnt/; restore -rf -) mount /dev/sd1d /mnt/data time dump -f - /data/ | (cd /mnt/data/; restore -rf -)
Full backup takes about 46 minutes for the root directory
A
OpenSSH provides remote login capabilities. The work accounts are chrooted by OpenSSH itself.
Match Group installers
X11Forwarding no
AllowTcpForwarding no
# PasswordAuthentication no
ChrootDirectory /data/chroot-relay
Banner /etc/maskin.banner.txt
There is one set of scripts per target machine.
$ groups maskin1 maskin1 dialer installers
The scripts are owned by root, the ability to execute them is determined by group membership.
usr/local/bin/: total 32 -rwxr-xr-- 1 root maskin0 171 Mar 9 10:17 install0 -rwxr-xr-- 1 root maskin1 171 Mar 9 10:17 install1 -rwxr-xr-- 1 root maskin2 171 Mar 9 10:17 install2 -rwxr-xr-- 1 root maskin3 171 Mar 9 10:17 install3 -rwxr-xr-- 1 root maskin0 284 Mar 9 10:17 maskin0 -rwxr-xr-- 1 root maskin1 284 Mar 9 10:17 maskin1 -rwxr-xr-- 1 root maskin2 284 Mar 9 10:17 maskin2 -rwxr-xr-- 1 root maskin3 284 Mar 9 10:17 maskin3
The devices inside the chroot are created by MAKDEV(8). The special device stdmakes the devices which are absolutely necessary. Additional devices are tty00through tty03and ttyU0through ttyU3.
crw-rw---- 1 root installers 83, 1 Mar 11 21:36 gpio1
$ maskin1 pin 18: state 1 $ maskin1 on pin 18: state 1 -> 1 $ maskin1 off pin 18: state 1 -> 0
Additional tun devices can be added using
MAKEDEVfound in /dev/. e.g.
cd /dev &&./MAKEDEV tun4. Apologies that this
is 2009 and we are still using IPv4.
+--------------------------------+
<--------+-----+ 10.2.200.127 |
| |
10.1.0.2 +----+ 10.1.0.1 --- +--+ 10.100.100.1 +--+ 10.100.100.32
| |
10.1.1.2 +----+ 10.1.1.1 --- +--+ 10.101.101.1 +--+ 10.101.101.32
| |
10.1.2.2 +----+ 10.1.2.1 --- +--+ 10.102.102.1 +--+ 10.102.102.32
| |
10.1.3.2 +----+ 10.1.3.1 --- +--+ 10.103.103.1 +--+ 10.103.103.32
+--------------------------------+
$ ls -l /dev/tun? crw-rw---- 1 root installers 40, 0 Apr 11 14:50 /dev/tun0 crw-rw---- 1 root installers 40, 1 Apr 11 14:50 /dev/tun1 crw-rw---- 1 root installers 40, 2 Apr 11 14:50 /dev/tun2 crw-rw---- 1 root installers 40, 3 Apr 11 14:50 /dev/tun3
ifconfig tun1 10.1.0.2 10.1.0.1 netmask 255.255.255.252 ifconfig tun1 10.1.1.2 10.1.1.1 netmask 255.255.255.252 ifconfig tun1 10.1.2.2 10.1.2.1 netmask 255.255.255.252 ifconfig tun1 10.1.3.2 10.1.3.1 netmask 255.255.255.252 # route add 10.2.200.0/24 10.1.1.1
sudo ifconfig tun1 10.0.0.200 pointopoint 10.0.0.100 sudo route add -net 10.101.101.1 netmask 255.255.255.0 gw 10.0.0.200 tun1
Sun Mar 8 13:11:37 EET 2009