These are slapped-together notes, nothing more. If time and ROI allow, they will be cleaned up.


Prototype relay board

One power input is to regular 230VAC / 50Hz house current, and this goes to four chasis-mounted power sockets each controlled by a separate DPDT-relay. A second power input is +5 VDC to supply current to the relays. The data input connects to the Net5501 via JP5 and control the transistors.

The transistors control the relays. The relays control the house current going to the installation machines.

Front and top views of the relay box.

Schematics for relay box. Designed and built by Mr. Jouko Alaniva, automation laboratory engineer.

Physical Connection

JP5 controls the transistors at +3.3 VDC, which in turn control the relays. The relays are powered from JP10 or JP15 at +5 VDC. The relay box is connected directly to the house current so the relays switch on and off 230 VAC. Each relay operates separately (See JP10 table below) and has a fuse rated at 2A.

In the picture below, the connection to the controller is a serial connection via USB to Serial adapter from an Asus EEE 701 running OpenBSD 4.4 and a customized FVWM.

Two Net 5501s
each with 2 x USB and 2 x Serial. One with 4 x 10/100 Mb/s Ethernet, the other with 6 x 10/100 Mb/s Ethernet.

Table 1a: JP10 DC Power
JP10, DC Power
Pin Function
1 GND
2 GND
3
4 +5V
5
6
Table 1b: JP15 DC Power
JP15, DC Power
Function
+12V
GND
GND
+5V

On the Net4801, JP10 is used only for powering the relays at +5 VDC, so we are not interested in the other pins.

Same for JP15 on the Net5501. JP15 is used there because JP10 is used to feed power to the motherboard. The regular power feed is blocked by the rack-mount casing.

 

The pins listed in Table 2 must be configured before use. The configuration can only occur while the system is running in securelevel 0. For this task, aside from the grounds, only pins 3, 5, 7 and 9 are used.

Table 2: User I/O pins used on JP5
JP5, User I/O
Pin Device Function
01   +3.3V
03 gpio1 16
05 gpio1 18
07 gpio1 20
09 gpio1 22
11   GND
13   ?
15 gpio1 11
17   GND
19   ?

The intent is to use a pair of these switches with the dual net5501 setup below. Each board will act as a network switch, using PF, so that one ethernet port connects to the laboratory LAN and all the others connect to individual installation machines.

An alternate method would be to use a board with only two ethernet ports and then have a cheap hub on the back end. The drawback there, is that the individual installation machines are not isolated from eachother.

Serial Connections

The user interface to the machines being installed is accessed via serial connections. These are provided by multi-head USB-to-serial adapters. These come in 2-, 4- and 8-port varieties. We have been using the 4-port, since with the Net4801's, it gives one Ethernet port, one serial port and one relay per installation machine, with one ethernet port left over for a connection to the network and as access point for the students.

dual port USB to serial adapter.

2-port USB-to-Serial dmesg excerpt:

uhub1 at uhub0 port 1 "Genesys Logic USB2.0 Hub" rev 2.00/7.02 addr
2
uplcom0 at uhub1 port 2 "Prolific Technology Inc. USB-Serial
Controller" rev 1.10/3.00 addr 3
ucom0 at uplcom0
uplcom1 at uhub1 port 3 "Prolific Technology Inc. USB-Serial
Controller" rev 1.10/3.00 addr 4

quad port USB to serial adapter.

4-port USB-to-Serial dmesg excerpt:

uplcom0 at uhub1 port 1 "Prolific Technology Inc. USB-Serial
Controller" rev 1.10/3.00 addr 3
ucom0 at uplcom0
uplcom1 at uhub1 port 2 "Prolific Technology Inc. USB-Serial
Controller" rev 1.10/3.00 addr 4
ucom1 at uplcom1
uplcom2 at uhub1 port 3 "Prolific Technology Inc. USB-Serial
Controller" rev 1.10/3.00 addr 5
ucom2 at uplcom2
uplcom3 at uhub1 port 4 "Prolific Technology Inc. USB-Serial
Controller" rev 1.10/3.00 addr 6
ucom3 at uplcom3

The choice of the USB-to-Serial adapters was almost random were. These were selected because they are the first ones I happened to get hold of. These use the Prolific chipset, but FTDI might also work and appears to be more readily available in the four-port model.

I stuck with the Prolific chipset for the second order, where the four-port adapters were purchased because payment and receiving at the polytechnic is such a collosal nightmare that I would lose probably 24 m-hr if returns were needed.

Logical control

For the pins to be available during regular system operation, they must first be configured before the system leaves secure level 0. That can be done in the shell script /etc/rc.secure.

That is done to minimize the risk of accidental (or intentional) damage from a pin changing modes between input direction, output direction, bi-directional, open-drain output, push-pull output, tri-state (output disabled), internal pull-up enabled, internal pull-down enabled, invert input, or invert output. Not all hardware will tolerate all modes, so it is necessary to confine the possiblity during operation to acceptable modes.

Note: gpioctlchanges significantly between OpenBSD 4.4 and 4.5.

/etc/rc.secure:

/usr/sbin/gpioctl gpio1 16 set out pp;  # JP5 pin 03
/usr/sbin/gpioctl gpio1 18 set out pp;  # JP5 pin 05
/usr/sbin/gpioctl gpio1 20 set out pp;  # JP5 pin 07
/usr/sbin/gpioctl gpio1 22 set out pp;  # JP5 pin 09

/usr/sbin/gpioctl gpio1 11 set out pp;  # JP5 pin 15

In this case, the four pins in use will use output direction, meaning that when they are turned on they will be at +3.3 VDC.

So, to toggle pin 9 on and off, gpioctl would take the following options:

Turn pin 09 on if it is off, and off it is already on:

gpioctl gpio1 21 2;

BIOS settings

BIOS are a largely antiquated firmware application used for, among other things, booting x86 systems. The legacy has arrived to us from the old days. It needs to be mentioned here since it is involved in booting on x86 hardware. For such legacy devices, the BIOS need to be set to allow PXE booting. Some other setting related to power control may also be relevant, depending on the model of hardware.

The following three settings were needed in for the old Dell Optiplex 270s used for this test. The two key BIOS settings are that the machine automatically turns itself on after power is restored and that PXE Boot is the first method tried on startup.

Open Firmware is, in contrast, an IEEE standard and used on non-x86 hardware.

Installing to the target machines

All of the above is used to turn on and off computers connected via serial and ethernet to the controller. Students log into the controller and can then power-cycle their machine to begin installation via PXE. DHCPd is configured to allow PXE-booting of system installers. The system installers are configured to use serial consoles for interaction with the student. USB-to-serial adapters are used to allow one controller to connect to many installation machines. Ethernet is used for the network installations.

In preparation for network installation, at least three services must be prepared: dhcpd, inetd, and PF. The actual files served and the specific configurations are dependent on which operating system is to be installed.

Once the ability to do a network install is in place, the next steps are to connect to the controller and use it to work with the other machines. A key here is that it can all take place remotely, with a distance of a few centimeters or a few thousand kilometers between the student and their server.

Key Packages:

pfctl      userland control of the packet filter
(PF) device
cu         call up another system and act as a
dial-in terminal
pftop      curses-based real time state and rule
display for pf
dhcpd      The Internet Software Consortium DHCP
Server
inetd      allow one daemon to invoke others,
reducing system load
tcpdump    print the headers of packets on a
network interface
dmassage   dmesg parser, tune the kernel
nmap       scan ports and fingerprint stack of
network hosts
p0f        passive OS fingerprinting tool
rsync      mirroring/synchronization over low
bandwidth links
scanssh    SSH remote version scanner
screen     multi-screen window manager
snort      highly flexible sniffer/NIDS
squid      proxy caching to minimize net traffic
wol        Wake On Lan client

Netboot

There are a number of server systems available. The ones with checkmarks are the ones we have used successfully with this set up. The others might work, but haven't been tried yet. The prerequisites are that the system 1) allow network installation, 2) allow serial console interface, 3) be FOSS.

Fedora

The boot option methodmust be appended to the kernel in isolinux.cfg. The boot options are listed in http://docs.fedoraproject.org/install-guide/f10/en_US/ap-admin-options.html.

e.g.
append method=

References

Note that other hardware could be used, these are just the ones that happened to be used here, this time.

Relevant hardware manuals:

Relevant manual pages:


PXE Jukebox

Note: apologies in advance for still using IPv4 for this.

Students connect via OpenSSHto a controlling gateway and from there carry out their installations. OpenSSH is a widely used, perhaps the most widely used, method of secure interactive connections. Guidance and supervision can be carried out through tunnelling krfbor other remote viewer from their desktop. krfbis available on many systems and on the systems where it is unavailable, it is interoperable with any other application supporting the Remote Frame Buffer protocol.

Working with server operating systems is the natural choice here since they do not use graphical components or require even a graphics card. That allows work to be done over a serial connection from the gateway, which, when combined with the ability to turn the machine on and off, gives as much access as would be available anyway even sitting physically beside the machine.

Components of the PXE Jukebox

For reasons of convenience, simplicity, cleaness of implementation, plus other factors, OpenBSD is used as the platform for the jukebox. A larger runner-up would be Debian.

There are four components.

DHCPd: Pointing to the installer

OpenBSD's implementation of the ISC's DHCPd is used to point to the modified second stage bootloader used to complete the netboot. Installation of OpenBSD uses pxebootand the Linuxes use pxelinux.0.

excerpt from dhcpd configuration to install openbsd:

        subnet 10.2.200.0 netmask 255.255.255.128 {
                option routers 10.2.200.1;
                filename "pxeboot";
                range 10.2.200.32 10.2.200.63;
        }

Linux is almost the same, only the second stage bootloader has a different name.

excerpt from dhcpd configuration to install the linuxes:

        subnet 10.2.200.0 netmask 255.255.255.128 {
                option routers 10.2.200.1;
                filename "pxelinux.0";
                range 10.2.200.32 10.2.200.63;
        }

tftpd, inetd: Transferring the installer

tftp, a rudimentery file transfer protocol, is used to serve the installation media. inetdis used to provide on-demand tftp service rather than leaving the tftp server running all the time. The full path to the directory containing the installation media must be given, this is obviously different for each system.

excerpt from inet configuration to serve tftp on-demand:

tftp            dgram   udp     wait    root    /usr/libexec/tftpd 
    tftpd -l -s /var/pxebooters/obsd-4.4-i386/netboot 
tftp            dgram   udp6    wait    root    /usr/libexec/tftpd 
    tftpd -l -s /var/pxebooters/obsd-4.4-i386/netboot

FTP: Transfer of packages and other installation data

It appears simplest to me to just reset the home directory for the user ftp than to try to move files around.

PF: filtering and routing

We are still using that sad and embarassing hack, NAT, and that outdated networking protocol IPv4.

excerpts from one version of pf.conf:

pass in on $ext_if proto tcp to ($ext_if) port ftp
pass in on $ext_if proto udp to ($ext_if) port { bootps, bootpc,
tftp }

Installation via Serial Console

The installation menu must be told which console settings to use. For some versions and distros this is all handled inside pxelinux.cfg/default. In others, defaultis used to set the installer menu, then one of the .cfg file in boot-screens must have the console settings appended to the kernel parameters. grep for "append" among files ending in .cfg

The Linuxes

excerpt from default or one of the .cfg files:

SERIAL 0 19200 0

The installation itself must continue on the console, so the console settings must be appended to the kernel boot parameters.

excerpt from default or one of the .cfg files:

append 
. . . console=ttyS0,19200n8

OpenBSD

boot.conf:

set image /bsd.rd
set tty com0
stty com0 19200

Backup / Restore / Clone

Partition a second CF unit for FFS and install OpenBSD, using onlythe set bsd.rdso that there is something to boot from if needed. Then dump- restoredata from one chip to the other.

backup / copy CF chip:

mount /dev/sd1a /mnt
time dump -f - / | (cd /mnt/; restore -rf -)

mount /dev/sd1d /mnt/data
time dump -f - /data/ | (cd /mnt/data/; restore -rf -)

Full backup takes about 46 minutes for the root directory


Gateway

A

Shell Access

OpenSSH provides remote login capabilities. The work accounts are chrooted by OpenSSH itself.

excerpt from sshd_config:

Match Group installers
        X11Forwarding no
        AllowTcpForwarding no
        # PasswordAuthentication no
        ChrootDirectory /data/chroot-relay
        Banner /etc/maskin.banner.txt

Scripts

There is one set of scripts per target machine.

group membership:

$ groups maskin1
maskin1 dialer installers

The scripts are owned by root, the ability to execute them is determined by group membership.

scripts:

usr/local/bin/:
total 32
-rwxr-xr--  1 root  maskin0  171 Mar  9 10:17 install0
-rwxr-xr--  1 root  maskin1  171 Mar  9 10:17 install1
-rwxr-xr--  1 root  maskin2  171 Mar  9 10:17 install2
-rwxr-xr--  1 root  maskin3  171 Mar  9 10:17 install3
-rwxr-xr--  1 root  maskin0  284 Mar  9 10:17 maskin0
-rwxr-xr--  1 root  maskin1  284 Mar  9 10:17 maskin1
-rwxr-xr--  1 root  maskin2  284 Mar  9 10:17 maskin2
-rwxr-xr--  1 root  maskin3  284 Mar  9 10:17 maskin3

The devices inside the chroot are created by MAKDEV(8). The special device stdmakes the devices which are absolutely necessary. Additional devices are tty00through tty03and ttyU0through ttyU3.

devices:

crw-rw----  1 root  installers   83,   1 Mar 11 21:36 gpio1
$ maskin1
pin 18: state 1
$ maskin1 on
pin 18: state 1 -> 1
$ maskin1 off
pin 18: state 1 -> 0

SSH VPN

Additional tun devices can be added using MAKEDEVfound in /dev/. e.g. cd /dev &&./MAKEDEV tun4. Apologies that this is 2009 and we are still using IPv4.

               +--------------------------------+
<--------+-----+ 10.2.200.127                   |
               |                                |
 10.1.0.2 +----+ 10.1.0.1 --- +--+ 10.100.100.1 +--+ 10.100.100.32 
               |                                |
 10.1.1.2 +----+ 10.1.1.1 --- +--+ 10.101.101.1 +--+ 10.101.101.32 
               |                                |
 10.1.2.2 +----+ 10.1.2.1 --- +--+ 10.102.102.1 +--+ 10.102.102.32 
               |                                |
 10.1.3.2 +----+ 10.1.3.1 --- +--+ 10.103.103.1 +--+ 10.103.103.32 
               +--------------------------------+

tun devices:

$ ls -l /dev/tun?
crw-rw----  1 root  installers   40,   0 Apr 11 14:50 /dev/tun0
crw-rw----  1 root  installers   40,   1 Apr 11 14:50 /dev/tun1
crw-rw----  1 root  installers   40,   2 Apr 11 14:50 /dev/tun2
crw-rw----  1 root  installers   40,   3 Apr 11 14:50 /dev/tun3

Server side

ifconfig tun1 10.1.0.2 10.1.0.1 netmask 255.255.255.252
ifconfig tun1 10.1.1.2 10.1.1.1 netmask 255.255.255.252
ifconfig tun1 10.1.2.2 10.1.2.1 netmask 255.255.255.252
ifconfig tun1 10.1.3.2 10.1.3.1 netmask 255.255.255.252
# route add 10.2.200.0/24 10.1.1.1

Client side, openbsd

Client side, linux

 sudo ifconfig tun1 10.0.0.200 pointopoint 10.0.0.100 
 sudo route add -net 10.101.101.1 netmask 255.255.255.0 gw
10.0.0.200 tun1  

Sun Mar 8 13:11:37 EET 2009