PVM over firewalls and the Internet

06.06.2007 23:06

PVM was designed to connect different machines into a cluster over low-latency high-speed LANs. Some applications however do not need good connectivity between cluster nodes in order to work efficiently.

Now imagine you have a four-way Opteron machine at your disposal and the only thing that prevents you from adding it to your virtual machine is the fact that it is 30 km away and the only route to it leads through two firewalls and a block of public Internet.

PVM uses a combination of TCP and UDP connections for communication between nodes so you can't use SSH port forwarding. However there is a way to tunnel any kind of traffic over SSH and make a simple virtual private network using a combination of SSH and PPP daemons (see VPN HOWTO) which will also support PVM traffic.

The complete installation described in the HOW-TO is too complicated for one time use. So here are simple step-by-step instructions:

  1. Make sure you can make an SSH connection from master to opteron with a single command and without entering a password (e.g. by using RSA or DSA authentication). To traverse firewalls you can use SSH port forwarding for this connection (since you only need one standard TCP connection from master to opteron to achieve this - in my case it was SSH-over-SSH-over-SSH)
  2. Download and install pty-redir on master. It's only 100 lines of code, so it's easy to audit if you're worried about security.
  3. Install pppd on both master (the local host that runs the pvm console and controls the cluster) and opteron (the remote host you want to add to the cluster). Pray that both machines have ppp support enabled in kernel.
  4. Put the following into /etc/ppp/options on master:
  5. noauth
    
  6. Put the following into /etc/ppp/options on opteron:
  7. ipcp-accept-local
    ipcp-accept-remote
    proxyarp
    noauth
    
  8. Run ./pty-redir /usr/bin/ssh -C -c blowfish -t -e none -o 'Batchmode yes' -i path-to-your-rsa-key opteron-host-name /usr/sbin/pppd on master.
  9. Run pppd /dev/ttyp0 192.168.X.1:192.168.X.2 on master (replace /dev/ttyp0 with the device name that is printed by the previous command) Choose X so that these two IPs do not conflict with any other networks master and opteron are connected to. 192.168.X.1 becomes master's IP and 192.168.X.2 becomes opteron's IP in this little VPN.
  10. Write a PVM hostfile like this:
  11. 192.168.X.1 sp=1000
    192.168.X.2 sp=10000
    ...and any other local machines in your cluster
    
  12. Start PVM on master: pvm -n192.168.X.1 hostfile
  13. Enjoy :)
Posted by Tomaž | Categories: Code

Add a new comment


(No HTML tags allowed. Separate paragraphs with a blank line.)