Unimore PLE/Docs
<accesscontrol>luser</accesscontrol>
How to add a slice to nodes
After creating a slice, you need to add it to the remote nodes, so that you can access remote machines authenticating with slice name and the SSH public key sent to the PlanetLab web site.
You can choose the nodes you want from the Web site, using an intuitive interface, but this is useful for few nodes only: if you need hundreds of nodes, you can use some Python commands for accessing the PlanetLab API and using some of its commands. If you find references to a "PlanetLab Shell", then unfortunately that tool is not working anymore so Python must be used.
- First, you need to generate a node list: if you want to use only one node for each PlanetLab site, and only the active ones, then the following link will generate a node list that you can save, for example, as nodelist.txt:
- Second, you add the slice to the nodes of the nodelist. Start the Python interpreter by issuing "python" command in a Linux shell, then type the following:
import xmlrpclib
server = xmlrpclib.ServerProxy('https://www.planet-lab.eu/PLCAPI/')
auth={'AuthMethod':'password','Username':'<your-email>','AuthString':'<your-password>'}
node_list=[line.strip() for line in open('pathToNodeList/nodelist.txt')]
server.AddSliceToNodes(auth,'yourSliceName',node_list)
If successful, this should return '1'.
If you want to do another verify that the above procedure went fine, you can issue, still in Python environment:
slice_nodeids=server.GetSlices(auth,'yourSliceName')[0]['node_ids']
added_nodeid=server.GetNodes(auth,node_list)[0]['node_id']
added_nodeid in slice_nodeids
If successful, this returns "True", otherwise "False" and something went wrong. You can then leave Python with exit() or CTRL-D.
- Third, please wait for PlanetLab to add your slice to the selected nodes: this may take some time and, during this, you can't log in into the remote nodes using your SSH credentials because they will still be unknown to the new nodes. During this time, you can start having a look at CoDeploy software package (see the link to PLE Tools into the Docs).
- Finally, you can use the nodes by logging into them with SSH, copying files with scp or rsync or the CoDeploy tools.
PLE Tools
The PlanetLab provides several tools to be used when dealing with slices involving a large number of nodes. These tools are useful for deploying software (e.g. a p2p client) from a local host to the remote nodes, or for transferring files (e.g. logs) from remote nodes to the local host.
The main tools are part of the "CoDeploy" software suite, which includes Multiquery, Multicopy and CoDeploy itself.
You should add some environment variable to the Linux system for properly using the software, without having to type initialization commands at every boot. On an Ubuntu system, add at the end of the /etc/profile the following lines:
export PATH=yourCoDeployPath:$PATH
hash -r
export MQ_NODES='pathToYourNodeList/nodelist.txt'
export MQ_SLICE='yourSliceName'
In this way, you can run the programs from any directory and they will use "yourSliceName" as user name for logging in at nodes specified in nodelist.txt. Another useful parameter is MQ_MAXLOAD, which defaults to 5 (when it is not specified) but can be changed according to the user needs. This option specifies the maximum number of simultaneous connections to the remote nodes, when dealing with multiple copies and queries. Also, with MQ_DELAY you can specify how many seconds (default is 0) you want between SSH connections.
Now let's have a look at the different programs of the suite:
- Unimore PLE/Docs/Tools/Multiquery is used for running commands on remote nodes of the slice, given a node list. For example, you can start programs or kill their processes.
- Unimore PLE/Docs/Tools/Multicopy is used for copying data from local host to the slice nodes and vice versa. This can be useful, for example, for deploying the configuration files of a program to the nodes after editing them locally, or for receiving many log files after an experiment.
- Unimore PLE/Docs/Tools/CoDeploy is meant for deploying of files to the nodes. So, its function is similar to Multicopy used for transfers from local host to remote nodes, however it aims at a more efficient distribution using a CDN (Content Distribution Network).
Useful Linux commands
When dealing with PlanetLab nodes, you may have the need of installing some software: this can be a compiler, a particular library and so on.
PlanetLab nodes use a Linux distribution based on the Fedora Core 8. If you need to install a program, then you need to gain root privileges by issuing
sudo su
which gives this prompt: bash-3.2# (you can leave this root mode by giving exit command)
From there you can give the command yum which is the package manager for Fedora. Without any parameter it supplies a list of commands and options. When installing a package, then you give
yum install packageName
or for removing one you give
yum erase packageName
These are the most useful commands. Dependencies are automatically solved by yum, which installs the needed packages together with the required one. For yum to work, the required package has to be in the repositories. If yum tells it can't find the package, then you have to check its name or install from RPM packages using the command rpm, which if not installed can be added to the system with
yum install rpm
RPM packages can be find on many Web sites, just search on Google by specifying the package name and the Linux distribution you are interested in. Packages from newer Fedora releases may work, but some other packages may need to be upgraded if required by the dependencies and you may have to download their RPMs too. Fortunately enough, RPM tells you if other packages are needed. For installing a RPM package, just issue (with root privileges)
rpm -i packageName.rpm
If you specify more packages, e.g. by issuing
rpm -i package1.rpm package2.rpm package3.rpm
then RPM will manage their installation so that, if there are dependencies between them, these can be solved automatically.
If you use a particular node compiling software to be distributed to other nodes, then you can download the sources by giving, without the need to be root, the following command:
wget link_to_file
This will download the source file into the current directory. Sources usually are in .tar.gz or .tar.bz2 compressed format, so you need to decompress the archives for further working on them. You can do this with
tar xvf fileName
and this extracts the file into the current directory, also creating the required subdirectories. You can then follow the instructions for compiling usually found in source packages: be sure to check for errors from the compiler (after issuing make) or from previous steps, some libraries may be missing from your system and you will need to install them for a successful compile.