RSS

(root)/integrade : /deployer/README (revision 1180)

To get this branch, use:
bzr branch /integrade
Line Revision Contents
1 1170.1.9
InteGrade: Object-Oriented Grid Middleware v0.5
2 989
3 1007
IG-Deployer: Taktuk powered version v0.2
4 989
5
Deployment Manual
6
7 1168.1.4
Last updated: June 18th 2009.
8 989
9
-----------------------------------------------------------------------------
10
INTRODUCTION
11
12 1170.1.9
IG-Deployer is an automatic deployment tool for InteGrade. It is based on a tool for deploying parallel remote executions of commands to a large set of remote nodes called Taktuk (http://taktuk.gforge.inria.fr/). For further information about IG-Deployer see http://www.integrade.org.br/files/nascimento-wscad.pdf.
13
14
This document describes the procedures for the deployment of InteGrade, using IG-deployer. It's intended to be used by users not familiar with InteGrade, nor the technologies it leverages. Follow the Quick Install Notes to have InteGrade up and running in localhost quickly.
15
16 1178
For information about this subject or any problems, questions, musings, or suggestions regarding InteGrade, please contact the InteGrade support team at integrade-support @t nospam googlegroups or consult the InteGrade portal (http://www.integrade.org.br).
17 989
18
-----------------------------------------------------------------------------
19
SETTING UP THE ENVIRONMENT
20
21 1170.1.9
--------------- Quick Install Note ---------------
22 1170.1.10
In a fresh Ubuntu 9.04 install, the following command is enough to install dependencies:
23
24
$ sudo aptitude install g++ libreadline5-dev default-jdk libssl-dev openssh-server
25
26
Now configure a passwordless ssh connection as explained in http://www.integrade.org.br/ssh-without-password . Then you can skip this section.
27 1170.1.9
-------------------- End Note --------------------
28
29
InteGrade v0.5 has many dependencies, and currently only works in GNU/Linux platform. Some of the dependencies are included in this version of IG-Deployer, but others are required as pre-requisites of this tool. For ease of notation, we will split the machines that are part of the installation set into two groups, the spanning node, composed by the node where the ig-deployer will be installed, and the grid, that is the rest of the machines that will receive InteGrade. Frequently, the spanning node will also be part of the grid.
30
31
In order to begin a proper deployment of InteGrade, this set of packages must be available in all grid machines, with an special exception for PERL, the only dependency of the spanning node. This exception of dependencies allow the deployer to run hypothetically in any operational system with a perl implementation and ssh client. The pre-requisites are:
32 989
33
- PERL v5.8.8 or greather.
34
35 1170.1.9
- GCC 4.0 or greater - The GNU C Compiler.
36
  Debian and Fedora packages: gcc-4.0, gcc-4.0-base
37
38
- G++ 4.0 or greater - The GNU C++ Compiler.
39
  Debian and Fedora package: g++-4.0
40
  Ubuntu 9.04 package: g++
41
42
- GNU Make - The popular UNIX-based build tool.
43 989
  Debian and Fedora package: make
44
45 1170.1.9
- JDK 1.5 or greater - The Java Development Kit v1.5.0, including both the compiler and runtime environment for the Java Language.
46
  Ubuntu 9.04 package: default-jdk
47
  Available at http://java.sun.com/javase/downloads/index_jdk5.jsp
48 989
49 1168.1.6
- Readline Library.
50 1170.1.9
  Debian/Ubuntu package: libreadline5-dev
51
52 1170.1.10
- SSL libraries.
53
  Ubuntu package: libssl-dev
54
55 1170.1.9
If a secure application repository is necessary (this is recommended for jobs running on shared machines), the following packages are also required:
56
57
 - General purpose cryptographic C++ library.
58
   Debian package: libcrypto++-dev
59
60
 - Kerberos with all libraries and header files.
61
   Debian packages: libkrb5-dev, krb5-user, krb5-admin-server, krb5-kdc
62
63
The deployer also requires SecureShell (ssh) connection without password in all machines as explained in http://www.integrade.org.br/ssh-without-password
64
65
Deployer already contains the main InteGrade dependencies:
66
67
- Ant 1.5 or greater - A Java-based platform-independent build tool.
68
69
- JacORB 2.2.3 - A Java-based open-source CORBA ORB that runs on cluster manager machines. 
70
71
- OiLPack (incl. Lua-5.0-Plus, Compat-Lua-5.1-r5, LuaSocket-2.0, and OiL 0.3.1) - A software bundle containing the compiler and runtime for the Lua scripting language, a library for building Lua-based socket applications, and the OiL lightweight CORBA ORB, which runs on resource provider nodes.
72 1007
73
-----------------------------------------------------------------------------
74
75
INSTALLATION
76
77 1170.1.9
The easy way to install InteGrade, is to download the ig-with-deployer package. It is a pack containing three elements: the InteGrade, the deployer and this manual. Check http://www.integrade.org.br/software for the latest version.
78 1168.1.4
79
Extract the package:
80
81 1170.1.9
$ tar zxvf packageName.tar.gz
82
or
83
$ tar xvf packageName.tar.lzma --lzma
84 989
85
To see if everything worked well just type:
86
87 1170.1.9
$ cd integrade/deployer
88 990
$ ./deployer check
89 989
90
You should see:
91
92 1168.1.4
Reading default.yml
93 1170.1.9
localhost(grm lrm), GRM localhost will install in /usr/local/integrade with java at /usr/lib/jvm/java-6-sun/
94
 and shared as Null.
95 1168.1.4
96
-----------------------------------------------------------------------------
97 1170.1.9
CONFIGURATION FILES (config/default.yml)
98
99
--------------- Quick Install Note ---------------
100 1170.1.10
Check the 'path' in config/default.yml. This installation dir must exist (with proper permissions). Another suggestion: ~/integrade (if you haven't already extracted the package there).
101 1170.1.9
102
If you've installed the default-jdk package (Ubuntu) change 'java' to '/usr/lib/jvm/default-java', otherwise check the correct java path, commonly found in /usr/lib/jvm:
103
104
$ ls /usr/lib/jvm
105
106
You may skip this section if you only want to install in localhost.
107
-------------------- End Note --------------------
108
109
The syntax of the configuration files are very simple. It is based on an human friendly data serialization standard called YAML (YAML Ain't Markup Language) available at (http://www.yaml.org/). These fields must be specified on the configuration file for each machine.
110
111
installgrm: True if the Global Resource Manager should be installed on this machine. The grm may be seen as the InteGrade server, and one special machine must be chosen to be GRM. False otherwise.
112
113
installlrm: True if the Local Resource Manager should be installed on this machine. The lrm may be seen as the InteGrade client. False otherwise.
114 1168.1.4
115
grm: The name of the machine where the GRM will be installed.
116
117 1168.1.6
path: The default path of installation. The directory must already exists.
118 1168.1.4
119 1170.1.9
shared: The path for the NFS installation. If omitted the dependencies will be installed in the machine, if a path is given, it will assume that this machine have access to the dependency folder.
120 1168.1.4
121
java: The path where the java dependency is installed.
122 989
123
An example should be:
124
125
---
126
ubuntu:
127 1168.1.4
   installgrm: true
128
   installlrm: true
129 1170.1.9
   grm: ubuntu
130 1168.3.1
   path: /usr/local/integrade
131
   shared: /global/integrade
132 1168.1.4
   java: /usr/lib/jvm/java-6-sun/
133
134 989
suse:
135 1168.1.4
   installgrm: false
136
   installlrm: true
137
   grm: ubuntu
138 1168.3.1
   path: /usr/local/integrade
139
   shared: /global/integrade
140 1168.1.4
   java: /usr/lib/jvm/java-6-sun/
141
142 989
fedora:
143 1168.1.4
   installgrm: false
144
   installlrm: true
145
   grm: ubuntu
146 1168.3.1
   path: /usr/local/integrade
147
   shared: /global/integrade
148 1168.1.4
   java: /usr/lib/jvm/java-6-sun/
149 989
150 1170.1.9
With this configuration, InteGrade will be installed in 3 machines, ubuntu, suse and fedora. All the machines will have InteGrade at /usr/local/integrade (the folder must exists before installation and must have write permissions for the user running ig-deployer). All the dependencies will be installed in ubuntu, at /global/integrade and it's assumed that all the other machines will have access to that folder. It will also assume that/usr/lib/jvm/java-6-sun/ contains the path for the Java Virtual Machine. Note that you don't need the shared information if you're going to replicate dependencies into all target nodes or if you want to install only into one machine.
151 1004
152 1168.1.4
All configuration files must be in the config directory.
153
154 1004
-----------------------------------------------------------------------------
155
COMMANDS
156
157 1170.1.9
--------------- Quick Install Note ---------------
158
To install using the default configuration (config/default.yml), run:
159
$ ./deployer all
160
-------------------- End Note --------------------
161
162 1004
The syntax of ig-deployer is the following: 
163
164 1168.1.4
deployer [YAML FILE] COMMAND [TOPOLOGY]
165
166 1170.1.9
The YAML FILE is the configuration file located in config directory. If omitted, default.yml will be used.
167 1004
168 1170.1.9
Where COMMAND is mandatory and is one of the commands introduced later in this document, TOPOLOGY stands for the configuration file named TOPOLOGY.yml, notice that the topology name must be given without the .yml extension, and if you supply no topology name, the default.yml topology will be assumed.
169 989
170
"ALL" COMMAND
171 1170.1.9
   The all command runs the sequence of commands CHECK, PING, CLEAN, PACK, DEPLOY, INSTALL, START.
172 991
173 989
"CHECK" COMMAND
174 1170.1.9
   This command performs a test of syntax against the given YAML file, to guarantee that file represents a valid topology. A valid topology must include a GRM, at least one LRM, a path to install the files and a java path.
175
176 989
"PACK" COMMAND
177 1170.1.9
   The command pack searches and packs two folders. The first one is InteGrade, that must be at ../integrade. The second one is the dependency package that is inside the folder packs/dependency. After this step two files will be generated in /pack folder, the deps.tar.gz and the deploy.tar.gz. If one of this files exists at the time of command invocation, the respective file is not created.
178 1168.1.4
179 989
"DEPLOY" COMMAND
180
   Sends the files deploy.tar.gz and deps.tar.gz for remote hosts.
181 1168.1.4
182 989
"INSTALL" COMMAND
183 1170.1.9
   Unpack and build all the dependencies. The result of this command is a complete installation of InteGrade. It depends of previous executions of the commands PACK and DEPLOY.
184 1168.1.4
185 1101.8.9
"INSTALLPORTAL" COMMAND
186 1170.1.9
   Installs the portal in the first GRM available. It depends of previous executions of the commands PACK and DEPLOY.
187 1168.1.4
188 989
"START" COMMAND
189 990
   Stop all services, start the grid GRM and them all the clients.
190 1170.1.9
 
191 989
"STOP" COMMAND
192 990
   Stop all services.
193 1168.1.4
194 989
"PING" COMMAND
195 1170.1.9
   Try to connect in all hosts in order to verify existence and functionality of them.
196 1168.1.4
197 989
"CLEAN" COMMAND
198 1170.1.9
   Erases local deployment files deps.tar.gz and deploy.tar.gz. It also removes remote files needed for installation.
199 1168.1.4
200 989
"UNINSTALL" COMMAND
201 990
   Removes InteGrade from the entire grid.
202 1168.1.4
203 989
-----------------------------------------------------------------------------
204 1170.1.9
RUNNING INTEGRADE
205
206
After deployer successfully finish, you can leave this deployer/README and jump to the section "LAUNCHING INTEGRADE" in integrade/INSTALL.
207
208
-----------------------------------------------------------------------------
209 1168.1.4
KNOWN ISSUES
210
211
If you are having any problem, it's a list of known bugs in deployer.
212
213 1170.1.9
- If the deps folder has the same path as the users home directory, the installation fails.
214
215
- The target "start" blocks its execution, it starts the InteGrade but it never knows that the start processes finishes.
216
217
- Sometimes the main service, the GRM, starts after some LRM, this is a problem in a set of large nodes because some machines will not be able to register to their GRM, and will not be part of the grid. You should then manually starts the server first (run $IG_HOME/startservices.sh servers) and then all the LRM (run $IG_HOME/startservices.sh client).
218
219
- Openssl Dependency is not being compiled nor properly linked. We recommend you to install it from your distribution package.
220 1168.1.6
221 1170.1.2
-----------------------------------------------------------------------------
222
TROUBLESHOOTING
223
224 1178
These are most common errors and their solutions. If your problem is not covered here, e-mail the error messages to integrade-support in nospam googlegroups.
225 1170.1.2
226
227
1. [hostname] ([pid number]): state > Connection failed
228
229
This error happens if the hostname provided in configuration file (defaults to localhost in config/default.yml) cannot be resolved.
230
231
To solve this problem, check if the hostname in the configuration file is correct. If it still can't be resolved (try "ping hostname"), you can add it to /etc/hosts. For instance, if hostname is "localhost", add "127.0.0.1 localhost" to /etc/hosts.
232
233
Then keep following the instructions, you need to have a passwordless ssh connection to that hostname, otherwise you get the next issue.
234
235
236
2. [hostname] ([pid number]): connector > Permission denied
237
238
It means you cannot "ssh hostname" without password. As written in section "SETTING UP THE ENVIRONMENT", follow the instructions in http://www.integrade.org.br/ssh-without-password

Loggerhead 1.17 is a web-based interface for Bazaar branches