Difference between revisions of "Remotemanager-call"
| (3 intermediate revisions by one other user not shown) | |||
| Line 28: | Line 28: | ||
* Description | * Description | ||
* [[FTPManager]] | [[FTPSManager]] | [[SFTPManager]] | * [[FTPManager]] | [[FTPSManager]] | [[SFTPManager]] | ||
| − | <div class="version_ge3. | + | <div class="version_ge3.4.0.10"> |
| [[HDFSRemoteManager]] | | [[HDFSRemoteManager]] | ||
</div> | </div> | ||
| Line 55: | Line 55: | ||
The remote commands can have the following subelements: | The remote commands can have the following subelements: | ||
* Description | * Description | ||
| − | <div class="version_ge3. | + | <div class="version_ge3.4.0.10"> |
* [[PropertyDef]] | * [[PropertyDef]] | ||
</div> | </div> | ||
| Line 106: | Line 106: | ||
| isCritical || optional || If true an internal error interrupt the command sequence. Default true. | | isCritical || optional || If true an internal error interrupt the command sequence. Default true. | ||
|} | |} | ||
| + | |||
| + | <div class="version_ge3.4.0.10"> | ||
| + | The following table shows the PropertyDef possible elements that can be set when the command is executed on the [[HDFSRemoteManager|HDFS]] file system: | ||
| + | {|class="gvtable" | ||
| + | ! Property !! Possible Values | ||
| + | |- | ||
| + | | dfs.file.overwrite || true/false; default=false | ||
| + | |} | ||
| + | This parameter can be used to owerwrite the target destination with the source one when a move operation is used on the HDFS file system. | ||
====RemoteDeleteCommand==== | ====RemoteDeleteCommand==== | ||
| Line 155: | Line 164: | ||
|} | |} | ||
| − | <div class="version_ge3. | + | <div class="version_ge3.4.0.10"> |
The following table shows the PropertyDef possible elements that can be set when the command is executed on the [[HDFSRemoteManager|HDFS]] file system: | The following table shows the PropertyDef possible elements that can be set when the command is executed on the [[HDFSRemoteManager|HDFS]] file system: | ||
{|class="gvtable" | {|class="gvtable" | ||
! Property !! Possible Values | ! Property !! Possible Values | ||
|- | |- | ||
| − | | dfs.file.delete-source || | + | | dfs.file.delete-source || fixed=false |
|- | |- | ||
| − | | dfs.file.raw-local-filesystem || true | + | | dfs.file.raw-local-filesystem || fixed=true |
|- | |- | ||
| − | | dfs.stream-buffer-size || default: | + | | dfs.stream-buffer-size || default: 4096 |
|} | |} | ||
These parameters can be used to specify the related action when a download operation is used on the HDFS file system. | These parameters can be used to specify the related action when a download operation is used on the HDFS file system. | ||
| Line 194: | Line 203: | ||
|} | |} | ||
| − | <div class="version_ge3. | + | <div class="version_ge3.4.0.10"> |
The following table shows the PropertyDef possible elements that can be set when the command is executed on the [[HDFSRemoteManager|HDFS]] file system: | The following table shows the PropertyDef possible elements that can be set when the command is executed on the [[HDFSRemoteManager|HDFS]] file system: | ||
{|class="gvtable" | {|class="gvtable" | ||
! Property !! Possible Values | ! Property !! Possible Values | ||
|- | |- | ||
| − | | dfs.file.delete-source || true/false | + | | dfs.file.delete-source || true/false; default=false |
|- | |- | ||
| − | | dfs.file.overwrite || true/false | + | | dfs.file.overwrite || true/false; default=false |
|- | |- | ||
| − | | dfs.permissions.path || Unix-like permission Example: 777 | + | | dfs.permissions.path || Unix-like permission Example: 777; default 700 |
|- | |- | ||
| dfs.stream-buffer-size || default: 8192 | | dfs.stream-buffer-size || default: 8192 | ||
| Line 209: | Line 218: | ||
| dfs.replication || default: 1 | | dfs.replication || default: 1 | ||
|- | |- | ||
| − | | dfs.blocksize || default: | + | | dfs.blocksize || default: 4194304 |
|- | |- | ||
| dfs.permissions.user || hdfs user owner | | dfs.permissions.user || hdfs user owner | ||
Latest revision as of 18:57, 20 November 2015
Contents
Description
This GreenVulcano® ESB plug-in allows you to perform a sequence of file operations in a remote file system accessible via various protocols.
VulCon / GV Console Configuration
The operations allowed are:
- Download a file/directory from a remote directory to a local directory
- Upload a file/directory from a local directory to a remote directory
- Control of one or more files (whose name corresponds to a date regular expressions) in a remote directory
- Deleting a file/directory from a remote directory
- Rename/move a file/directory in a remote directory
The remotemanager-call Element is used by Channel and routed-call. It has the attributes:
| Attribute | Type | Description |
|---|---|---|
| type | fixed | This attribute must assume the value call. |
| class | fixed | This attribute must assume the value it.greenvulcano.gvesb.virtual.file.remote.RemoteManagerCall. |
| name | required | Operation name. Used in the Flow section to associate workflow nodes to VCL operations. |
and the subelements:
- Description
- FTPManager | FTPSManager | SFTPManager
- RemoteCommands
RemoteCommands
The RemoteCommands element contains the sequence of FTP operations to be carried out. These operations will be performed in the order they were configured. An error in one of the operation marked as critical will terminate the execution of the entire sequence.
The RemoteCommands Element is used by remotemanager-call and, internally, it is possible to configure the following operations:
| Remote Command | Use this for... |
|---|---|
| RemoteSearchCommand | Controlling one or more files (whose name corresponds to a regular expressions) in a remote directory |
| RemoteMoveCommand | Renaming/moving a file/directory in a remote directory |
| RemoteDeleteCommand | Deleting a file/directory from a remote directory |
| RemoteDownloadCommand | Downloading a file/directory from a remote directory to a local directory |
| RemoteUploadCommand | Uploading a file/directory from a local directory to a remote directory |
The remote commands can have the following subelements:
- Description
RemoteSearchCommand
The GreenVulcano® ESB operation RemoteSearchCommand checks if one or more files, whose name conforms to a given regular expression, exists within the specified directory.
On completion and in the absence of errors the following properties of output GVBuffer are setted:
- GVRM_FIND_FILES_NUM Number of files found (>= 0)
- GVRM_FIND_FILES_LIST List of names of found files, separated by ','. Set only if the value of field 'GVRM_FIND_FILES_NUM' is greater than 0.
Both the pattern of the file and that the directory can be static or contain placeholders that are replaced at runtime.
The following table shows its attributes:
| Attribute | Type | Description |
|---|---|---|
| type | fixed | This attribute must assume the value remote-command. |
| class | fixed | This attribute must assume the value it.greenvulcano.gvesb.virtual.file.remote.command.GVSearch. |
| sourcePath | required | Absolute path of the directory to search in. |
| filePattern | required | File name to search for. Can be a regular expression. |
| isCritical | optional | If true an internal error interrupt the command sequence. Default true. |
RemoteMoveCommand
The GreenVulcano® ESB operation RemoteMoveCommand moves/renames a file/directory.
The file name, the source and destination directory names can be static or contain placeholders that are replaced at runtime. After the replacement of the placeholder, the value of the filename may contain a regular expression, in which case, all files of the source directory, whose name is in accordance with the corresponding pattern, will be moved to the destination directory. If the destination directory already contains files with the same name, they will be overwritten.
The following table shows its attributes:
| Attribute | Type | Description |
|---|---|---|
| type | fixed | This attribute must assume the value remote-command. |
| class | fixed | This attribute must assume the value it.greenvulcano.gvesb.virtual.file.remote.command.GVMove. |
| targetPath | required | Absolute path of the file/directory where is the file to move. |
| oldName | required | Name of the file to be moved. |
| newName | required | Name of the file moved. |
| isCritical | optional | If true an internal error interrupt the command sequence. Default true. |
The following table shows the PropertyDef possible elements that can be set when the command is executed on the HDFS file system:
| Property | Possible Values |
|---|---|
| dfs.file.overwrite | true/false; default=false |
This parameter can be used to owerwrite the target destination with the source one when a move operation is used on the HDFS file system.
RemoteDeleteCommand
The GreenVulcano® ESB operation RemoteDeleteCommand deletes a file or directory.
Both the file name and the directory can be static or contain placeholders that are replaced at runtime. After the replacement of the placeholder, the value of the filename may contain a regular expression, in which case, will erase all files in the specified directory whose name is consistent with the corresponding pattern.
The following table shows its attributes:
| Attribute | Type | Description |
|---|---|---|
| type | fixed | This attribute must assume the value remote-command. |
| class | fixed | This attribute must assume the value it.greenvulcano.gvesb.virtual.file.remote.command.GVDelete |
| targetPath | required | Absolute path of the file/directory to remove. |
| filePattern | optional | File name to remove. Might contain a regular expression. It is used only if targetPath is a directory, and if empty the filter is ignored. |
| isCritical | optional | If true an internal error interrupt the command sequence. Default true. |
RemoteDownloadCommand
The GreenVulcano® ESB operation RemoteDownloadCommand downloads a file or directory from a remote directory and saved in the local directory you specified (or in GVBuffer).
File names and directories can be static or contain placeholders that are replaced at runtime.
The following table shows its attributes:
| Attribute | Type | Description |
|---|---|---|
| type | fixed | This attribute must assume the value remote-command. |
| class | fixed | This attribute must assume the value it.greenvulcano.gvesb.virtual.file.remote.command.GVDownload. |
| remotePath | required | Pathname of remote directory containing the file to download. It can be an absolute pathname or relative, in the second case it is relative to home directory of ftp user. |
| remoteFilePattern | required | The name of the file to download. |
| toGVBufferExpression | optional | GVBuffer OGNL expression used to specify how to handle the current downloaded file. In the expression, the downloaded file will be referenced by the variable #input, and will be a byte []. |
| targetPath | optional | Pathname of the local directory to save the downloaded file. Must be an absolute pathname. |
| isCritical | optional | If true an internal error interrupt the command sequence. Default true. |
The following table shows the PropertyDef possible elements that can be set when the command is executed on the HDFS file system:
| Property | Possible Values |
|---|---|
| dfs.file.delete-source | fixed=false |
| dfs.file.raw-local-filesystem | fixed=true |
| dfs.stream-buffer-size | default: 4096 |
These parameters can be used to specify the related action when a download operation is used on the HDFS file system.
RemoteUploadCommand
The GreenVulcano® ESB operation RemoteUploadCommand uploads a file from a local directory (or GVBuffer) to the remote directory specified.
File and/or directory names can be static or contain placeholders that are replaced at runtime.
The following table shows its attributes:
| Attribute | Type | Description |
|---|---|---|
| type | fixed | This attribute must assume the value remote-command. |
| class | fixed | This attribute must assume the value it.greenvulcano.gvesb.virtual.file.remote.command.GVUpload. |
| sourcePath | optional | Pathname of the local directory where the file is transferred. Must be an absolute pathname. |
| sourceFilePattern | optional | The name of the file to transfer. |
| fromGVBufferExpression | optional | OGNL Expression applied to the current GVBuffer that specifies how to build the file content. In the expression, the GVBuffer is associated with #object, and must return a byte[]. |
| remotePath | required | Pathname of remote directory where upload the file. It can be an absolute pathname or relative, in the second case it is relative to home directory of ftp user. |
| isCritical | optional | If true an internal error interrupt the command sequence. Default true. |
The following table shows the PropertyDef possible elements that can be set when the command is executed on the HDFS file system:
| Property | Possible Values |
|---|---|
| dfs.file.delete-source | true/false; default=false |
| dfs.file.overwrite | true/false; default=false |
| dfs.permissions.path | Unix-like permission Example: 777; default 700 |
| dfs.stream-buffer-size | default: 8192 |
| dfs.replication | default: 1 |
| dfs.blocksize | default: 4194304 |
| dfs.permissions.user | hdfs user owner |
| dfs.permissions.group | hdfs user group |
These parameters can be used to owerwrite the target destination with the source one, set the replication, permissions and the other properties on destination target when an upload operation is used on the HDFS file system.
