ONB Web Archive Fits Characterisation using ToMaR
Created: 2013-12-09 15:58:54
Last updated: 2013-12-10 17:06:09
Workflow for Web Archive Content Characterisation using Fits.
The workflow uses Spacip (https://github.com/shsdev/spacip) to prepare ARC web is a tool to prepare ARC web archive container files by unpacking the compressed files in HDFS and creates input files which can be used by Tomar (https://github.com/openplanets/tomar). After merging the mapper output files from Spacip (MergeTomarInput) into one single file, it can be used by Tomar as input. The tool invokation depends on a tool specification file which must be available in HDFS, this is explained in the Tomar documentation.
Preview
Run
Run this Workflow in the Taverna Workbench...
Workflow Components
Authors (1)
Titles (1)
ONB Web Archive Fits Characterisation using ToMaR |
Descriptions (1)
Workflow for Web Archive Content Characterisation using Fits.
The workflow uses Spacip (https://github.com/shsdev/spacip) to prepare ARC web is a tool to prepare ARC web archive container files by unpacking the compressed files in HDFS and creates input files which can be used by Tomar (https://github.com/openplanets/tomar). After merging the mapper output files from Spacip (MergeTomarInput) into one single file, it can be used by Tomar as input. The tool invokation depends on a tool specification file which must be available in HDFS, this is explained in the Tomar documentation.
|
Dependencies (0)
Inputs (2)
Name |
Description |
hdfs_input_path |
HDFS input path
|
num_files_per_invokation |
Number of files per invokation
|
Processors (4)
Name |
Type |
Description |
Spacip |
externaltool |
|
MergeTomarInput |
externaltool |
|
Tomar |
externaltool |
|
toolspecs_hdfs_dir_value |
stringconstant |
Value/user/scape/scape-toolspecs |
Outputs (1)
Name |
Description |
Tomar_STDOUT |
|
Datalinks (6)
Source |
Sink |
hdfs_input_path |
Spacip:hdfs_input_path |
num_files_per_invokation |
Spacip:num_files_per_invokation |
Spacip:STDOUT |
MergeTomarInput:spacip_joboutput_hdfs_dir |
toolspecs_hdfs_dir_value:value |
Tomar:toolspecs_hdfs_dir |
MergeTomarInput:STDOUT |
Tomar:merged_tomar_input |
Tomar:STDOUT |
Tomar_STDOUT |
Uploader
License
All versions of this Workflow are
licensed under:
Version 1 (earliest)
(of 2)
Credits (1)
(People/Groups)
Attributions (0)
(Workflows/Files)
None
Shared with Groups (3)
Featured In Packs (0)
None
Log in to add to one of your Packs
Attributed By (0)
(Workflows/Files)
None
Favourited By (0)
No one
Statistics
Other workflows that use similar services
(0)
There are no workflows in myExperiment that use similar services to this Workflow.
Comments (0)
No comments yet
Log in to make a comment