Hadoop hOCR parser

Created: 2012-08-07 15:01:44 Last updated: 2012-08-07 15:10:07

Download Workflow

Big data processing: chaining Hadoop jobs using Taverna. This workflow demonstrates a simple way of linking different hadoop job components using the standard output of the hadoop jobs. It is not for thought for productive use, but for demonstration using small data sets. The code for the hadoop jobs is available on Github: tb-lsdr-hocrparser and tb-lsdr-seqfilecreator.

Preview

Download as scalable diagram (SVG)

Run

Run this Workflow in the Taverna Workbench...

Option 1:

Copy and paste this link into File > 'Open workflow location...'
http://www.myexperiment.org/workflows/3069/download?version=1
[ More Info Expand ]

Workflow Components

Authors (1)

Titles (1)

Descriptions (0)

Dependencies (0)

Inputs (2)

Name	Description
hdfs_input_dir	HDFS input directory with (a) text file(s) containing absolute paths to the files that should be added to the sequence file.
hadoop_job_name_prefix	Hadoop job name prefix

Processors (3)

Name	Type	Description
HadoopHocrAvBlockWidthMapReduce	externaltool
HadoopSequenceFileCreator	externaltool
HadoopFsCat	externaltool

Beanshells (0)

Outputs (1)

Name	Description
Tool_STDOUT

Datalinks (6)

Source	Sink
HadoopSequenceFileCreator:STDOUT	HadoopHocrAvBlockWidthMapReduce:hdfs_input_dir
hadoop_job_name_prefix	HadoopHocrAvBlockWidthMapReduce:hadoop_job_name_prefix
hdfs_input_dir	HadoopSequenceFileCreator:hdfs_input_path
hadoop_job_name_prefix	HadoopSequenceFileCreator:hadoop_job_name_prefix
HadoopHocrAvBlockWidthMapReduce:STDOUT	HadoopFsCat:hdfs_result_file
HadoopFsCat:STDOUT	Tool_STDOUT