Slim_Migrate_And_QA_nfs_output_path00 /home/bolette/TestOutput/ 2014-02-16 18:54:47.634 UTC Output directory fora the migrated wav files on nfs. 2014-02-16 18:54:23.579 UTC mp3_list_on_hdfs_input_path00 path to input file on hdfs containing list of paths to mp3 files on nfs to be migrated 2014-02-16 18:52:53.590 UTC input/mp3/filelist.txt 2014-02-16 18:53:38.293 UTC hdfs_output_path_200 Output directory for preservation event files and other log files. 2014-02-13 13:13:40.529 UTC output/test-output/MigrateMp3ToWav/ 2014-02-13 13:14:10.587 UTC mapreduce_output_path00 output/test2014-009 2014-02-16 18:51:54.334 UTC output directory for Hadoop output 2014-02-16 18:51:32.272 UTC jar_input_path00 /scape/shared/jars/ /home/bolette/Projects/scape-audio-qa/migrate_mp3_to_wav_hadoop/target 2014-02-26 10:03:13.601 UTC The directory where the jar file with the hadoop jobs is. 2014-02-26 09:59:22.177 UTC max_split_size00 max-split-size is the max input size to a Hadoop map task. The input to these Hadoop jobs are file lists, and we actually want a very small max-split-size, so each map task only gets few files to process. 2014-02-26 14:16:10.840 UTC 256 2014-02-26 14:17:05.227 UTC remove_wav_files_really_remove00xcorrSound_waveform__GetResultsFromHadoopJob_STDERRxcorrSound_waveform__GetResultsFromHadoopJob_STDOUTxcorrSound_waveform__HadoopJob_STDERRxcorrSound_waveform__HadoopJob_STDOUTWriteFilePairListToHDFS_STDERRWriteFilePairListToHDFS_STDOUTremove_wav_files_successremove_wav_files_2_successFfmpegMigrate_Tavernnfs_output_path0mp3_list_on_hdfs_input_path0hdfs_output_path_20mapreduce_output_path0jar_input_path0max_split_size0GetResultsFromHadoopJob_STDOUT00net.sf.taverna.t2.activitiesdataflow-activity1.4net.sf.taverna.t2.activities.dataflow.DataflowActivitynet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeMpg321Convert_Tavernhdfs_output_path_20nfs_output_path0mp3_list_on_hdfs_input_path0mapreduce_output_path0jar_input_path0max_split_size0GetResultsFromHadoopJob_STDOUT00net.sf.taverna.t2.activitiesdataflow-activity1.4net.sf.taverna.t2.activities.dataflow.DataflowActivitynet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeMakeWavFilePairsListffmpegMigratedWavPaths1mpg321ConvertedWavPaths1wavFilePathPairs11net.sf.taverna.t2.activitiesbeanshell-activity1.4net.sf.taverna.t2.activities.beanshell.BeanshellActivity ffmpegMigratedWavPaths 1 text/plain java.lang.String true mpg321ConvertedWavPaths 1 text/plain java.lang.String true wavFilePathPairs 1 1 workflow net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeOutputDirFfmpegJoboutputdir0ffmpegHadoopJobOutputDir00net.sf.taverna.t2.activitiesbeanshell-activity1.4net.sf.taverna.t2.activities.beanshell.BeanshellActivity outputdir 0 text/plain java.lang.String true ffmpegHadoopJobOutputDir 0 0 workflow net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeOutputDirMpg321Joboutputdir0mpg321HadoopJobOutputDir00net.sf.taverna.t2.activitiesbeanshell-activity1.4net.sf.taverna.t2.activities.beanshell.BeanshellActivity outputdir 0 text/plain java.lang.String true mpg321HadoopJobOutputDir 0 0 workflow net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeSplit_string_into_string_list_by_regular_expressionregex0string0split11net.sf.taverna.t2.activitieslocalworker-activity1.4net.sf.taverna.t2.activities.localworker.LocalworkerActivity string 0 'text/plain' java.lang.String true regex 0 'text/plain' java.lang.String true split 1 l('text/plain') 1 workflow org.embl.ebi.escience.scuflworkers.java.SplitByRegex net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeFormatterlines1formatted11net.sf.taverna.t2.activitiesbeanshell-activity1.4net.sf.taverna.t2.activities.beanshell.BeanshellActivity lines 1 text/plain java.lang.String true formatted 1 1 workflow net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeFormatter_2lines1formatted11net.sf.taverna.t2.activitiesbeanshell-activity1.4net.sf.taverna.t2.activities.beanshell.BeanshellActivity lines 1 text/plain java.lang.String true formatted 1 1 workflow net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Invokeregex_valuevalue00net.sf.taverna.t2.activitiesstringconstant-activity1.4net.sf.taverna.t2.activities.stringconstant.StringConstantActivity \n net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeSplit_string_into_string_list_by_regular_expression_2string0regex0split11net.sf.taverna.t2.activitieslocalworker-activity1.4net.sf.taverna.t2.activities.localworker.LocalworkerActivity string 0 'text/plain' java.lang.String true regex 0 'text/plain' java.lang.String true split 1 l('text/plain') 1 workflow org.embl.ebi.escience.scuflworkers.java.SplitByRegex net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeMerge_String_List_to_a_Stringstringlist1concatenated00net.sf.taverna.t2.activitieslocalworker-activity1.4net.sf.taverna.t2.activities.localworker.LocalworkerActivity stringlist 1 l('text/plain') java.lang.String true seperator 0 'text/plain' java.lang.String true concatenated 0 'text/plain' 0 workflow org.embl.ebi.escience.scuflworkers.java.StringListMerge net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeWriteFilePairListToHDFSmapreduce_output_path0file_pair_list_on_nfs0STDERR00STDOUT00net.sf.taverna.t2.activitiesexternal-tool-activity1.4net.sf.taverna.t2.activities.externaltool.ExternalToolActivity 789663B8-DA91-428A-9F7D-B3F3DA185FD4 default local <?xml version="1.0" encoding="UTF-8"?> <localInvocation><shellPrefix>/bin/sh -c</shellPrefix><linkCommand>/bin/ln -s %%PATH_TO_ORIGINAL%% %%TARGET_NAME%%</linkCommand></localInvocation> 9523bce2-86c7-4caf-a720-709f6ccd877d # Write FilePairList to HDFS hadoop fs -mkdir %%mapreduce_output_path%%; hadoop fs -put %%file_pair_list_on_nfs%% %%mapreduce_output_path%%/ 1200 1800 file_pair_list_on_nfs mapreduce_output_path mapreduce_output_path mapreduce_output_path false false false UTF-8 false false false file_pair_list_on_nfs file_pair_list_on_nfs false false false UTF-8 false false false false true true 0 false net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeWrite_Text_Filefilecontents0encoding0outputFile0outputFile00net.sf.taverna.t2.activitieslocalworker-activity1.4net.sf.taverna.t2.activities.localworker.LocalworkerActivity outputFile 0 'text/plain' java.lang.String true filecontents 0 'text/plain' java.lang.String true encoding 0 'text/plain' java.lang.String true outputFile 0 'text/plain' 0 workflow net.sourceforge.taverna.scuflworkers.io.TextFileWriter net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Invokeutf8value00net.sf.taverna.t2.activitiesstringconstant-activity1.4net.sf.taverna.t2.activities.stringconstant.StringConstantActivity utf8 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeWavFilePairListFullPathOnNFSstring20string10output00net.sf.taverna.t2.activitieslocalworker-activity1.4net.sf.taverna.t2.activities.localworker.LocalworkerActivity string1 0 'text/plain' java.lang.String true string2 0 'text/plain' java.lang.String true output 0 0 workflow org.embl.ebi.escience.scuflworkers.java.StringConcat UserNameHere 2014-02-17 10:06:21.470 UTC net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokewavFilePairsList.txtvalue00net.sf.taverna.t2.activitiesstringconstant-activity1.4net.sf.taverna.t2.activities.stringconstant.StringConstantActivity wavFilePairsList.txt net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeOutputDirWaveformCompareJoboutputdir0waveformcompareHadoopJobOutputDir00net.sf.taverna.t2.activitiesbeanshell-activity1.4net.sf.taverna.t2.activities.beanshell.BeanshellActivity outputdir 0 text/plain java.lang.String true waveformcompareHadoopJobOutputDir 0 0 workflow net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokexcorrSound_waveform_wav_file_pairs_list_on_hdfs_input_path0nfs_output_path0mapreduce_output_path0hdfs_output_path_20jar_input_path0max_split_size0GetResultsFromHadoopJob_STDERR00GetResultsFromHadoopJob_STDOUT00HadoopJob_STDERR00HadoopJob_STDOUT00net.sf.taverna.t2.activitiesdataflow-activity1.4net.sf.taverna.t2.activities.dataflow.DataflowActivitynet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeWavFilePairListFullPathOnHDFSstring20string10output00net.sf.taverna.t2.activitieslocalworker-activity1.4net.sf.taverna.t2.activities.localworker.LocalworkerActivity string1 0 'text/plain' java.lang.String true string2 0 'text/plain' java.lang.String true output 0 0 workflow org.embl.ebi.escience.scuflworkers.java.StringConcat UserNameHere 2014-02-17 10:06:21.470 UTC net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Invokeremove_wav_filesfile_list1really_remove0success00net.sf.taverna.t2.activitiesbeanshell-activity1.4net.sf.taverna.t2.activities.beanshell.BeanshellActivity file_list 1 text/plain java.lang.String true really_remove 0 text/plain java.lang.String true success 0 0 workflow net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Invokeremove_wav_files_2file_list1really_remove0success00net.sf.taverna.t2.activitiesbeanshell-activity1.4net.sf.taverna.t2.activities.beanshell.BeanshellActivity file_list 1 text/plain java.lang.String true really_remove 0 text/plain java.lang.String true success 0 0 workflow net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeReally_really_removereally_remove0really_really_remove00net.sf.taverna.t2.activitiesbeanshell-activity1.4net.sf.taverna.t2.activities.beanshell.BeanshellActivity really_remove 0 text/plain java.lang.String true really_really_remove 0 0 workflow net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeFfmpegMigrate_Tavernnfs_output_pathnfs_output_pathFfmpegMigrate_Tavernmp3_list_on_hdfs_input_pathmp3_list_on_hdfs_input_pathFfmpegMigrate_Tavernhdfs_output_path_2hdfs_output_path_2FfmpegMigrate_Tavernmapreduce_output_pathOutputDirFfmpegJobffmpegHadoopJobOutputDirFfmpegMigrate_Tavernjar_input_pathjar_input_pathFfmpegMigrate_Tavernmax_split_sizemax_split_sizeMpg321Convert_Tavernhdfs_output_path_2hdfs_output_path_2Mpg321Convert_Tavernnfs_output_pathnfs_output_pathMpg321Convert_Tavernmp3_list_on_hdfs_input_pathmp3_list_on_hdfs_input_pathMpg321Convert_Tavernmapreduce_output_pathOutputDirMpg321Jobmpg321HadoopJobOutputDirMpg321Convert_Tavernjar_input_pathjar_input_pathMpg321Convert_Tavernmax_split_sizemax_split_sizeMakeWavFilePairsListffmpegMigratedWavPathsFormatterformattedMakeWavFilePairsListmpg321ConvertedWavPathsFormatter_2formattedOutputDirFfmpegJoboutputdirmapreduce_output_pathOutputDirMpg321Joboutputdirmapreduce_output_pathSplit_string_into_string_list_by_regular_expressionregexregex_valuevalueSplit_string_into_string_list_by_regular_expressionstringFfmpegMigrate_TavernGetResultsFromHadoopJob_STDOUTFormatterlinesSplit_string_into_string_list_by_regular_expressionsplitFormatter_2linesSplit_string_into_string_list_by_regular_expression_2splitSplit_string_into_string_list_by_regular_expression_2stringMpg321Convert_TavernGetResultsFromHadoopJob_STDOUTSplit_string_into_string_list_by_regular_expression_2regexregex_valuevalueMerge_String_List_to_a_StringstringlistMakeWavFilePairsListwavFilePathPairsWriteFilePairListToHDFSmapreduce_output_pathmapreduce_output_pathWriteFilePairListToHDFSfile_pair_list_on_nfsWavFilePairListFullPathOnNFSoutputWrite_Text_FilefilecontentsMerge_String_List_to_a_StringconcatenatedWrite_Text_Fileencodingutf8valueWrite_Text_FileoutputFileWavFilePairListFullPathOnNFSoutputWavFilePairListFullPathOnNFSstring2wavFilePairsList.txtvalueWavFilePairListFullPathOnNFSstring1nfs_output_pathOutputDirWaveformCompareJoboutputdirmapreduce_output_pathxcorrSound_waveform_wav_file_pairs_list_on_hdfs_input_pathWavFilePairListFullPathOnHDFSoutputxcorrSound_waveform_nfs_output_pathnfs_output_pathxcorrSound_waveform_mapreduce_output_pathOutputDirWaveformCompareJobwaveformcompareHadoopJobOutputDirxcorrSound_waveform_hdfs_output_path_2hdfs_output_path_2xcorrSound_waveform_jar_input_pathjar_input_pathxcorrSound_waveform_max_split_sizemax_split_sizeWavFilePairListFullPathOnHDFSstring2wavFilePairsList.txtvalueWavFilePairListFullPathOnHDFSstring1mapreduce_output_pathremove_wav_filesfile_listFormatterformattedremove_wav_filesreally_removeReally_really_removereally_really_removeremove_wav_files_2file_listFormatter_2formattedremove_wav_files_2really_removeReally_really_removereally_really_removeReally_really_removereally_removeremove_wav_files_really_removexcorrSound_waveform__GetResultsFromHadoopJob_STDERRxcorrSound_waveform_GetResultsFromHadoopJob_STDERRxcorrSound_waveform__GetResultsFromHadoopJob_STDOUTxcorrSound_waveform_GetResultsFromHadoopJob_STDOUTxcorrSound_waveform__HadoopJob_STDERRxcorrSound_waveform_HadoopJob_STDERRxcorrSound_waveform__HadoopJob_STDOUTxcorrSound_waveform_HadoopJob_STDOUTWriteFilePairListToHDFS_STDERRWriteFilePairListToHDFSSTDERRWriteFilePairListToHDFS_STDOUTWriteFilePairListToHDFSSTDOUTremove_wav_files_successremove_wav_filessuccessremove_wav_files_2_successremove_wav_files_2success 0caeed66-b873-464d-9ce2-3213969627e9 2014-02-16 18:54:51.566 UTC 9a045cde-7ab7-4fd5-8d73-4437e592630f 2014-02-26 14:17:16.314 UTC 3365051c-85dc-407f-ab95-46b8537104dc 2014-02-26 10:07:10.538 UTC This workflow migrates an input list (available on HDFS) of mp3 files (available on NFS) to wav files (in output directory on NFS) using an ffmpeg Hadoop job. The workflow then compares content of the original mp3 and the migrated wav by first converting the two files to wav using an mpg123 Hadoop job and the identity function respectively, and then using an xcorrSound waveform-compare Hadoop job. The needed Hadoop jobs are available from https://github.com/statsbiblioteket/scape-audio-qa-experiments 2014-06-30 09:13:58.378 UTC 27e4ff4b-82a0-404f-b753-593b5308f0c9 2014-06-30 09:13:35.489 UTC c96e41f1-f42d-468d-b52b-9e4a0985ba64 2014-05-02 08:36:35.55 UTC 9a413a7f-132b-4c3c-907a-6b03ba62e307 2014-02-17 10:57:56.887 UTC 252983cc-784d-49a9-b01f-cce3fd6a9c78 2014-05-05 07:47:35.458 UTC d4921c6e-ce9f-40b2-bb73-920796fd520d 2014-02-13 13:09:59.190 UTC cd485079-dc72-4ff6-b83d-10c4ed874268 2014-02-26 10:03:17.305 UTC 571b5955-f50b-4023-a631-a9ae30fd8e37 2014-02-26 10:03:11.355 UTC 371f1f72-0284-4172-a890-84a48953a9a6 2014-02-16 18:53:05.825 UTC 103b3005-b5b3-4be9-a16b-8c0dc33a44a0 2014-02-16 19:27:42.513 UTC 64b48150-91d7-4bc1-a4f8-57629a2d104c 2014-02-17 11:07:16.308 UTC 2cbe6947-ee2e-49ad-90b4-ec73cdf97100 2014-02-16 19:30:20.245 UTC 0c9b11e3-e4ca-4a7d-b14d-d4e5cbbbf6da 2014-02-17 10:19:56.227 UTC fba56f43-ae59-4ab0-bb53-96d440656aa3 2014-02-13 13:08:59.949 UTC 73fe964f-b4eb-4754-b0f0-5162604773d8 2014-03-18 07:54:36.272 UTC 45f29765-faca-4f3b-bc20-06fe4bb28d6e 2014-02-14 18:57:27.188 UTC e8cc6437-a0a1-444c-94e6-1c3db9ad4847 2014-02-17 09:34:23.577 UTC 98f6eb8c-384e-4f13-9560-822077e714fb 2014-02-13 20:48:34.593 UTC 4e72ff15-62e8-4288-9890-385fc226fed1 2014-02-26 09:59:43.201 UTC 5725cda0-45dd-4d55-90b8-730d1a5017f4 2014-05-02 09:00:39.755 UTC d61b63e1-ffb1-4626-a236-81f8f032119a 2014-02-26 14:17:01.797 UTC 921fd4bd-d9b4-4c0f-9eb5-2a2b1ad53a1b 2014-02-16 19:12:41.848 UTC 73836ffb-1126-4cac-b0a6-e2bc2852c7b6 2014-02-15 07:53:51.249 UTC 0d5751ff-9d71-4b98-9d49-d64175bef90a 2014-02-13 13:14:10.852 UTC a2cd843f-4333-4bd4-b8f2-3f9779bf5720 2014-02-13 13:15:59.352 UTC e48412d7-902a-44a8-bebd-09b9d0a2d589 2014-02-13 13:26:11.975 UTC 7d3220af-b97f-464e-947d-07c78e254e35 2014-06-30 09:21:00.413 UTC c323dc39-e523-4f39-95af-e2c83a160347 2014-02-15 07:50:03.286 UTC 06cb246f-ba63-4250-af6b-9fc73841438f 2014-02-16 19:09:24.815 UTC 1e8d7bab-8999-4a84-ae6f-358edc0e8dac 2014-02-14 12:25:21.932 UTC cb085352-ad80-4726-a4bc-86c8850af4ab 2014-06-23 08:17:58.553 UTC 51fbbf97-f74c-4631-8457-6e67d85de442 2014-02-14 18:58:03.44 UTC 9ebbac67-c787-43f3-a529-5b321dec0cb6 2014-02-14 11:42:19.49 UTC ec222728-c58a-45d9-93a3-170443dea987 2014-02-17 10:47:02.594 UTC Slim Migrate And QA mp3 to Wav Using Hadoop Jobs. 2014-02-13 13:09:30.476 UTC c7d4b459-77e7-42c5-8c9d-ff83e2d22a46 2014-02-17 10:08:54.599 UTC f2a09fac-2aa2-477a-81d2-789674641b3c 2014-02-17 10:28:42.783 UTC de11c668-240b-4244-a35a-be596b09804d 2014-02-17 10:41:25.125 UTC cd4670f1-aaf3-4311-a539-41683050f181 2014-02-16 18:53:19.461 UTC 70f547a3-5fd4-4b3b-ac10-7c3445475b76 2014-04-08 07:47:05.137 UTC a84423d2-144a-4c61-8639-f473826be3da 2014-03-17 08:53:10.897 UTC 62df025e-55c3-4657-be4b-867e67c463bc 2014-03-17 11:43:29.195 UTC 92c9a0e2-b0b8-4de5-82f8-f05e142f7ac9 2014-03-19 09:38:19.864 UTC f7cdbaad-60f9-4c4a-a71d-0fc8344e7f23 2014-02-26 14:20:04.140 UTC 83b6603a-0f4d-4b33-99c8-99036af8336f 2014-02-14 19:25:25.29 UTC dc25ff57-0f95-4db2-a81e-d22df7f9e3cc 2014-03-17 11:44:26.604 UTC 8a79036c-0688-48a6-8772-74aa1058b08e 2014-02-16 19:20:19.21 UTC cfc2d52f-4395-4a8d-9d67-13f7cc26f6bc 2014-02-17 10:34:13.568 UTC 9a937735-2940-4ca6-983c-55cee03f8e7c 2014-02-17 10:37:51.543 UTC Bolette A. Jurik, Statsbiblioteket & SCAPE 2014-02-13 13:10:00.54 UTC 43c70b3f-ead8-48ef-a9c2-c89d12b89d31 2014-02-14 18:49:24.888 UTC 4d52b623-e925-424a-9522-7b9dd44ddce9 2014-02-17 11:10:42.102 UTC e592ba05-3c7a-49d4-8590-3b7fd44ea489 2014-02-26 14:22:43.551 UTC e3d47c73-e233-4f12-a3db-70d89f40fcb7 2014-02-15 10:29:25.957 UTC bcfe7d75-096b-4759-beca-fd2d9bb2e5d3 2014-02-13 13:11:49.886 UTC FfmpegMigrate_Tavernmp3_list_on_hdfs_input_path00 path to input file on hdfs containing list of paths to mp3 files on nfs to be migrated 2014-02-16 18:53:19.779 UTC input/mp3/filelist.txt 2014-01-14 10:45:01.105 UTC mapreduce_output_path00 output/test2014-009 2014-01-14 10:44:42.669 UTC output directory for Taverna output 2014-01-14 10:44:26.511 UTC hdfs_output_path_200 Output directory for preservation event files and other log files. 2014-01-30 15:13:40.77 UTC output/test-output/MigrateMp3ToWav/ 2014-01-30 15:14:15.437 UTC nfs_output_path00 Output directory fora the migrated wav files on nfs. 2014-02-16 18:54:13.255 UTC /home/bolette/TestOutput/ 2014-01-30 15:15:12.216 UTC jar_input_path00max_split_size00HadoopJob_STDOUTHadoopJob_STDERRGetResultsFromHadoopJob_STDERRGetResultsFromHadoopJob_STDOUTFfmpegMigrateHadoopJobhdfs_input_path0mapreduce_output_path0hdfs_output_path_20nfs_output_path0jar_input_path0max_split_size0STDOUT00STDERR00net.sf.taverna.t2.activitiesexternal-tool-activity1.4net.sf.taverna.t2.activities.externaltool.ExternalToolActivity 789663B8-DA91-428A-9F7D-B3F3DA185FD4 default local <?xml version="1.0" encoding="UTF-8"?> <localInvocation><shellPrefix>/bin/sh -c</shellPrefix><linkCommand>/bin/ln -s %%PATH_TO_ORIGINAL%% %%TARGET_NAME%%</linkCommand></localInvocation> 9523bce2-86c7-4caf-a720-709f6ccd877d # Configure migrate_mp3_to_wav_hadoop_JAR_PATH=%%jar_input_path%%/migrate_mp3_to_wav_hadoop-0.1-SNAPSHOT-jar-with-dependencies.jar # Hadoop job hadoop jar ${migrate_mp3_to_wav_hadoop_JAR_PATH} eu.scape_project.audio_qa.ffmpeg_migrate.FfmpegMigrate -Dmapred.max.split.size=%%max_split_size%% %%hdfs_input_path%% %%mapreduce_output_path%% %%hdfs_output_path_2%% %%nfs_output_path%% 1200 1800 hdfs_input_path hdfs_output_path_2 jar_input_path mapreduce_output_path max_split_size nfs_output_path max_split_size max_split_size false false false UTF-8 false false false hdfs_input_path hdfs_input_path false false false UTF-8 false false false nfs_output_path nfs_output_path false false false UTF-8 false false false mapreduce_output_path mapreduce_output_path false false false UTF-8 false false false hdfs_output_path_2 hdfs_output_path_2 false false false UTF-8 false false false jar_input_path jar_input_path false false false UTF-8 false false false false true true 0 false net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeGetResultsFromHadoopJobmapreduce_output_path0STDERR00STDOUT00net.sf.taverna.t2.activitiesexternal-tool-activity1.4net.sf.taverna.t2.activities.externaltool.ExternalToolActivity 789663B8-DA91-428A-9F7D-B3F3DA185FD4 default local <?xml version="1.0" encoding="UTF-8"?> <localInvocation><shellPrefix>/bin/sh -c</shellPrefix><linkCommand>/bin/ln -s %%PATH_TO_ORIGINAL%% %%TARGET_NAME%%</linkCommand></localInvocation> 9523bce2-86c7-4caf-a720-709f6ccd877d # Read HDFS Hadoop job output hadoop fs -cat %%mapreduce_output_path%%/part-r-00000 1200 1800 mapreduce_output_path mapreduce_output_path mapreduce_output_path false false false UTF-8 false false false false true true 0 false net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeFfmpegMigrateHadoopJobhdfs_input_pathmp3_list_on_hdfs_input_pathFfmpegMigrateHadoopJobmapreduce_output_pathmapreduce_output_pathFfmpegMigrateHadoopJobhdfs_output_path_2hdfs_output_path_2FfmpegMigrateHadoopJobnfs_output_pathnfs_output_pathFfmpegMigrateHadoopJobjar_input_pathjar_input_pathFfmpegMigrateHadoopJobmax_split_sizemax_split_sizeGetResultsFromHadoopJobmapreduce_output_pathmapreduce_output_pathHadoopJob_STDOUTFfmpegMigrateHadoopJobSTDOUTHadoopJob_STDERRFfmpegMigrateHadoopJobSTDERRGetResultsFromHadoopJob_STDERRGetResultsFromHadoopJobSTDERRGetResultsFromHadoopJob_STDOUTGetResultsFromHadoopJobSTDOUT Bolette A. Jurik, Statsbiblioteket & SCAPE 2014-01-14 10:46:02.398 UTC 31153f8c-ce5e-44ed-bb7f-1ea6c1f59cb1 2014-02-13 13:07:40.427 UTC 014fa84d-e620-47de-9f8b-fc5ec16259a5 2014-02-26 09:58:04.134 UTC b3b071bb-6a05-4e22-804d-3ee6b160cfa6 2014-01-30 15:15:18.104 UTC 1caf69fe-5742-40a7-b776-e053f59e29e1 2014-02-16 18:53:05.626 UTC 92bb3da2-f8f4-446d-9a52-5b33c113340e 2014-02-13 10:47:40.857 UTC 04aab78f-3312-48aa-b5b3-5ade853dcbc2 2014-02-16 18:54:51.313 UTC 18331161-9134-4992-826d-0c61a145fa2a 2014-02-13 10:36:45.35 UTC a6a4362d-ffef-429c-b1f4-517c937cbc3a 2014-01-30 15:15:10.184 UTC 1ef352ba-0251-4ab0-8d2e-901142ac08f3 2014-01-14 10:21:04.29 UTC 33b9466a-2cdb-42b5-b9a8-8a9e4b49ed1c 2014-01-31 07:56:06.362 UTC 0b665efc-5f67-47c9-b9de-42f47fe204f2 2014-02-26 14:12:44.398 UTC dcb03855-b944-46cb-9991-5445b4414d7f 2014-02-13 13:07:08.919 UTC 3629a1cb-67ca-4ccf-b3ae-960c6f29543d 2014-01-30 15:19:13.424 UTC 20ff7e22-21a2-4b68-8cb0-aee205216e91 2014-02-16 18:53:19.287 UTC 75d877e4-d973-43fd-aee8-013f9724f918 2014-01-14 10:47:27.703 UTC 3b0c8bd3-a022-4e1e-9f96-d1456a9c9921 2014-02-11 13:42:20.515 UTC FfmpegMigrate Taverna Workflow using FfmpegMigrate Hadoob Job to migrate a list of mp3 files to wav files. 2014-02-13 13:07:11.522 UTC xcorrSound_waveform_wav_file_pairs_list_on_hdfs_input_path00 input/wav_file_pairs.txt 2014-02-13 12:51:22.428 UTC path to input file on hdfs containing list of pairs of paths to wav files on nfs to be compared 2014-02-13 12:49:29.669 UTC mapreduce_output_path00 output/test2014-009 2014-01-14 10:44:42.669 UTC output directory for Taverna output 2014-01-14 10:44:26.511 UTC hdfs_output_path_200 Output directory for preservation event files and other log files. 2014-01-30 15:13:40.77 UTC output/test-output/MigrateMp3ToWav/ 2014-01-30 15:14:15.437 UTC nfs_output_path00 /home/bolette/TestOutput/ 2014-01-30 15:15:12.216 UTC Output directory for the migrated wav files on nfs. 2014-01-30 15:14:46.670 UTC jar_input_path00max_split_size00HadoopJob_STDOUTHadoopJob_STDERRGetResultsFromHadoopJob_STDERRGetResultsFromHadoopJob_STDOUTWavefileCompareHadoopJobhdfs_input_path0mapreduce_output_path0hdfs_output_path_20nfs_output_path0jar_input_path0max_split_size0STDOUT00STDERR00net.sf.taverna.t2.activitiesexternal-tool-activity1.4net.sf.taverna.t2.activities.externaltool.ExternalToolActivity 789663B8-DA91-428A-9F7D-B3F3DA185FD4 default local <?xml version="1.0" encoding="UTF-8"?> <localInvocation><shellPrefix>/bin/sh -c</shellPrefix><linkCommand>/bin/ln -s %%PATH_TO_ORIGINAL%% %%TARGET_NAME%%</linkCommand></localInvocation> 9523bce2-86c7-4caf-a720-709f6ccd877d # Configure migrate_mp3_to_wav_hadoop_JAR_PATH=%%jar_input_path%%/migrate_mp3_to_wav_hadoop-0.1-SNAPSHOT-jar-with-dependencies.jar # Hadoop job hadoop jar ${migrate_mp3_to_wav_hadoop_JAR_PATH} eu.scape_project.audio_qa.waveform_compare.WaveformCompare -Dmapred.max.split.size=%%max_split_size%% %%hdfs_input_path%% %%mapreduce_output_path%% %%hdfs_output_path_2%% %%nfs_output_path%% 1200 1800 hdfs_input_path hdfs_output_path_2 jar_input_path mapreduce_output_path max_split_size nfs_output_path max_split_size max_split_size false false false UTF-8 false false false hdfs_input_path hdfs_input_path false false false UTF-8 false false false nfs_output_path nfs_output_path false false false UTF-8 false false false mapreduce_output_path mapreduce_output_path false false false UTF-8 false false false hdfs_output_path_2 hdfs_output_path_2 false false false UTF-8 false false false jar_input_path jar_input_path false false false UTF-8 false false false false true true 0 false net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeGetResultsFromHadoopJobmapreduce_output_path0STDERR00STDOUT00net.sf.taverna.t2.activitiesexternal-tool-activity1.4net.sf.taverna.t2.activities.externaltool.ExternalToolActivity 789663B8-DA91-428A-9F7D-B3F3DA185FD4 default local <?xml version="1.0" encoding="UTF-8"?> <localInvocation><shellPrefix>/bin/sh -c</shellPrefix><linkCommand>/bin/ln -s %%PATH_TO_ORIGINAL%% %%TARGET_NAME%%</linkCommand></localInvocation> 9523bce2-86c7-4caf-a720-709f6ccd877d # Read HDFS Hadoop job output hadoop fs -cat %%mapreduce_output_path%%/part-r-00000 1200 1800 mapreduce_output_path mapreduce_output_path mapreduce_output_path false false false UTF-8 false false false false true true 0 false net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeWavefileCompareHadoopJobhdfs_input_pathwav_file_pairs_list_on_hdfs_input_pathWavefileCompareHadoopJobmapreduce_output_pathmapreduce_output_pathWavefileCompareHadoopJobhdfs_output_path_2hdfs_output_path_2WavefileCompareHadoopJobnfs_output_pathnfs_output_pathWavefileCompareHadoopJobjar_input_pathjar_input_pathWavefileCompareHadoopJobmax_split_sizemax_split_sizeGetResultsFromHadoopJobmapreduce_output_pathmapreduce_output_pathHadoopJob_STDOUTWavefileCompareHadoopJobSTDOUTHadoopJob_STDERRWavefileCompareHadoopJobSTDERRGetResultsFromHadoopJob_STDERRGetResultsFromHadoopJobSTDERRGetResultsFromHadoopJob_STDOUTGetResultsFromHadoopJobSTDOUT 33b9466a-2cdb-42b5-b9a8-8a9e4b49ed1c 2014-01-31 07:56:06.362 UTC 6755a2ed-2aa1-485a-9e11-aa6170587ee5 2014-02-13 12:42:07.29 UTC 3629a1cb-67ca-4ccf-b3ae-960c6f29543d 2014-01-30 15:19:13.424 UTC 75d877e4-d973-43fd-aee8-013f9724f918 2014-01-14 10:47:27.703 UTC b3b071bb-6a05-4e22-804d-3ee6b160cfa6 2014-01-30 15:15:18.104 UTC Bolette A. Jurik, Statsbiblioteket & SCAPE 2014-01-14 10:46:02.398 UTC 58181448-3893-4460-934b-7c04b5557eaf 2014-02-13 12:51:16.574 UTC a6a4362d-ffef-429c-b1f4-517c937cbc3a 2014-01-30 15:15:10.184 UTC 064f3c60-536e-4d12-b4bd-e234d7ef2c75 2014-02-26 10:06:44.556 UTC 92bb3da2-f8f4-446d-9a52-5b33c113340e 2014-02-13 10:47:40.857 UTC 3b0c8bd3-a022-4e1e-9f96-d1456a9c9921 2014-02-11 13:42:20.515 UTC 18331161-9134-4992-826d-0c61a145fa2a 2014-02-13 10:36:45.35 UTC 1ef352ba-0251-4ab0-8d2e-901142ac08f3 2014-01-14 10:21:04.29 UTC ebd85210-f89f-4efd-99c9-a6213e1a8545 2014-02-26 14:22:06.102 UTC xcorrSound waveform-compare Taverna Workflow using WaveformCompare Hadoob Job to compare a list of pairs of wav files. 2014-02-13 12:48:13.480 UTC b5bc31e6-dfce-4d16-8092-0f10cc66720f 2014-02-13 12:53:09.539 UTC Mpg321Convert_Tavernmp3_list_on_hdfs_input_path00 input/mp3/filelist.txt 2014-01-14 10:45:01.105 UTC path to input file on hdfs containing list of paths to mp3 files on nfs to be migrated 2014-01-30 15:12:35.851 UTC mapreduce_output_path00 output/test2014-009 2014-01-14 10:44:42.669 UTC output directory for Taverna output 2014-01-14 10:44:26.511 UTC hdfs_output_path_200 output/test-output/MigrateMp3ToWav/ 2014-01-30 15:14:15.437 UTC Output directory for preservation event files and other log files. 2014-01-30 15:13:40.77 UTC nfs_output_path00 /home/bolette/TestOutput/ 2014-01-30 15:15:12.216 UTC Output directory for the migrated wav files on nfs. 2014-01-30 15:14:46.670 UTC jar_input_path00max_split_size00HadoopJob_STDOUTHadoopJob_STDERRGetResultsFromHadoopJob_STDERRGetResultsFromHadoopJob_STDOUTMpg321ConvertHadoopJobhdfs_input_path0mapreduce_output_path0hdfs_output_path_20nfs_output_path0jar_input_path0max_split_size0STDOUT00STDERR00net.sf.taverna.t2.activitiesexternal-tool-activity1.4net.sf.taverna.t2.activities.externaltool.ExternalToolActivity 789663B8-DA91-428A-9F7D-B3F3DA185FD4 default local <?xml version="1.0" encoding="UTF-8"?> <localInvocation><shellPrefix>/bin/sh -c</shellPrefix><linkCommand>/bin/ln -s %%PATH_TO_ORIGINAL%% %%TARGET_NAME%%</linkCommand></localInvocation> 9523bce2-86c7-4caf-a720-709f6ccd877d # Configure migrate_mp3_to_wav_hadoop_JAR_PATH=%%jar_input_path%%/migrate_mp3_to_wav_hadoop-0.1-SNAPSHOT-jar-with-dependencies.jar # Hadoop job hadoop jar ${migrate_mp3_to_wav_hadoop_JAR_PATH} eu.scape_project.audio_qa.mpg321_convert.Mpg321Convert -Dmapred.max.split.size=%%max_split_size%% %%hdfs_input_path%% %%mapreduce_output_path%% %%hdfs_output_path_2%% %%nfs_output_path%% 1200 1800 hdfs_input_path hdfs_output_path_2 jar_input_path mapreduce_output_path max_split_size nfs_output_path max_split_size max_split_size false false false UTF-8 false false false hdfs_input_path hdfs_input_path false false false UTF-8 false false false nfs_output_path nfs_output_path false false false UTF-8 false false false mapreduce_output_path mapreduce_output_path false false false UTF-8 false false false hdfs_output_path_2 hdfs_output_path_2 false false false UTF-8 false false false jar_input_path jar_input_path false false false UTF-8 false false false false true true 0 false net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeGetResultsFromHadoopJobmapreduce_output_path0STDERR00STDOUT00net.sf.taverna.t2.activitiesexternal-tool-activity1.4net.sf.taverna.t2.activities.externaltool.ExternalToolActivity 789663B8-DA91-428A-9F7D-B3F3DA185FD4 default local <?xml version="1.0" encoding="UTF-8"?> <localInvocation><shellPrefix>/bin/sh -c</shellPrefix><linkCommand>/bin/ln -s %%PATH_TO_ORIGINAL%% %%TARGET_NAME%%</linkCommand></localInvocation> 9523bce2-86c7-4caf-a720-709f6ccd877d # Read HDFS Hadoop job output hadoop fs -cat %%mapreduce_output_path%%/part-r-00000 1200 1800 mapreduce_output_path mapreduce_output_path mapreduce_output_path false false false UTF-8 false false false false true true 0 false net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize 1 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry 1.0 1000 5000 0 net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeMpg321ConvertHadoopJobhdfs_input_pathmp3_list_on_hdfs_input_pathMpg321ConvertHadoopJobmapreduce_output_pathmapreduce_output_pathMpg321ConvertHadoopJobhdfs_output_path_2hdfs_output_path_2Mpg321ConvertHadoopJobnfs_output_pathnfs_output_pathMpg321ConvertHadoopJobjar_input_pathjar_input_pathMpg321ConvertHadoopJobmax_split_sizemax_split_sizeGetResultsFromHadoopJobmapreduce_output_pathmapreduce_output_pathHadoopJob_STDOUTMpg321ConvertHadoopJobSTDOUTHadoopJob_STDERRMpg321ConvertHadoopJobSTDERRGetResultsFromHadoopJob_STDERRGetResultsFromHadoopJobSTDERRGetResultsFromHadoopJob_STDOUTGetResultsFromHadoopJobSTDOUT 75d877e4-d973-43fd-aee8-013f9724f918 2014-01-14 10:47:27.703 UTC 3b0c8bd3-a022-4e1e-9f96-d1456a9c9921 2014-02-11 13:42:20.515 UTC 18331161-9134-4992-826d-0c61a145fa2a 2014-02-13 10:36:45.35 UTC 3629a1cb-67ca-4ccf-b3ae-960c6f29543d 2014-01-30 15:19:13.424 UTC 6c962801-9103-47c0-b495-bbb0cbfbfded 2014-02-26 10:02:21.322 UTC b3b071bb-6a05-4e22-804d-3ee6b160cfa6 2014-01-30 15:15:18.104 UTC a6a4362d-ffef-429c-b1f4-517c937cbc3a 2014-01-30 15:15:10.184 UTC Mpg321Convert Taverna Workflow using Mpg321Convert Hadoob Job to convert a list of mp3 files to wav files. 2014-02-13 12:38:02.24 UTC 85f11a4f-95f6-47db-ac89-118919873378 2014-02-26 14:19:18.787 UTC 33b9466a-2cdb-42b5-b9a8-8a9e4b49ed1c 2014-01-31 07:56:06.362 UTC 92bb3da2-f8f4-446d-9a52-5b33c113340e 2014-02-13 10:47:40.857 UTC 6755a2ed-2aa1-485a-9e11-aa6170587ee5 2014-02-13 12:42:07.29 UTC 1ef352ba-0251-4ab0-8d2e-901142ac08f3 2014-01-14 10:21:04.29 UTC Bolette A. Jurik, Statsbiblioteket & SCAPE 2014-01-14 10:46:02.398 UTC