Data Refinement Workflow v17

Created: 2012-04-11 10:08:25 Last updated: 2014-12-17 12:51:15

Download Workflow

The aim of the (Taxonomic) Data Refinement Workflow is to provide a streamlined workflow environment for preparing observational and specimen data sets for use in scientific analysis on the Taverna platform. The workflow has been designed in a way that, • accepts input data in a recognized format, but originating from various sources (e.g. services, local user data sets), • includes a number of graphical user interfaces to view and interact with the data, • the output of each part of the workflow is compatible with the input of each part, implying that the user is free to choose a specific sequence of actions, • allows for the use of custom-built as well as third-party tools applications and tools.

Currently, the data refinement workflow is made up of three distinct parts:

1.Taxonomic Name Resolution / Occurrence retrieval 2.Geo-temporal data selection 3.Data quality checks / filtering

Preview

Download as scalable diagram (SVG)

Run

Run this Workflow in the Taverna Workbench...

Option 1:

Copy and paste this link into File > 'Open workflow location...'
http://www.myexperiment.org/workflows/2874/download?version=15
[ More Info Expand ]

Workflow Components

Authors (1)

Titles (1)

Descriptions (1)

Dependencies (1)

Inputs (0)

Processors (3)

Name	Type	Description
Data_Cleaning_Worklow_Loop	workflow	Nested workflow which allows the user to plug in the output of any part of the workflow to the input of any other part of the workflow until the 'end workflow' choice is made.
InputFileChooser	interaction	Interaction Page http://ww2.bgbm.org/biovel/public/webui/1.0.3/InputChooser.html
FirstElement_Depth1	beanshell	Script if(stringlist.size() > 0) { firstelement = stringlist.get(0); } else { firstelement = ""; }

Beanshells (27)

Name	Inputs	Outputs
FirstElement_Depth1	stringlist	firstelement
BGBMEDITChecklist	inputXMLList checklistInfo	outputXML
Export_Data_Options	projectID refineServer refinePort	exportedCSV
ParseCSV	csv	inputXMLList
MergeTNRList	tnrXMLList	tnrMergedXML
CollectChecklistInfo		checklistJson
ConvertToCIList	checklistsJson inputXMLList	colXMLList editChecklistInfo editXMLList gbifChecklistInfo gbifXMLList pesiXMLList
OccurrenceBankInfo	names	occJson sciNamesList
ConvertToOBList	occurrencesJson sciNamesList	gbifNamesList
getOccCSVHeader		csvHeader
NamesChecker	names	csv_output names
Intermediate_Output	grefineCSVList biostifCSVList tnrList endWFCSVList	intermediateCSV
GetCSVData	internalCSVData	csvData emptyData
End_Workflow	csvData	csv_output
passthrough	csvin	csvout
species2000ColChecklist	inputXML	outputXML
Export_Data_Conditional	gref_answer	save_true cancel_true
FirstElement_Depth1	stringlist	firstelement
Upload_Data	refineServer refinePort csvData	projectID
Delete_Project	refineServer refinePort projectID
FirstElement_Depth1	stringlist	firstelement
CreateSubLists	inputXMLList	outputXMLSubLists
pesiChecklist	inputXML	outputXML
QueryGBIFOccurrenceBank	name	occurrenceRecords
trimRESTurlResult	url	resultUrl
checkDataUpload	status	dataUpload_ok dataUpload_failed uploadStatus
GBIFBetaChecklist	inputXML checklistInfo	outputXML

Outputs (2)

Name	Description
csv_output	Output of the workflow in CSV format.
endWFlow

Datalinks (4)

Source	Sink
InputFileChooser:file_contents	Data_Cleaning_Worklow_Loop:internalCSVData
Data_Cleaning_Worklow_Loop:internalCSVData	FirstElement_Depth1:stringlist
FirstElement_Depth1:firstelement	csv_output
Data_Cleaning_Worklow_Loop:endWFlow	endWFlow

Coordinations (0)

Information Workflow Type

Taverna 2

Information Uploader

Cherian Mathew

Information License

All versions of this Workflow are licensed under:

Information Version 15 (of 17)

View version:

Information Credits (1)

(People/Groups)

Cherian Mathew

Information Attributions (0)

(Workflows/Files)

None

Information Tags (21)

Uploader tags

Log in to add Tags

Information Shared with Groups (2)

Information Featured In Packs (1)

Private pack

Log in to add to one of your Packs

Information Attributed By (0)

(Workflows/Files)

None

Information Favourited By (1)

Pgrobe

Information Statistics

12160 viewings

2620 downloads

[ see breakdown ]

Citations (0)

None

Version History

In chronological order:

Taxonomic Data Refinement (Integrated) Workflow

Created by Cherian Mathew on Wednesday 11 April 2012 10:08:25 (UTC)

Last edited by Cherian Mathew on Wednesday 11 April 2012 10:09:19 (UTC)
Taxonomic Data Refinement (Integrated) Workflow

Created by Cherian Mathew on Wednesday 13 June 2012 10:31:04 (UTC)

Last edited by Cherian Mathew on Wednesday 13 June 2012 10:32:17 (UTC)

Revision comment:

Integrated new version of Synonym Expansion / Occurrence Retrieval part of the workflow
Taxonomic Data Refinement (Integrated) Workflow

Created by Cherian Mathew on Tuesday 26 June 2012 12:47:53 (UTC)
Taxonomic Data Refinement (Integrated) Workflow

Created by Cherian Mathew on Tuesday 26 June 2012 13:22:41 (UTC)

Last edited by Cherian Mathew on Thursday 28 June 2012 08:00:16 (UTC)

Revision comment:

Added new version of BioSTIF workflow which call the web service on http, removing the need for the credentials in Taverna.
Taxonomic Data Refinement (Integrated) Workflow

Created by Cherian Mathew on Thursday 26 July 2012 14:50:56 (UTC)

Last edited by Cherian Mathew on Thursday 26 July 2012 14:54:16 (UTC)

Revision comment:

Added new version of taxonomic name expansion nested workflow
Taxonomic Data Refinement (Integrated) Workflow

Created by Cherian Mathew on Tuesday 07 August 2012 08:14:45 (UTC)

Last edited by Cherian Mathew on Tuesday 07 August 2012 08:16:55 (UTC)
Taxonomic Data Refinement (Integrated) Workflow

Created by Cherian Mathew on Friday 28 September 2012 13:56:56 (UTC)

Last edited by Cherian Mathew on Friday 28 September 2012 13:57:17 (UTC)
Data Refinement Workflow

Created by Cherian Mathew on Thursday 01 November 2012 09:15:21 (UTC)
Data Refinement Workflow

Created by Cherian Mathew on Thursday 08 November 2012 13:12:58 (UTC)
Data Refinement Workflow

Created by Cherian Mathew on Thursday 31 January 2013 09:02:16 (UTC)
[PLUGIN]Data Refinement Workflow

Created by Cherian Mathew on Tuesday 02 April 2013 11:47:04 (UTC)

Revision comment:

This version of the workflow is compatible with the newly created DRF Plugin which installs the dependency jars using the plugin framework of taverna.
Data Refinement Workflow

Created by Cherian Mathew on Tuesday 02 April 2013 11:54:29 (UTC)

Revision comment:

changed title
Data Refinement Workflow v13

Created by Cherian Mathew on Thursday 01 August 2013 13:31:20 (UTC)

Revision comment:

Added checks for Google Refine.
Changed copyright notices to be popups.
Added BGBM EDIT Platform as a target.
Misc. look and feel changes
Data Refinement Workflow v14

Created by Cherian Mathew on Tuesday 04 March 2014 11:13:25 (UTC)

Revision comment:

This new version of the data refinement workflow has:
* Access to the new GBIF API for taxonomic name resolution and for retrieval of occurrence points.
* Integration of PESI name resolution web service.
* Added search filter for aggregated name checklists
* Minor changes in GUI, mainly in the TNRS subworkflow
* Minor bugs fixed
Data Refinement Workflow v15

Created by Biodiversity eLaboratory on Thursday 08 May 2014 12:52:48 (UTC)
Data Refinement Workflow v16

Created by Biodiversity eLaboratory on Tuesday 02 September 2014 08:35:48 (UTC)

Revision comment:

This new version of the data refinement workflow has:
* The incluision of the World Register of Marine Species (WoRMS) as a new checklist to resolve species names in the Taxonomic Name Resolution subworkflow. The WoRMS checklist is queried by using the WoRMS web service created by VLIZ.
* Minor bugs fixed
Data Refinement Workflow v17

Created by Biodiversity eLaboratory on Wednesday 17 December 2014 12:51:15 (UTC)

Revision comment:

Change the BioSTIF services from Fraunhofer to EGI