Syntetic population mapper

Created: 2010-06-23 13:12:41 Last updated: 2010-12-09 10:43:38

Download Workflow

This workflow creates individual-level population from census data and then joins with the BHPS (British Household Panel Survey) and reaggregates. In the last stage it maps the chosen BHPS field.

Preview

Download as scalable diagram (SVG)

Run

Run this Workflow in the Taverna Workbench...

Option 1:

Copy and paste this link into File > 'Open workflow location...'
http://myexperiment.org/workflows/1353/download?version=3
[ More Info Expand ]

Workflow Components

Authors (0)

Titles (0)

Descriptions (0)

Dependencies (0)

Inputs (4)

Name	Description
fieldToAggregate	Field from csv file to aggregate
dataURL	The URL of the census data
regionName	The name of the region
FieldToMap

Processors (20)

Name	Type	Description
runBHPSLinker	wsdl	Wsdl http://geo-s12.leeds.ac.uk:8080/NeissModel_WS/ModelService?wsdl Wsdl Operation runBHPSLinker
runBHPSLinker_input	xmlsplitter
year	stringconstant	Value 2001
runBHPSLinker_output	xmlsplitter
XPath_From_Text	localworker	Script import org.dom4j.Document; import org.dom4j.Node; import org.dom4j.io.SAXReader; SAXReader reader = new SAXReader(false); reader.setIncludeInternalDTDDeclarations(false); reader.setIncludeExternalDTDDeclarations(false); Document document = reader.read(new StringReader(xmltext)); List nodelist = document.selectNodes(xpath); // Process the elements in the nodelist ArrayList outputList = new ArrayList(); ArrayList outputXmlList = new ArrayList(); String val = null; String xmlVal = null; for (Iterator iter = nodelist.iterator(); iter.hasNext();) { Node element = (Node) iter.next(); xmlVal = element.asXML(); val = element.getStringValue(); if (val != null && !val.equals("")) { outputList.add(val); outputXmlList.add(xmlVal); } } List nodelist=outputList; List nodelistAsXML=outputXmlList;
xpath_to_extract_region_name	stringconstant	Value /neiss_xml/results_data/data_element/result_url
xpath_to_extract_csv_url	stringconstant	Value /neiss_xml/results_data/data_element/result_url
XPath_From_Text_2	localworker	Script import org.dom4j.Document; import org.dom4j.Node; import org.dom4j.io.SAXReader; SAXReader reader = new SAXReader(false); reader.setIncludeInternalDTDDeclarations(false); reader.setIncludeExternalDTDDeclarations(false); Document document = reader.read(new StringReader(xmltext)); List nodelist = document.selectNodes(xpath); // Process the elements in the nodelist ArrayList outputList = new ArrayList(); ArrayList outputXmlList = new ArrayList(); String val = null; String xmlVal = null; for (Iterator iter = nodelist.iterator(); iter.hasNext();) { Node element = (Node) iter.next(); xmlVal = element.asXML(); val = element.getStringValue(); if (val != null && !val.equals("")) { outputList.add(val); outputXmlList.add(xmlVal); } } List nodelist=outputList; List nodelistAsXML=outputXmlList;
Get_Web_Page_from_URL	localworker	Script URL inputURL = null; if (base != void) { inputURL = new URL(new URL(base), url); } else { inputURL = new URL(url); } URLConnection con = inputURL.openConnection(); InputStream in = con.getInputStream(); InputStreamReader isr = new InputStreamReader(in); Reader inReader = new BufferedReader(isr); StringBuffer buf = new StringBuffer(); int ch; while ((ch = inReader.read()) > -1) { buf.append((char)ch); } inReader.close(); contents = buf.toString(); //String NEWLINE = System.getProperty("line.separator"); // //URL inputURL = null; //if (base != void) { // inputURL = new URL(new URL(base), url); //} else { // inputURL = new URL(url); //} //StringBuffer result = new StringBuffer(); //BufferedReader reader = new BufferedReader(new InputStreamReader(inputURL.openStream())); //String line = null; //while ((line = reader.readLine()) != null) { // result.append(line); // result.append(NEWLINE); //} // //contents = result.toString();
Extract_First_Item_From_List	localworker	Script if (!stringList.isEmpty()){ firstItem = stringList.get(0); } else{ firstItem = null; }
Extract_First_Item_From_List_2	localworker	Script if (!stringList.isEmpty()){ firstItem = stringList.get(0); } else{ firstItem = null; }
XPath_From_Text_3	localworker	Script import org.dom4j.Document; import org.dom4j.Node; import org.dom4j.io.SAXReader; SAXReader reader = new SAXReader(false); reader.setIncludeInternalDTDDeclarations(false); reader.setIncludeExternalDTDDeclarations(false); Document document = reader.read(new StringReader(xmltext)); List nodelist = document.selectNodes(xpath); // Process the elements in the nodelist ArrayList outputList = new ArrayList(); ArrayList outputXmlList = new ArrayList(); String val = null; String xmlVal = null; for (Iterator iter = nodelist.iterator(); iter.hasNext();) { Node element = (Node) iter.next(); xmlVal = element.asXML(); val = element.getStringValue(); if (val != null && !val.equals("")) { outputList.add(val); outputXmlList.add(xmlVal); } } List nodelist=outputList; List nodelistAsXML=outputXmlList;
xpath_to_extract_map_url	stringconstant	Value /neiss_xml/results_data/data_element/result_url
Extract_First_Item_From_List_3	localworker	Script if (!stringList.isEmpty()){ firstItem = stringList.get(0); } else{ firstItem = null; }
runPRMWithDataSimple	wsdl	Wsdl http://geo-s12.leeds.ac.uk:8080/NeissModel_WS/ModelService?wsdl Wsdl Operation runPRMWithDataSimple
runPRMWithDataSimple_output	xmlsplitter
runPRMWithDataSimple_input	xmlsplitter
runMapperFromURL	wsdl	Wsdl http://geo-s12.leeds.ac.uk:8080/NeissModel_WS/ModelService?wsdl Wsdl Operation runMapperFromURL
runMapperFromURL_input	xmlsplitter
runMapperFromURL_output	xmlsplitter

Beanshells (0)

Outputs (7)

Name	Description
output
csv_url
region_name
csv_file
mapper_xml
map_url
PRMOutput

Datalinks (30)

Source	Sink
runBHPSLinker_input:output	runBHPSLinker:parameters
fieldToAggregate	runBHPSLinker_input:fieldToAggregate
year:value	runBHPSLinker_input:year
Extract_First_Item_From_List:firstItem	runBHPSLinker_input:region
runBHPSLinker:parameters	runBHPSLinker_output:input
xpath_to_extract_region_name:value	XPath_From_Text:xpath
runPRMWithDataSimple_output:return	XPath_From_Text:xml-text
xpath_to_extract_csv_url:value	XPath_From_Text_2:xpath
runBHPSLinker_output:return	XPath_From_Text_2:xml-text
Extract_First_Item_From_List_2:firstItem	Get_Web_Page_from_URL:url
XPath_From_Text:nodelist	Extract_First_Item_From_List:stringList
XPath_From_Text_2:nodelist	Extract_First_Item_From_List_2:stringList
xpath_to_extract_map_url:value	XPath_From_Text_3:xpath
runMapperFromURL_output:return	XPath_From_Text_3:xml-text
XPath_From_Text_3:nodelist	Extract_First_Item_From_List_3:stringList
runPRMWithDataSimple_input:output	runPRMWithDataSimple:parameters
runPRMWithDataSimple:parameters	runPRMWithDataSimple_output:input
dataURL	runPRMWithDataSimple_input:dataURL
regionName	runPRMWithDataSimple_input:regionName
runMapperFromURL_input:output	runMapperFromURL:parameters
Extract_First_Item_From_List_2:firstItem	runMapperFromURL_input:dataURL
FieldToMap	runMapperFromURL_input:fieldToMap
runMapperFromURL:parameters	runMapperFromURL_output:input
runBHPSLinker_output:return	output
Extract_First_Item_From_List_2:firstItem	csv_url
Extract_First_Item_From_List:firstItem	region_name
Get_Web_Page_from_URL:contents	csv_file
runMapperFromURL_output:return	mapper_xml
Extract_First_Item_From_List_3:firstItem	map_url
runPRMWithDataSimple_output:return	PRMOutput

Coordinations (1)

Controller	Target
runPRMWithDataSimple	runBHPSLinker

Information Workflow Type

Taverna 2

Information Uploader

Alex Nenadic

Information License

All versions of this Workflow are licensed under:

Information Version 3 (latest) (of 3)

Information Credits (1)

(People/Groups)

Alex Nenadic

Information Attributions (0)

(Workflows/Files)

None

Information Tags (0)

None

Log in to add Tags

Information Shared with Groups (1)

NeISS

Information Featured In Packs (0)

None

Log in to add to one of your Packs

Information Attributed By (0)

(Workflows/Files)

None

Information Favourited By (1)

Chrisser

Information Statistics

1630 viewings

1196 downloads

[ see breakdown ]

Citations (0)

None

Version History

In chronological order:

PRM and MapTube Integration Workflow

Created by Alex Nenadic on Wednesday 23 June 2010 13:12:41 (UTC)

Last edited by Alex Nenadic on Wednesday 23 June 2010 13:14:44 (UTC)
[untitled]

Created by Nick Malleson on Tuesday 21 September 2010 13:53:23 (UTC)

Last edited by Nick Malleson on Tuesday 21 September 2010 13:55:31 (UTC)

Revision comment:

Slightly simplified by replacing runPRMWithData service with runPRMWithDataSimple
Syntetic population mapper

Created by Nick Malleson on Tuesday 21 September 2010 16:05:28 (UTC)

Last edited by Alex Nenadic on Thursday 09 December 2010 10:43:38 (UTC)

Revision comment:

Simplified the mapper service, fewer constants required.