Syntetic population mapper

Created: 2010-06-23 13:12:41 Last updated: 2010-12-09 10:43:38

Download Workflow

This workflow takes a csv file with census data, runs the PRM over it and the resulting csv is visualsed using the MapTube software.

Preview

Download as scalable diagram (SVG)

Run

Run this Workflow in the Taverna Workbench...

Option 1:

Copy and paste this link into File > 'Open workflow location...'
http://myexperiment.org/workflows/1353/download?version=1
[ More Info Expand ]

Workflow Components

Authors (0)

Titles (0)

Descriptions (0)

Dependencies (0)

Inputs (5)

Name	Description
data_url	PRM (Population Reconstruction Model) - takes a csv file with census data and generates another csv file that contains URL to the census data PRM (Population Reconstruction Model) - takes a csv file with census data and generates another csv file that contains PRM
iterationPerArea	Number of iterations the service will try to make realistic population in each area.
numTestRecords	The number of unique individuals to include in the final population.
sarRecords	The number of unique individuals to include in the final population.
fieldToAggregate	Field from csv file to aggregate

Processors (23)

Name	Type	Description
runPRMWithData	wsdl	Wsdl http://geo-s12.leeds.ac.uk:8080/NeissModel_WS/ModelService?wsdl Wsdl Operation runPRMWithData
runPRMWithData_input	xmlsplitter
runBHPSLinker	wsdl	Wsdl http://geo-s12.leeds.ac.uk:8080/NeissModel_WS/ModelService?wsdl Wsdl Operation runBHPSLinker
runBHPSLinker_input	xmlsplitter
runPRMWithData_output	xmlsplitter
year	stringconstant	Value 2001
runBHPSLinker_output	xmlsplitter
XPath_From_Text	localworker	Script import org.dom4j.Document; import org.dom4j.Node; import org.dom4j.io.SAXReader; SAXReader reader = new SAXReader(false); reader.setIncludeInternalDTDDeclarations(false); reader.setIncludeExternalDTDDeclarations(false); Document document = reader.read(new StringReader(xmltext)); List nodelist = document.selectNodes(xpath); // Process the elements in the nodelist ArrayList outputList = new ArrayList(); ArrayList outputXmlList = new ArrayList(); String val = null; String xmlVal = null; for (Iterator iter = nodelist.iterator(); iter.hasNext();) { Node element = (Node) iter.next(); xmlVal = element.asXML(); val = element.getStringValue(); if (val != null && !val.equals("")) { outputList.add(val); outputXmlList.add(xmlVal); } } List nodelist=outputList; List nodelistAsXML=outputXmlList;
xpath_to_extract_region_name	stringconstant	Value /neiss_xml/results_data/data_element/result_url
xpath_to_extract_csv_url	stringconstant	Value /neiss_xml/results_data/data_element/result_url
XPath_From_Text_2	localworker	Script import org.dom4j.Document; import org.dom4j.Node; import org.dom4j.io.SAXReader; SAXReader reader = new SAXReader(false); reader.setIncludeInternalDTDDeclarations(false); reader.setIncludeExternalDTDDeclarations(false); Document document = reader.read(new StringReader(xmltext)); List nodelist = document.selectNodes(xpath); // Process the elements in the nodelist ArrayList outputList = new ArrayList(); ArrayList outputXmlList = new ArrayList(); String val = null; String xmlVal = null; for (Iterator iter = nodelist.iterator(); iter.hasNext();) { Node element = (Node) iter.next(); xmlVal = element.asXML(); val = element.getStringValue(); if (val != null && !val.equals("")) { outputList.add(val); outputXmlList.add(xmlVal); } } List nodelist=outputList; List nodelistAsXML=outputXmlList;
Get_Web_Page_from_URL	localworker	Script URL inputURL = null; if (base != void) { inputURL = new URL(new URL(base), url); } else { inputURL = new URL(url); } URLConnection con = inputURL.openConnection(); InputStream in = con.getInputStream(); InputStreamReader isr = new InputStreamReader(in); Reader inReader = new BufferedReader(isr); StringBuffer buf = new StringBuffer(); int ch; while ((ch = inReader.read()) > -1) { buf.append((char)ch); } inReader.close(); contents = buf.toString(); //String NEWLINE = System.getProperty("line.separator"); // //URL inputURL = null; //if (base != void) { // inputURL = new URL(new URL(base), url); //} else { // inputURL = new URL(url); //} //StringBuffer result = new StringBuffer(); //BufferedReader reader = new BufferedReader(new InputStreamReader(inputURL.openStream())); //String line = null; //while ((line = reader.readLine()) != null) { // result.append(line); // result.append(NEWLINE); //} // //contents = result.toString();
Extract_First_Item_From_List	localworker	Script if (!stringList.isEmpty()){ firstItem = stringList.get(0); } else{ firstItem = null; }
Extract_First_Item_From_List_2	localworker	Script if (!stringList.isEmpty()){ firstItem = stringList.get(0); } else{ firstItem = null; }
runMapper	wsdl	Wsdl http://geo-s12.leeds.ac.uk:8080/NeissModel_WS/ModelService?wsdl Wsdl Operation runMapper
runMapper_input	xmlsplitter
runMapper_output	xmlsplitter
geometry_value	stringconstant	Value OA
areaKey_value	stringconstant	Value OA_Code
dataValues_value	stringconstant	Value Gender_1
XPath_From_Text_3	localworker	Script import org.dom4j.Document; import org.dom4j.Node; import org.dom4j.io.SAXReader; SAXReader reader = new SAXReader(false); reader.setIncludeInternalDTDDeclarations(false); reader.setIncludeExternalDTDDeclarations(false); Document document = reader.read(new StringReader(xmltext)); List nodelist = document.selectNodes(xpath); // Process the elements in the nodelist ArrayList outputList = new ArrayList(); ArrayList outputXmlList = new ArrayList(); String val = null; String xmlVal = null; for (Iterator iter = nodelist.iterator(); iter.hasNext();) { Node element = (Node) iter.next(); xmlVal = element.asXML(); val = element.getStringValue(); if (val != null && !val.equals("")) { outputList.add(val); outputXmlList.add(xmlVal); } } List nodelist=outputList; List nodelistAsXML=outputXmlList;
xpath_to_extract_map_url	stringconstant	Value /neiss_xml/results_data/data_element/result_url
Extract_First_Item_From_List_3	localworker	Script if (!stringList.isEmpty()){ firstItem = stringList.get(0); } else{ firstItem = null; }

Beanshells (0)

Outputs (7)

Name	Description
output
PRM_xml
csv_url
region_name
csv_file
mapper_xml
map_url

Datalinks (34)

Source	Sink
runPRMWithData_input:output	runPRMWithData:parameters
data_url	runPRMWithData_input:dataURL
iterationPerArea	runPRMWithData_input:iterPerArea
numTestRecords	runPRMWithData_input:numTestRecords
sarRecords	runPRMWithData_input:sarRecords
runBHPSLinker_input:output	runBHPSLinker:parameters
fieldToAggregate	runBHPSLinker_input:fieldToAggregate
year:value	runBHPSLinker_input:year
Extract_First_Item_From_List:firstItem	runBHPSLinker_input:region
runPRMWithData:parameters	runPRMWithData_output:input
runBHPSLinker:parameters	runBHPSLinker_output:input
runPRMWithData_output:return	XPath_From_Text:xml-text
xpath_to_extract_region_name:value	XPath_From_Text:xpath
xpath_to_extract_csv_url:value	XPath_From_Text_2:xpath
runBHPSLinker_output:return	XPath_From_Text_2:xml-text
Extract_First_Item_From_List_2:firstItem	Get_Web_Page_from_URL:url
XPath_From_Text:nodelist	Extract_First_Item_From_List:stringList
XPath_From_Text_2:nodelist	Extract_First_Item_From_List_2:stringList
runMapper_input:output	runMapper:parameters
Extract_First_Item_From_List_2:firstItem	runMapper_input:csvURL
geometry_value:value	runMapper_input:geometry
areaKey_value:value	runMapper_input:areaKey
dataValues_value:value	runMapper_input:dataValues
runMapper:parameters	runMapper_output:input
xpath_to_extract_map_url:value	XPath_From_Text_3:xpath
runMapper_output:return	XPath_From_Text_3:xml-text
XPath_From_Text_3:nodelist	Extract_First_Item_From_List_3:stringList
runBHPSLinker_output:return	output
runPRMWithData_output:return	PRM_xml
Extract_First_Item_From_List_2:firstItem	csv_url
Extract_First_Item_From_List:firstItem	region_name
Get_Web_Page_from_URL:contents	csv_file
runMapper_output:return	mapper_xml
Extract_First_Item_From_List_3:firstItem	map_url

Coordinations (1)

Controller	Target
runPRMWithData	runBHPSLinker

Information Workflow Type

Taverna 2

Information Uploader

Alex Nenadic

Information License

All versions of this Workflow are licensed under:

Information Version 1 (earliest) (of 3)

Information Credits (1)

(People/Groups)

Alex Nenadic

Information Attributions (0)

(Workflows/Files)

None

Information Tags (0)

None

Log in to add Tags

Information Shared with Groups (1)

NeISS

Information Featured In Packs (0)

None

Log in to add to one of your Packs

Information Attributed By (0)

(Workflows/Files)

None

Information Favourited By (1)

Chrisser

Information Statistics

1630 viewings

1196 downloads

[ see breakdown ]

Citations (0)

None

Version History

In chronological order:

PRM and MapTube Integration Workflow

Created by Alex Nenadic on Wednesday 23 June 2010 13:12:41 (UTC)

Last edited by Alex Nenadic on Wednesday 23 June 2010 13:14:44 (UTC)
[untitled]

Created by Nick Malleson on Tuesday 21 September 2010 13:53:23 (UTC)

Last edited by Nick Malleson on Tuesday 21 September 2010 13:55:31 (UTC)

Revision comment:

Slightly simplified by replacing runPRMWithData service with runPRMWithDataSimple
Syntetic population mapper

Created by Nick Malleson on Tuesday 21 September 2010 16:05:28 (UTC)

Last edited by Alex Nenadic on Thursday 09 December 2010 10:43:38 (UTC)

Revision comment:

Simplified the mapper service, fewer constants required.