Text preprocessing

Created: 2011-01-07 16:13:58 Last updated: 2011-01-07 16:17:13

Download Workflow

The input to this workflow is plain text. The text is preprocessed so that non- alfanumeric symbols are removed, the text is transformed to to lower case and stop words are removed.

The workflow first removes the charachters from this set: `~!@#$%^&*()_+=-{}|\][":;'?><,./.

Then it transforms the text to lower case. The user will be prompted to select a dictionary for stop words from a list. The workflow will, based on the selected list, remove the stop words.
Stop words are words that do not carry meaning, like, the, an,... The web service for stop words removal integrates six English stop words dictionaries and one for the Slovenian language.

The output of the workflow is text in lower case without non-alfanumeric charachters and without stop words.

Preview

Download as scalable diagram (SVG)

Run

Run this Workflow in the Taverna Workbench...

Option 1:

Copy and paste this link into File > 'Open workflow location...'
http://myexperiment.org/workflows/1750/download?version=1
[ More Info Expand ]

Workflow Components

Authors (1)

Titles (1)

Descriptions (1)

Dependencies (0)

Inputs (1)

Name	Description
Text	Plain text.

Processors (10)

Name	Type	Description
removeStopwords	wsdl	Wsdl http://zulu.ijs.si:8086/SW_service?wsdl Wsdl Operation removeStopwords
removeStopwords_input	xmlsplitter
Workflow1	workflow
wsdl_value	stringconstant	Value http://zulu.ijs.si:8086/SW_service?wsdl
VariableName_value	stringconstant	Value stopwordsCollectioID
removeStopwords_output	xmlsplitter
Beanshell	beanshell	Script LowerText = text.toLowerCase();
filterText	wsdl	Wsdl http://zulu.ijs.si:8086/TF_service?wsdl Wsdl Operation filterText
filterText_input	xmlsplitter
unwantedCharacters_value	stringconstant	Value `~!@#$%^&*()_+=-{}\|\][":;'?><,./

Beanshells (1)

Name	Description	Inputs	Outputs
Beanshell		text	LowerText

Outputs (1)

Name	Description
TextWithoutStopWords

Datalinks (11)

Source	Sink
removeStopwords_input:output	removeStopwords:parameters
Workflow1:Selection	removeStopwords_input:stopwordsCollectioID
Beanshell:LowerText	removeStopwords_input:inputText
wsdl_value:value	Workflow1:wsdl
VariableName_value:value	Workflow1:VariableName
removeStopwords:parameters	removeStopwords_output:input
filterText:parameters	Beanshell:text
filterText_input:output	filterText:parameters
Text	filterText_input:inputText
unwantedCharacters_value:value	filterText_input:unwantedCharacters
removeStopwords_output:filteredText	TextWithoutStopWords

Coordinations (0)

Information Workflow Type

Taverna 2

Information Uploader

Petra Kralj Novak

Information License

All versions of this Workflow are licensed under:

Information Version 1 (of 1)

Information Credits (1)

(People/Groups)

Petra Kralj Novak

Information Attributions (0)

(Workflows/Files)

None

Information Tags (3)

Uploader tags

Log in to add Tags

Information Shared with Groups (1)

e-LICO

Information Featured In Packs (0)

None

Log in to add to one of your Packs

Information Attributed By (0)

(Workflows/Files)

None

Information Favourited By (0)

No one

Information Statistics

1487 viewings

1071 downloads

[ see breakdown ]

Citations (0)

None

Version History

In chronological order:

Text preprocessing

Created by Petra Kralj Novak on Friday 07 January 2011 16:13:58 (UTC)

Last edited by Petra Kralj Novak on Friday 07 January 2011 16:17:13 (UTC)

Reviews (0)

No reviews yet

Be the first to review!

Comments (0)

View Timeline

No comments yet

Log in to make a comment

Other workflows that use similar services (2)

Taverna 2

Uploader

Petra Kralj Novak

Lemmatization (3)

Download

The workflow lemmatizes the text in the input port. Takes text as input and returns (language dependent) lemmatized text as output. All the words in the resulting text are in the same order as in the original text, but they are transformed to their dictionary form. The workflow asks for the language of lemmatization. Currently, 12 languages are supported: en,sl,ge,bg,cs,et,fr,hu,ro,sr,it,sp.

Created: 2010-12-17 | Last updated: 2010-12-23

Credits: Petra Kralj Novak

Attributions: Select from a list of possible web service parameter values

Taverna 2

Uploader

Petra Kralj Novak

Select from a list of possible web service... (1)

Download

The workflow for selecting from a list of possible web service parameter values has two input ports: the wsdl address of the web service and the variable name. It parses the web service wsdl description (the web service http://ropot.ijs.si/webservices/janez/getvalues.php?wsdl does that) and then it asks the user to select one value from a drop-down menu. This workflow is very useful when web services have inputs which expect as a parameter one value from a list of possible values.

Created: 2010-12-23 | Last updated: 2010-12-23

Credits: Petra Kralj Novak Janez Kranjc

Text preprocessing

Preview

Run

Run this Workflow in the Taverna Workbench...

Workflow Components

Wsdl

Wsdl Operation

Value

Value

Script

Wsdl

Wsdl Operation

Value

Reviews (0)

Comments (0)

Other workflows that use similar services (2)