wf4ever_document_extraction_and_storage

Created: 2013-07-24 14:48:38 Last updated: 2013-08-29 10:50:46

Download Workflow

The workflow uses parts of the existing BioAid workflow by Marco Roos (http://www.myexperiment.org/workflows/74.html) The workflow stores found articles in a Solr database. Make sure that Solr is running in the correct directory (stored in the Solr_directory value).

Preview

Download as scalable diagram (SVG)

Run

Run this Workflow in the Taverna Workbench...

Option 1:

Copy and paste this link into File > 'Open workflow location...'
http://myexperiment.org/workflows/3657/download?version=1
[ More Info Expand ]

Workflow Components

Authors (1)

Titles (1)

Descriptions (1)

Dependencies (0)

Inputs (2)

Name	Description
Query	Fill in a search query, similar to pubmed. For advanced queries look up the Lucene syntax (http://lucene.apache.org/java/2_9_1/queryparsersyntax.html).
maxHits_parameter	Maximum number of documents to extract proteins from. Use <10 for testing, 100 as default, >100 if you want to live dangerously and can wait (may cause memory problems).

Processors (9)

Name	Type	Description
AIDA_Retrieve_documents_in_parts	workflow	The workflow extracts protein names from documents retrieved from MedLine based on a user Query (cf Apache Lucene syntax). Authors: Marco Roos (workflow), Sophia Katrenko, Willem van Hage, Edgar Meij, Martijn Schuemie (services)
document_index_value	stringconstant	Value MedLine
search_field_value	stringconstant	Value content
Lucene_bioquery_optimizer_by_year	workflow	The workflow extracts protein names from documents retrieved from MedLine based on a user Query (cf Apache Lucene syntax). Authors: Marco Roos (workflow), Sophia Katrenko, Willem van Hage, Edgar Meij, Martijn Schuemie (services)
Write_Text_File	localworker	Creates a file from the abstract in .txt format. This is a modified version of the standard Write_Text_File service in Taverna. Script BufferedWriter out; if (encoding == void) { out = new BufferedWriter(new FileWriter(outputFile)); } else { out = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(outputFile), encoding)); } out.write(filecontents); out.flush(); out.close(); outputFile = outputFile;
createFileNames	beanshell	Creates the file location to save the .txt file. The directory is stored in the variable directory. Script outputFile = "%%directory%%" + title + ".txt"
SolrImport	externaltool	This process uses post.jar to put the files in a SOLR database. Make sure that the SOLR server is up and running. The tool accepts all kinds of data formats as long as it is accepted by Solr.
directory	stringconstant	The directory used to store the .txt files. Value /home/sander/
postJar_Directory	stringconstant	This value contains the directory of post.jar that is being used by Solr to import a file. Value /home/sander/Downloads/solr-4.3.1/example/exampledocs/post.jar

Beanshells (2)

Name	Description	Inputs	Outputs
createFileNames	Creates the file location to save the .txt file. The directory is stored in the variable directory.	title directory	outputFile
Prioritise_lucene_query		query_string priority_string	lucene_query

Outputs (7)

Name	Description
pubmed_id	These are the pubmed id's extracted from the BioAid workflow.
title_abstract	These are the abstracts extracted by the BioAID workflow.
pubmed_URL	These are the URL's to Pubmed extracted by the BioAID workflow.
title	These are the titles, extracted by the BioAID workflow.
Write_Text_File_outputFile	This is the path to the .txt file created by the Write_Text_File process.
SolrImport_STDERR	This is the standerdized error of the SolrImport process.
SolrImport_STDOUT	This is the standerdized output of the SolrImport process

Datalinks (18)

Source	Sink
document_index_value:value	AIDA_Retrieve_documents_in_parts:document_index
search_field_value:value	AIDA_Retrieve_documents_in_parts:search_field
maxHits_parameter	AIDA_Retrieve_documents_in_parts:maxHits
Lucene_bioquery_optimizer_by_year:extended_lucene_query	AIDA_Retrieve_documents_in_parts:queryString
Query	Lucene_bioquery_optimizer_by_year:query_string
AIDA_Retrieve_documents_in_parts:abstract	Write_Text_File:filecontents
createFileNames:outputFile	Write_Text_File:outputFile
AIDA_Retrieve_documents_in_parts:pubmed_id	createFileNames:title
directory:value	createFileNames:directory
Write_Text_File:outputFile	SolrImport:inputFile
postJar_Directory:value	SolrImport:directory
AIDA_Retrieve_documents_in_parts:pubmed_id	pubmed_id
AIDA_Retrieve_documents_in_parts:title_abstract	title_abstract
AIDA_Retrieve_documents_in_parts:pubmed_URL	pubmed_URL
AIDA_Retrieve_documents_in_parts:title	title
Write_Text_File:outputFile	Write_Text_File_outputFile
SolrImport:STDERR	SolrImport_STDERR
SolrImport:STDOUT	SolrImport_STDOUT

Coordinations (0)

Information Workflow Type

Taverna 2

Information Uploader

Sander van boom

Information License

All versions of this Workflow are licensed under:

Information Version 1 (of 1)

Information Credits (0)

(People/Groups)

None

Information Attributions (0)

(Workflows/Files)

None

Information Tags (0)

None

Log in to add Tags

Information Shared with Groups (1)

Concept profile generation pipeline

Information Featured In Packs (1)

BioSemantics Concept Profile Generation Workflows

Log in to add to one of your Packs

Information Attributed By (0)

(Workflows/Files)

None

Information Favourited By (0)

No one

Information Statistics

930 viewings

664 downloads

[ see breakdown ]

Citations (0)

None

Version History

In chronological order:

wf4ever_document_extraction_and_storage

Created by Sander van boom on Wednesday 24 July 2013 14:48:37 (UTC)

Reviews (0)

No reviews yet

Be the first to review!

Comments (0)

View Timeline

No comments yet

Log in to make a comment

Other workflows that use similar services (6)

Only the first 2 workflows that use similar services are shown. View all workflows that use these services.

Taverna 1

Uploader

Marco Roos

BioAID_ProteinDiscovery_filterOnHumanUnipr... (11)

Download

This workflow finds proteins relevant to the query string via the following steps: A user query: a single gene/protein name. E.g.: (EZH2 OR "Enhancer of Zeste"). Retrieve documents: finds 'maximumNumberOfHits' relevant documents (abstract+title) based on query (the AIDA service inside is based on Apache's Lucene) Discover proteins: extract proteins discovered in the set of relevant abstracts with a 'named entity recognizer' trained on genomic terms using a Bayesian approach; the AIDA serv...

Created: 2009-05-28

Credits: Marco Roos Martijn Schuemie AID AID_myGrid_collaboration

Attributions: BioAID_DiseaseDiscovery_RatHumanMouseUniprotFilter

Taverna 1

Uploader

Marco Roos

BioAID_ProteinToDiseases (1)

Download

This workflow was based on BioAID_DiseaseDiscovery, changes: expects only one protein name, adds protein synonyms). This workflow finds diseases relevant to the query string via the following steps: A user query: a single protein name Add synonyms (service courtesy of Martijn Scheumie, Erasmus University Rotterdam) Retrieve documents: finds relevant documents (abstract+title) based on query Discover proteins: extract proteins discovered in the set of relevant abstracts 5. Link proteins ...

Created: 2007-11-14 | Last updated: 2007-11-15

Credits: Marco Roos Martijn Schuemie AID

Attributions: BioAID_DiseaseDiscovery_RatHumanMouseUniprotFilter

wf4ever_document_extraction_and_storage

Preview

Run

Run this Workflow in the Taverna Workbench...

Workflow Components

Value

Value

Script

Script

Value

Value

Reviews (0)

Comments (0)

Other workflows that use similar services (6)