Get Recent Cards From Indexed
Created: 2011-09-13 02:48:57
Last updated: 2011-09-13 02:48:59
This gets the jpgs from the "This is Indexed" URL: http://thisisindexed.com/page/2/ using the regular expression .*/uploads/*. pull everything in the "uploads" directory.
Based on "Fetch today's xkcd comic" by Tom Oinn, Stian Soiland-Reyes
Use the local java plugins and some filtering operations to fetch the comic strip image from http://xkcd.com/
Preview
Run
Run this Workflow in the Taverna Workbench...
Workflow Components
Authors (1)
Titles (1)
Get Recent Cards From Indexed |
Descriptions (1)
This gets the jpgs from the "This is Indexed" URL: http://thisisindexed.com/page/2/ using the regular expression .*/uploads/*. pull everything in the "uploads" directory.
Based on "Fetch today's xkcd comic" by Tom Oinn, Stian Soiland-Reyes
Use the local java plugins and some filtering operations to fetch the comic strip image from http://xkcd.com/ |
Dependencies (0)
Processors (6)
Name |
Type |
Description |
getPage |
localworker |
ScriptURL inputURL = null;
if (base != void) {
inputURL = new URL(new URL(base), url);
}
else {
inputURL = new URL(url);
}
URLConnection con = inputURL.openConnection();
InputStream in = con.getInputStream();
InputStreamReader isr = new InputStreamReader(in);
Reader inReader = new BufferedReader(isr);
StringBuffer buf = new StringBuffer();
int ch;
while ((ch = inReader.read()) > -1) {
buf.append((char)ch);
}
inReader.close();
contents = buf.toString();
//String NEWLINE = System.getProperty("line.separator");
//
//URL inputURL = null;
//if (base != void) {
// inputURL = new URL(new URL(base), url);
//} else {
// inputURL = new URL(url);
//}
//StringBuffer result = new StringBuffer();
//BufferedReader reader = new BufferedReader(new InputStreamReader(inputURL.openStream()));
//String line = null;
//while ((line = reader.readLine()) != null) {
// result.append(line);
// result.append(NEWLINE);
//}
//
//contents = result.toString();
|
IndexedURL |
stringconstant |
Valuehttp://thisisindexed.com/page/2/ |
getImageLinks |
localworker |
ScriptString lowerCaseContent = document.toLowerCase();
int index = 0;
List imagelinks = new ArrayList();
while ((index = lowerCaseContent.indexOf("#");
String strLink = st.nextToken();
imagelinks.add(strLink);
}
|
findComicURL |
localworker |
Scriptfilteredlist = new ArrayList();
for (Iterator i = stringlist.iterator(); i.hasNext();) {
String item = (String) i.next();
if (item.matches(regex)) {
filteredlist.add(item);
}
}
|
comicURLRegex |
stringconstant |
Value.*/uploads/.* |
getComicStrip |
localworker |
ScriptURL inputURL = null;
if (base != void) {
inputURL = new URL(new URL(base), url);
} else {
inputURL = new URL(url);
}
byte[] contents;
if (inputURL.openConnection().getContentLength() == -1) {
// Content size unknown, must read first...
byte[] buffer = new byte[1024];
int bytesRead = 0;
int totalBytesRead = 0;
InputStream is = inputURL.openStream();
while (bytesRead != -1) {
totalBytesRead += bytesRead;
bytesRead = is.read(buffer, 0, 1024);
}
contents = new byte[totalBytesRead];
} else {
contents = new byte[inputURL.openConnection().getContentLength()];
}
int bytesRead = 0;
int totalBytesRead = 0;
InputStream is = inputURL.openStream();
while (bytesRead != -1) {
bytesRead = is.read(contents, totalBytesRead, contents.length - totalBytesRead);
totalBytesRead += bytesRead;
if (contents.length==totalBytesRead) break;
}
image = contents;
|
Outputs (1)
Name |
Description |
RecentIndexed |
|
Datalinks (7)
Source |
Sink |
IndexedURL:value |
getPage:url |
getPage:contents |
getImageLinks:document |
comicURLRegex:value |
findComicURL:regex |
getImageLinks:imagelinks |
findComicURL:stringlist |
IndexedURL:value |
getComicStrip:base |
findComicURL:filteredlist |
getComicStrip:url |
getComicStrip:image |
RecentIndexed |
Uploader
License
All versions of this Workflow are
licensed under:
Version 1
(of 1)
Credits (0)
(People/Groups)
None
Attributions (0)
(Workflows/Files)
None
Shared with Groups (0)
None
Featured In Packs (0)
None
Log in to add to one of your Packs
Attributed By (0)
(Workflows/Files)
None
Favourited By (0)
No one
Statistics
Other workflows that use similar services
(0)
There are no workflows in myExperiment that use similar services to this Workflow.
Comments (0)
No comments yet
Log in to make a comment