wf4ever

Taverna looks at the sky

2012-05-18T10:12:50Z

On-going efforts to improve seamless building of scientific workflows in the Astronomy domain have already produced the first ready-to-install AstroTaverna plugins. Among the most important features are:

Access to Virtual Observatory Registry
Access to ConeSearch, SIA and SSA VO Services
Efficient visualization of VOTables data and description
VOTable data extraction and filtering

Upcoming versions will cover SAMP connectivity to other astronomical software, access to CDS SOAP Services, access to STIL Library and STILTS Tools via local services, as well as other local services allowing files format conversion and standard astronomical functions. Moreover, because astronomers are found to be heavy users of Python scripting language, some exploration studies are considering to provide Taverna users the possibility to add their own Jython beanshells when building Taverna workflows.

The work undertaken in the frame of the Wf4Ever project will foster the development of astronomical workflows, favoring the use of Virtual Observatory standards for interoperability among astronomical data and process.

Wf4Ever@IIPC General Assembly

2012-05-16T10:17:11Z

Sean Bechhofer appeared by invitation on a panel at the recent 2012 General Assembly of the International Internet Preservation Consortium in Washington, D.C. on May 3rd. The topic of the panel was Harvesting and Preserving the Future Web and issues discussed included the notions of replay and reproducability; the need for an understanding of decay in the context of workflows and the need for provenance information to support replay. The panel also featured representatives from the British Library, the Danish National Library, Bibliotheca Alexandrina, LANL, and the Internet Archive.

Querying Provenance of Workflow Results

2012-05-16T08:18:30Z

There exists different types of provenance which are being treated at WF4Ever project. Among all of them the provenance of workflow results records the actions which occurs at the execution of a workflow. This collected information can include inputs, intermediate results, outputs, and their execution timestamp (see definition). This information is specially relevant due to it allows the evaluation of some desirable characteristics of a RO (for example repeatibility or reproducibility).

Currently some initial data related with provenance of workflow results have been generated from different sourcing platforms(Taverna and WINGS) in order to provide a few examples which are compliant with the wfprov ontology developed at Wf4Ever as part of the RO ontology. Furthermore, the PROV-O ontology has been also used as a bridge between source ontologies and the final wfprov ontology improving its generalization capabilities of the process. These imported examples are also part of the testbed which is being currently use for validation purposes of the different funcionalities of the project.

This provenance data has been made available for testing purposes at a SPARQL endpoint and some examples queries have been created to get some of that provenance information in order to allow users feedback and for its use as part of the integrity and authenticity RO funcionalities.

NBIC Application Showcase Award for tool to run Taverna workflows from a web page

2012-05-08T18:27:08Z

Kostas Karasavvas has won the 'Application Showcase Award' at NBIC 2012*. This application makes it possible to run any (public) workfow on myExperiment directly as a Web Application without any further dependencies. It enables workflow developers to share their work with workflow-ignorant collaborators, without any need to develop some kind of user interface.

According to award committee chair and NBIC CTO Rob Hooft the prize was awarded for 'providing a real service that many people have been waiting for'. Its general high quality was appreciated and that Kostas prioritizes usefulness and stability over novelty. The award committee added that the prize is meant as an incentive for the finishing touches, such as further integratation with myExperiment.

Kostas and collaborators previously enabled the use of Taverna workflows in the server-based tool 'Galaxy', that is highly popular among genomics researchers. With the same amount of complexity as for any other tool in Galaxy, myExperiment users can include a workflow as a tool in Galaxy.

Kostas' work is the result of a NBIC-myGrid-myExperiment collaboration in the context of NBIC's BioAssist program. The project is supervised by wf4ever user Marco Roos, and driven by users associated with BioAssist and workflow designers at the Human Genetics Department of wf4ever partner LUMC**. The Application Showcase used biological workflows that also feature in wf4ever as examples.

* Netherlands Bioinformatics Centre
** Leiden University Medical Centre.

Wf4Ever in IPAW 2012

2012-04-27T18:32:28Z

Two Wf4ever papers have been accepted at the biannual IPAW workshop, which will take place in Santa Barbara, California during the week of 18-22 June 2012.

- Khalid Belhajjame, Paolo Missier, and Carole A. Goble. Detecting Duplicate Records in Scientific Workflow Results.

- Paolo Missier and Khalid Belhajjame. A PROV encoding for provenance analysis using deductive rules.

The first paper presents a new algorithm for detecting duplicates in workflow results by exploiting their provenance, whereas the second paper describes an implementation of the W3C provenance data model PROV using the DLV Datalog engine.

Wf4Ever project was presented last January 18th in "Virtual Observatory France"

2012-03-01T13:27:42Z

Wf4Ever project was presented last January 18th in "Virtual Observatory France" Annual Meeting held in Observatoire de Paris. In a special session organized by the Workflows Working Group, Jose Enrique Ruiz was invited to present the main goals and on-going work in the Wf4Ever project to the VO Workflows French community. Some discussion followed afterwards concerning community engagement of users interested in the creation, preservation and sharing of scientific workflows and research objects in astronomy, developed following a set of best practices. The migration of existing semi-automated recipes for astronomical data reduction was considered by Centre de données Astrophysique de Marseille -CeSAM, as well as the utility of migrating very specific tutorials to workflows and research objects in order to improve the use of workflows and the existing infrastructure of data. Others related efforts were presented, particularly interesting is the PDL (Parameter Description Language) initiative for the interoperability and semantic discovery of Web Services for astronomical data analysis. Wf4Ever needs to provide interoperable models, ontologies and vocabularies for the characterization of workflows, data and also processes involved in the astronomical research.This work has to be done taking into account standards provided by the IVOA, those developed in the Wf4Ever Project as well as others existing approved standards.

PDF File Presentation

Sharing Interoperable Workflows

2012-02-23T10:09:06Z

The SHIWA (SHaring Interoperable Workflows for large-scale scientific simulations on Available DCIs) project enables publicly available workflows to be used by different research communities working on different workflow systems, and to be run on multiple distributed computing infrastructures. SHIWA and Wf4Ever are complementary projects: Wf4Ever takes a holistic view of the lifecycle of the workflow object while SHIWA brings very significant functionality in workflow execution. We are pleased to announce co-operation between these projects, and members of the Wf4Ever team will be attending the e-Science Workflows event held jointly with EGI in Budapest in February 2012. A Memorandum of Understanding is under development.

Link for Shiwa https://www.shiwa-workflow.eu/about

Research objects

2012-02-23T10:06:14Z

Last month, the Wf4Ever project published v0.1 of the Research Object vocabulary. This set of OWL ontologies gave us a starting point for describing workflows, data aggregations and annotations of digital artefacts. Several Wf4Ever partners are participating in the W3C Provenance working group, which have recently published the first draft for the PROV-O standard, an OWL implementation of the draft PROV-DM model.

As a first prototype for exercising both the PROV-O and RO ontologies in combination with the scientific workflow system Taverna, we have developed a set of small utillities. This blog post is a kind of technical dip showing.

Extracting abstract workflow structures

scufl2-to-wfdesc is a tool and extension of scufl2, an API for processing Taverna workflows. As a command line tool, this can extract the workflow structure from a Taverna .t2flow workflow definition, and save this as a wfdesc description, which can be uploaded as an annotation on the aggregated t2flow in the research object.

For example, the abstract structure of a (very simple) Hello World workflow looks like this (RDF/Turtle from helloworld.wfdesc.ttl):

	@base <http://ns.taverna.org.uk/2010/workflowBundle/8781d5f4-d0ba-48a8-a1d1-14281bd8a917/workflow/Hello_World/> .
@prefix wfdesc: <http://purl.org/wf4ever/wfdesc#> .
@prefix wf4ever: <http://purl.org/wf4ever/wf4ever#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .

<> a wfdesc:Workflow , wfdesc:Description , wfdesc:Process ;
	rdfs:label "Hello_World" ;
	wfdesc:hasOutput <out/greeting> ;
	wfdesc:hasSubProcess <processor/hello/> ;
	wfdesc:hasDataLink <datalink?from=processor/hello/out/value&to=out/greeting> .

<out/greeting> a wfdesc:Output , wfdesc:Description , wfdesc:Input ;
	rdfs:label "greeting" .

<processor/hello/> a wfdesc:Process , wfdesc:Description ;
	rdfs:label "hello" ;
	wfdesc:hasOutput <processor/hello/out/value> .

<processor/hello/out/value> a wfdesc:Output , wfdesc:Description ;
	rdfs:label "value" .

<datalink?from=processor/hello/out/value&to=out/greeting> a wfdesc:DataLink ;
	wfdesc:hasSource <processor/hello/out/value> ;
	wfdesc:hasSink <out/greeting> .

wfdesc is intended as a minimal structure for describing scientific workflows from different systems. It is inspired by earlier work such as myExperiment's workflow component ontology and OPMW.

The next step for the scufl2-to-wfdesc tool is to convert it into a freestanding agent, which can submit SPARQL queries to the Research Object digital library (RODL) to find research objects which aggregate Taverna workflows, and then use the RODL REST API to upload the extracted wfdesc structure as an annotation on the workflow. This would be a stepping stone to allow other RO tools to work with workflow fragments (for instance recommendations, decay analysis and service replacement) and in particular to associate aggregated workflow results with their internal workflow provenance.

Exporting workflow provenance

Several provenance models already exists, such as OPM-V, PREMIS and SWAN, and new models such as the W3C PROV-O are under development. Wf4Ever have not currently made a deliberate choice between these, but have developed a light-weight ontology called wfprov for describing workflow provenance. This ontology is intended to be mapped to the different models, but also to the wfdesc ontology to associate workflow provenance with the abstract steps.

Extracting abstract workflow structures

For example, the abstract structure of a (very simple) Hello World workflow looks like this (RDF/Turtle from helloworld.wfdesc.ttl):

	@base <http://ns.taverna.org.uk/2010/workflowBundle/8781d5f4-d0ba-48a8-a1d1-14281bd8a917/workflow/Hello_World/> .
@prefix wfdesc: <http://purl.org/wf4ever/wfdesc#> .
@prefix wf4ever: <http://purl.org/wf4ever/wf4ever#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .

<> a wfdesc:Workflow , wfdesc:Description , wfdesc:Process ;
	rdfs:label "Hello_World" ;
	wfdesc:hasOutput <out/greeting> ;
	wfdesc:hasSubProcess <processor/hello/> ;
	wfdesc:hasDataLink <datalink?from=processor/hello/out/value&to=out/greeting> .

<out/greeting> a wfdesc:Output , wfdesc:Description , wfdesc:Input ;
	rdfs:label "greeting" .

<processor/hello/> a wfdesc:Process , wfdesc:Description ;
	rdfs:label "hello" ;
	wfdesc:hasOutput <processor/hello/out/value> .

<processor/hello/out/value> a wfdesc:Output , wfdesc:Description ;
	rdfs:label "value" .

<datalink?from=processor/hello/out/value&to=out/greeting> a wfdesc:DataLink ;
	wfdesc:hasSource <processor/hello/out/value> ;
	wfdesc:hasSink <out/greeting> .

wfdesc is intended as a minimal structure for describing scientific workflows from different systems. It is inspired by earlier work such as myExperiment's workflow component ontology and OPMW.

Exporting workflow provenance

The Third workshop on the Role of Semantic Web in Provenance Management

2011-12-28T20:28:46Z

Members of the Wf4ever team, namely Khalid Belhajjame, Jose Manual-Jomez and Jun Zhao, are co-organizing the 3rd international workshop on the Role of Semantic Web in Provenance Management. The SWPM'12 will be co-located with the ESWC conference in May 2012. It follows on from two successful editions, co-located with the International Semantic Web Conference (ISWC). The SWPM workshop has the following complementary objectives:
(1) to explore the opportunities offered by the Semantic Web technologies in the context of the management and exploitation of provenance.
(2) to explore the role of provenance in real-world Semantic Web applications.
Additionally, we will organize in this edition a panel on PROV-DM, the provenance model that is currently being developed by the W3C provenance working group and which is on the W3C recommendation track. The aim is to discuss and to identify the opportunities PROV-DM presents in leveraging semantic web applications.

For further information visit the workshop website at:

http://sites.google.com/site/swpm2012/

or contact the organizers at : swpm2012 (at) googlegroups (dot) com

Architectural Principles and Preliminary Architecture Published

2011-12-23T20:02:28Z

The Wf4Ever Architecture Task Force is designing and developing the Wf4Ever software architecture for the design and implementation of scientific workflow preservation systems, together with a reference implementation instantiating the architecture and enabling the preservation and efficient retrieval of scientific workflows across a range of domains. We are pleased to announce the publication of our design principles together with the preliminary architecture and early reference implementation. The architecture sets out to be compliant with standards like the OAIS reference model and the Linked Data initiative, to combine workflow lifecycle management, social networks and digital libraries, and to extend them with contributions from component-level research. Following a codesign methodology, we are building on an existing system, the myExperiment repository, to develop preservation capabilities that consider the complexity of scientific workflows and their related objects. Our ambition is that the design, software, services and methodologies established in the project will be applicable to other systems also. The Architecture is described in Wf4Ever Architecture - Phase I link to D1.3v1

Research Object Quality Preservation

2012-01-11T10:40:04Z

WF4Ever partners are working on how to use the provenance traces (among other data types) to support the measurement of integrity and authenticity of scientific research objects (RO). The provenance of a resource is defined as: ''a record that describes entities and processes involved in producing and delivering or otherwise influencing that resource'' (see deliverable D4.1).This type of information is going to allow the evaluation of some Information Quality (IQ) criteria which later on will be used for the definition of a RO properties (R's) which have been settled for the Integrity and Authenticity measurement.

A first prototype (screenshots)is being developed based on the open provenance model OPM and evaluating the Stability IQ dimension (see also matrixIQ-R's ) Stability is defined as the degree that a RO maintains its original characteristics throughout the time in order to be able to accomplish with its specific goals. The IQ dimension captures the trace of actions which have been done by a user over a RO and evaluates its decay degree. At present, an algorithm based on a-priori expert knowledge is being used and is expected to be improved using end-users validation (astronomy and bioinformatics domains). In near future this process will also be adapted to the wfdesc defined as the provenance model for the WF4Ever project (wfdesc ontology) .

Research Object Vocabulary Specification v0.1

2011-12-13T15:41:27Z

The first draft specification of the Research Object Vocabulary has been released as a set of interconnected ontology modules. The core of this vocabulary is the RO ontology (ro - http://purl.org/wf4ever/ro#), which specifies the core elements for describing Research Objects, such as the aggregated resources, annotations and manifest. The workflow definition ontology (wfdesc - http://purl.org/wf4ever/wfdesc#) describes an abstract workflow description structure and is meant as an upper ontology for more specific workflow definitions, which could either be hand-crafted by users ("ideal workflow template") or extracted from workflow definitions of existing workflow systems, like Taverna's .t2flow and Scufl2 formats. The workflow provenance ontology (wfprov - http://purl.org/wf4ever/wfprov#) describes provenance of workflow executions, i.e., it links workflow descriptions (wfdesc) to a provenance trace of a workflow execution. Finally, the wf4ever ontology (wf4ever - http://purl.org/wf4ever/wf4ever#) specifies extensions that are quite specific to Wf4Ever, and probably not as liable for reuse as the previous ontologies. It imports all the other ontologies, and so can also be used to get a complete picture of all the Wf4Ever ontologies. For additional information please refer to our wiki page http://www.wf4ever-project.org/wiki/display/docs/Research+Object+Vocabulary+Specification+v0.1.

Wf4Ever at Microsoft's 2011 eScience workshop and IEEE eScience Conference

2011-12-19T09:59:49Z

Prof. Dave De Roure spoke on Computational Research Objects at the Microsoft 2011 eScience Workshop (http://research.microsoft.com/en-us/events/escience2011/) and Khalid Belhajjame spoke on Fostering Scientific Workflow Preservation Through Discovery of Substitute Services, at the colocated IEEE eScience Conference.

The talk of Prof. De Roure addresses the case for Research Objects as research records for repeatable, reproducible, reusable, etc., and which describe processes (methods) for enactment/execution. They are usable by machines as well as humans and are composable with a distributed computational model.

The talk of Khalid Belhajjame presents a method that uses data links connecting inputs and outputs of service operations in existing workflow specifications to locate operations with parameters compatible with those of the missing operations. It exploits provenance traces collected from past executions of workflows to ensure that candidate substitutes perform tasks similar to those of the missing operations. The effectiveness of the proposed method has been empirically assessed.

The slides of Prof. De Roure's presentation are available at http://www.myexperiment.org/packs/235.html, and the paper of Khalid Belhajjame is available at http://repo.wf4ever-project.org/dlibra/docmetadata?id=25.

Wf4Ever at Dagstuhl: The Future of Research

2011-12-12T15:15:32Z

The Dagstuhl report is already available in http://drops.dagstuhl.de/opus/volltexte/2011/3315/. From the report, we highlight the following excerpt of Prof. Dave De Roure's talk in the Future of Research track on Wf4Ever's research objectives and synergies with myExperiment:

What happens when there are millions and millions of executable papers, sitting there and executing away...? "Executable journals" are a step towards this vision – a world of inter-related executable papers, in an altered ecosystem of scholarly publishing with new intermediaries like observatories and a new role for existing intermediaries like libraries and publishers. What will that world be like? It will help us do science-on-demand ("press this button to re-run your thesis") , and equally the papers can process new data autonomously, generating new results which in turn get processed by other papers. You'll receive an email notification when the paper you wrote five years ago is re-run with new inputs from other people's papers, and so will the people who used yours. Automated execution assists curation and indeed validation and quality checking - and whatever replaces peer review as we know it. Is this crazy or inevitable? The co-evolutionary design of the myExperiment website for sharing computational workflows gives us a glimpse into this world of executable "Research Objects", which is being further developed under the Wf4Ever project.

Scientific Workflows in Astronomy

2011-12-02T11:22:16Z

Wf4Ever project was presented at ADASS - Astronomical Data Analysis Software and Systems XXI Conference, held in Paris on November 6-10 2011. A BoF interest group meeting was organized by André Schaaff (CDS, Observatoire de Strasbourg) and Juan de Dios Santander-Vela, (ESO) in order to discuss the state of the art of scientific workflows in Astronomy, standards for their characterization, and their actual use and needs by the community. Juan de Dios Santander-Vela exposed the main goals and on-going work in the Wf4Ever project. Discussions on these topics are being raised recently by Jose Enrique Ruiz (IAA-CSIC) in IVOA - International Virtual Observatory Alliance forums. He presented the role of the Wf4Ever project in Data Curation and Preservation and Grid and Web Services sessions in the last IVOA Interoperability meeting held in held in Pune( India), on October 17-21 2011. Astronomers will soon be facing a new generation of facilities and archives dealing with huge amounts of data (ALMA, LSST, Pan-Starrs, LOFAR, SKA pathfinders,..) where scientific workflows will play an important role in the working methodology of astronomers. In this context the Wf4Ever project is defining requirements, developing models, standards and tools, in order to foster the use and contribute to the preservation of scientific workflows in Astronomy.

Carole Goble keynote at IC3K

2011-11-14T10:24:53Z

Wf4Ever PI Carole Goble recently presented a keynote talk at the 3rd International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K). The talk discussed how scientific method and scholarly communication are about facilitating "knowledge turns" -- that is, the turning of observation and hypothesis through experimentation, comparison, and analysis into new, pooled knowledge. The presentation also considered questions relating to reproducability -- a key topic for Wf4Ever.

In other Wf4Ever related news, a poster discussing the Wf4Ever Research Object model has been accepted at the SWAT4LS workshop in December.

Work is also underway on the definition of an OWL ontology to be used in demonstrator applications being developed for the first year project review.

Prof. Dave De Roure presented at the Microsoft Research workflow last week

2011-11-02T21:07:33Z

See Dave's blog entry: http://blogs.nature.com/eresearch/2011/10/30/a-gathering-storm-of-scholarly-transformation

A paper named "A New Approach for Publishing Workflows: Abstractions, Standards, and Linked Data" by Garijo and Gil accepted in WORKS11

2011-10-21T19:41:26Z

A paper named "A New Approach for Publishing Workflows: Abstractions, Standards, and Linked Data” by Garijo and Gil has been accepted in WORKS11, to be held on November 14th in Seattle within SC 2011.

The paper is the result of a collaboration between UPM, ISI and the UCSD. It describes (1) how to add an abstract workflow layer for better workflow understanding and implementation, (2) how to publish workflows according to the OPM standard to separate them from any execution platform and (3) how to make the workflows available as Linked Data, so other systems can share and reuse them.

The test target for the paper has been the TB-Drugome workflow by Kinnings et al. This workflow represents a two year experiment to find new drugs that cure the Tuberculosis based on approved ones, and has made a great impact in the biology domain. The workflow presents a series of challenges for its reproducibility and decay highly related to wf4Ever, and the abstract workflow proposed in the paper can be added as part of a RO.

Wf4Ever to be presented at iPRES 2011

2011-10-20T13:04:08Z

Wf4Ever's vision about workflow preservation will be presented at the International Conference on Preservation of Digital Objects (IPRES) in Singapore, November 2011.The paper, titled "Towards the Preservation of Scientific Workflows" by DeRoure et al. makes a strong emphasis on the relevance of decay analysis and provenance for workflow preservation.

Wf4Ever to be presented in the 7th IEEE e-Science conference

2011-10-20T13:07:12Z

A Wf4Ever paper titled as "Fostering Scientific Workflow Preservation Through Discovery of Substitute Services" by Belhajjame et al. has been accepted in this year's IEEE e-Science conference, to be held between 5th-8th December, in Stockholm Sweden. Details about the paper can be found here.