Sharing Large Scientific Datasets with LoCI

Joe St Sauver, Ph.D.
Director, User Services and Network Applications
joe@uoregon.edu

Files that are even just tens or hundreds of megabytes in size will often be too big to email, and datasets don't have to get very much larger to be too large to easily store and transfer via the Internet.

Although you could mail or FedEx CD-ROMs or DVDs to remote participants, that's still a relatively slow and expensive proposition, particularly if you're frequently sharing updated datasets with a large number of remote sites.

So…what can you do? You now have an online alternative: logistical networking, courtesy of the NSF-sponsored LoCI project. Logistical networking makes it possible to temporarily share large data files using a pool of publicly available network storage deployed over a worldwide network of storage "depots."

In October 2003, I wrote a column for "Syllabus: Technology for Higher Education," talking about the benefits of temporarily sharing large files using the University of Tennessee's NFS-funded LoCI "Logistical Networking" project. That article is still available online at http://www.syllabus.com/news_issue.asp?id=153&IssueDate=10/30/2003 and provides a nice quick start for some key LoCI-related concepts. (I won't rehash the information covered in that article here, except to say that we'd encourage you to review it for some fundamental background on the LoCI project before continuing with the rest of this Computing News article.)

Since the fall of 2003 much has changed for the better. I'm happy to report that the UO is now locally hosting an NSF-funded LoCI node.

Getting Set Up to Use the LoRS Visual Tools

While casual users can simply use the Java based "LoDN" tool from the LoCI website to upload and download files, if you regularly share large files I'd encourage you to take the time to install the LoRS Visual Tools (LoRS VT) package for your PC or Mac (there's also a LoRS Source Development Package available for those who'd rather build from source for their favorite Unix/Linux operating system). The PC and Mac LoRS Visual Tools packages are available as "featured downloads" from the LoCI Project Home Page at http://loci.cs.utk.edu/

Once you've downloaded and installed the LoRS Visual Tools, then what? As with most programs, the best thing to do is to take the time to go through the user's manual (LoRS VT includes its fine 46-page manual as a PDF document with the program). You'll be walked through configuring your copy of LoRS VT, uploading files, downloading files, and so forth. In no time at all you should be uploading, downloading, and sharing large scientific datasets.

Tell Us What You Think

If you try using the LoCI project to share large files, we're eager to hear what you think.

Did the installation of the LoRS Visual Tools go smoothly for you? Does it work well for you and your collaborators? Did you run into problems? We'd love to know!