OSF Pigeon

A preservation service that mirrors public Open Science Framework registrations — files, metadata, and archival context — to Internet Archive.

2.2 TBof research data archived for long-term preservation.

1.5 millionindividual files preserved through the archival pipeline.

Still growingas public OSF registrations continue to be mirrored to Internet Archive.

C:\OSF\PIGEON> STATUS
ARCHIVE_TARGET = INTERNET_ARCHIVE
DATA_ARCHIVED = 2.2_TERABYTES
FILES_ARCHIVED = 1.5_MILLION
MISSION = PRESERVE_PUBLIC_OSF_REGISTRATIONS
STATUS = STILL_GROWING

What it does

When an OSF registration becomes public, Pigeon packages the registration's files and metadata and sends a preservation copy to Internet Archive. The archive includes the research files plus machine-readable registration, contributor, schema, log, wiki, and DOI metadata so the record is useful outside OSF itself.

My role

I built OSF Pigeon while working at the Center for Open Science and wrote the great majority of the service's source code. It is one of my favorite examples of small, purpose-built infrastructure doing a very large amount of durable work.

Source, archive, and metrics

The source is public, and the resulting preservation collection can be browsed directly at Internet Archive.

View Source Code Browse the Archive Back to Crazy Projects

OSF Pigeon was built for the Center for Open Science. This page is my personal project write-up and impact summary.