Repository Operations

As a repository, ICPSR adheres to operational standards that demonstrate it is organizationally, procedurally, and technologically sound as a trustworthy data custodian. This page describes the processes and standards we take related to acquiring, enhancing, describing, preserving, and disseminating digital social science data.

ICPSR processes are based on the Reference Model for an Open Archival Information System (OAIS), an ISO standard that provides the functional framework for sustaining digital objects in managed repositories. The following image provides a detailed flowchart of ICPSR’s processes with respect to the OAIS Reference Model.

ICPSR’s Processes Mapped to the OAIS Reference Model

Ingest

At ICPSR, all data and metadata (the Submission Information Packages, or SIPs) are deposited through a secure electronic deposit form. Depositors authenticate using the ICPSR Researcher Passport registration and authentication system.

Files uploaded through this system are assigned unique deposit IDs and transferred to a secure area for processing. Depositors must sign a deposit agreement, attesting they “have all rights needed to make this Data Collection publicly available through ICPSR.” ICPSR confirms receipt of all materials.

To ensure authenticity and integrity, ICPSR applies checksums at every step of the data life cycle, beginning with the original deposit. Checksums act like digital fingerprints for files: they allow ICPSR to check for corruption or loss during transfer and storage.

The ICPSR Collection Development Policy defines the criteria and priorities that guide staff in selecting data for curation and long-term archiving.

Curation

As part of Ingest, ICPSR staff perform quality assurance checks to verify that deposits, including data and metadata, are complete, valid, and meet required standards. Selected data are enhanced—“curated”—with meaningful information to make it complete, self-explanatory, and usable for future researchers. These files are transformed into Archival Information Packages (AIPs) that include the data, associated documentation, and preservation metadata. Descriptive metadata are extracted and enhanced to support discovery in ICPSR’s catalog.

The curation activities applied to a data collection follow one of several curation levels (pdf). These activities may include:

  • Review data for confidentiality issues
  • Generate multiple data formats for dissemination and preservation
  • Create documentation compliant with the DDI specification
  • Recode variables to address confidentiality concerns
  • Check for undocumented/out-of-range codes
  • Add question text to variables
  • Create variable labels
  • Create value labels
  • Designate missing values
  • Standardize missing values
  • Identify and address foreign language characters
  • Adjust format widths
  • Create a metadata record
  • Check for consistency and skip patterns
  • Make online analysis version with question text
  • Gather citations to related publications for the Bibliography of Data-Related Literature

Staff who curate and manage data follow specific instructions, which include the documentation of all data transformations. Any changes made to data values during the curation workflow are recorded using syntax and processing history files, which contain commands for adjustments like adding variable labels, changing value labels, revising missing value designations, and/or inserting recodes. Processing history syntax files furthermore enable tracking and replication of data changes.

Metadata

Metadata (i.e., information about our data collections that help others discover, understand, and use them) are essential for maximizing the usefulness of data. Comprehensive metadata standardizes how the data are described, enables a deeper comprehension of a dataset, facilitates data searches by variables, and offers a variety of display options on the web.

ICPSR creates metadata primarily from information supplied by data depositors. Whenever possible, ICPSR uses existing metadata standards to describe our collections. ICPSR’s core study-level metadata are based on the Data Documentation Initiative (DDI), a widely adopted metadata standard for the social and behavioral sciences, which ICPSR helped establish and now leads.

Metadata creation, enhancement, and quality review is a team effort and involves staff from across all of ICPSR. The Metadata & Preservation unit checks for adherence to standards and advises on all metadata-related activities during the data curation lifecycle.

The ICPSR Metadata Documentation Portal is the official source of information for ICPSR metadata, including for human-readable descriptions of supported data elements, usage notes, examples, and a machine-actionable JSON Schema version.  For information about exporting metadata records, see Accessing Metadata Records.

Preservation

ICPSR is committed to digital preservation — the proactive and ongoing management of digital content to lengthen the lifespan and mitigate against loss. ICPSR preserves its data resources for the long-term, guarding against deterioration, accidental loss, and digital obsolescence. ICPSR has a decades-long track record of reliably storing research data. A key part of ICPSR’s mission is to preserve the data we steward, including the data archived at our founding in 1962.

ICPSR’s preservation efforts are outlined in the Digital Preservation Policy Framework, which informs processes across the organization, as well as covers organizational viability and financial sustainability to ensure the persistence of its collections. In addition, ICPSR maintains an ongoing technology responsiveness program to monitor file formats for obsolescence and vulnerabilities. ICPSR commits to selecting and preserving the content, structure, and context of certain file formats through file format normalization and migration.

Data are securely stored in multiple formats and locations, with evolving strategies to ensure resilience over time. Although ICPSR permanently archives materials, the ICPSR Deaccession Policy allows ICPSR to discontinue the distribution of data collections when necessary.

Dissemination

ICPSR disseminates data to researchers, students, policymakers, and journalists around the world based on its Access Policy Framework. Once published, ICPSR ensures its data collections are well-described, discoverable, and uniquely identified (more details).

Further Information

  • Inter-university Consortium for Political and Social Research (ICPSR), 2024, “2027-08-22 – Inter-university Consortium for Political and Social Research (ICPSR) – CoreTrustSeal Requirements 2023-2025”, https://doi.org/10.34894/WCWZU0, DataverseNL, V1
  • Mary Vardigan and Cole Whiteman, ‘ICPSR meets OAIS: applying the OAIS reference model to the social science archive context’, Archival Science, 7:1 (2007), pp. 73-87. https://hdl.handle.net/2027.42/60440