CyberInfrastructure Partnership Resources

Data Services and Documentation

In addition to running computational systems, CIP sites provide resources for short-term and long-term data storage. Higher-level data management and data collection services and expertise are also available. The table below contains a technical summary of these data resources and services and associated documentation.

Researchers with TeraGrid or Core computing allocations are eligible to use the general storage resources described below. Users with specialized requests should contact the sites individually about their data service needs in the areas of data management, data collection hosting, and unique data storage requirements. Eligible PIs who do not have a TeraGrid or Core computing allocation can request a data allocation for presented data services and resources.

For more information about accessing SDSC data services and resources, see DataCentral at SDSC or e-mail datacentral-allocations@sdsc.edu to contact one of SDSC's Data Central experts. For additional information about NCSA data capabilities, send email to help@ncsa.uiuc.eda with your request and a NCSA staff member will contact you.

Site
Data Service
Software & Documentation
Supporting Hardware
SDSC Database Hosting and Management DB2 Management System
DB2 @SDSC Basics
Using DB2 on DataStar
Using DB2 on TeraGrid
IBM DB2 Info

DB2 iMiner
IBM iMiner Info

MySQL
MySQL

Oracle 10g
DataStar Power 4 node dedicated to DB users
  • 128GB P690 with 32 processors @1.7GHz
  • Gigabit Ethernet to SAN, GPFS and HPSS
  • 12 TB attached and connected to SAN for fast data movement and archiving
  • Data Hosting GPFS and Sun SAM-QFS Info

    Areas dedicated for:
  • Data Collections
  • Data Parking - multi-month large data analysis area
  • Over 1 Petabyte of SAN online disk storage

    Collections:
  • 130 TB Fibre Channel
  • Over 200 TB SATA additional disk available in October


  • Parking:
  • 450 TB SATA disk
  • Long-Term Storage HPSS
    SDSC HPSS User Guide
    HSI User Guide

    Sun SAM-QFS HSM

    SDSC SAM-QFS User Guide
  • 6 Petabytes tape storage
  • Front end servers:
  • 2 HPSS dedicated P690s nodes with 32 processors and 128 MB memory each, 8.way Gigabit Ethernet each
  • Dual Federation switch interconnect to DataStar
  • 30TB disk cache
  • Disk Storage Sun SAM-QFS HSM
    SDSC SAM-QFS User Guide
     
    Collection Management Storage Resource Broker
    SRB User Guide
    Data Movers and Metadata catalog HW:
  • 2 Sun v240 2 CPU & 8 GB RAM each
  • 2 Sun v40z 4 CPU &16 GB RAM each All GigE connectivity
  • 2 Sun v490; 4 CPU & 16 GB RAM each, two 10 Gigbit NICs
  • NCSA Database Hosting and Management Oracle 10g

    MySQL
    SGI Altix
  • 8 processor
  • 12 GB memory
  • 4 Gb to SAN
  • 30 TB dedicated disk attached via SAN
  • Data Hosting - variety of technologies available for community datasets NFS

    Lustre


    IBRIX
    Storage-Area Network
  • 254 TB online disk

    30 + Linux-based Storage Servers
  • Long-Term Storage EMC/Legato DiskXtender/UniTree Mass Storage System
    NCSA UniTree documentation

    Man pages for UniTree transfer commands

    DiskXtender online documentation
    Mass Storage System Front End Servers
  • Two SGI Origin 3900s, each with 16 processors and 12 GB memory

    35 TB disk cache

    Tape Library
  • 3 PB tape storage
  • 38 IBM LT02 Fibre Channel tape drives
  • 1.3 GB aggregate throughput